← Initiatives
Platform · EducationIn Development

Chalkless

Chalkless analyzes recordings of board-based mathematics lectures and labels each segment by the role it plays in the lesson — topic, definition, example, key concept, or transition. It pairs lightweight per-segment inference with a deterministic pass that enforces how lectures tend to be ordered over time. The aim is an interpretable, label-free way to recover lecture structure (a problem usually handled with models trained on annotated data). Results so far are qualitative and preliminary, drawn from inspecting real lectures rather than a labeled benchmark.

Status
In preparation · Venue TBD
Track / domain
Research · Multimodal systems · Educational AI · Board-based mathematics lectures
Approach
Rule-based stabilization (interpretable, label-free)
Team
ORIA Research team
  • Lecture analysis
  • Multimodal
  • Sequence structure
  • Temporal rules
  • Educational AI

Paper “Local Classification is Insufficient: Recovering Lecture Structure Through Temporal Constraint Enforcement” — research paper, in preparation; preprint available on request via Apply.

Problem

Why local classification is insufficient

A short clip from a math lecture rarely carries its own meaning. The same equation can be a definition, a line in a worked example, or the key identity — what separates the roles is the segment’s place in the lesson.

Example: the Fourier coefficient formula aₙ = (1/π) ∫ f(x) cos(nx) dx and the orthogonality fact ∫ sin(mx) sin(nx) dx = 0 are both integrals of products of trig functions; a one-segment-at-a-time classifier sees nearly the same object, though one is a definition and the other is the idea that makes the definition work. Recovering lecture structure is therefore a sequence problem.

One lecture, five roles

A Fourier-series lecture as five ordered segments.

  1. Topicf(x + 2π) = f(x)Opens the lecture, names the subject (cue: a written, underlined title).
  2. Definitionf(x) = a₀/2 + Σ (aₙ cos nx + bₙ sin nx)Introduces the object and its coefficients (cue: “we define”).
  3. Examplef(x) = (4/π) Σ sin((2k+1)x)/(2k+1)Works a concrete case (the square wave).
  4. Key concept∫ sin(mx) sin(nx) dx = 0 (m ≠ n)Orthogonality, the idea that makes the coefficient formula work (cue: often said in passing).
  5. Transition(1/π) ∫ |f(x)|² dx = a₀²/2 + Σ (aₙ² + bₙ²)Hands off to the next lecture (Parseval’s identity).
Approach

A two-stage pipeline

  1. Input — audio + video of a board-based lecture, taken as-is (no transcripts, no labels, no manual prep).
  2. Local inference — speech transcription + a board-activity signal from the video + a lightweight model propose a role for each short segment.
  3. Stabilization — deterministic temporal rules reconcile the noisy per-segment guesses into one coherent sequence.
  4. Output — timestamped roles across the lecture (topic, definition, example, key concept, transition).

Stabilization rules (hand-written, not learned)

Hand-coding keeps the system legible and removes the need for labeled lectures. (No claim to beat a trained CRF/neural sequence model on accuracy — the value is interpretability and running without annotated data.)

  • Lectures usually open with a topic.
  • Examples run in continuous spans.
  • A key concept commonly follows its motivating example.
  • Frequent back-and-forth role switching is usually inference noise.
Preliminary observations

Qualitative, not measured

Applying the temporal rules appears to produce a more coherent sequence than per-segment guesses alone; the key-concept role seems hardest to identify locally and appears to need surrounding context.

Limitations

Limitations

  • Narrow scope (board-based math only).
  • Qualitative evidence (no labeled benchmark yet).
  • Hand-built rules.
  • Not yet validated against trained baselines.