Your Session Recordings Have No Speaker Metadata Because Nobody Captured It at Load-In
Six weeks after the event, someone opens the archive and finds a 47-minute recording, a transcript with no names, and a support ticket asking who said the thing about the migration budget. The recording is fine. The transcript is fine. What is missing is a speaker-identity record that should have existed before the room opened.
This is not a transcription problem. Diarization tools will happily split a recording into anonymous voice clusters. What they cannot do is tell you that cluster 3 is the person who registered as the moderator, or that cluster 1 is a panelist whose display name changed between the rehearsal and the live session. That mapping is a schema decision, and it has to be made at load-in.
This article is for the post-event content producer or archive owner who inherits the recordings. It proposes a speaker-identity table as the source of truth, a WebVTT emission rule that turns that table into searchable, navigable captions, and a load-in checklist that makes the whole thing reversible.
What the formats actually give you
It helps to be precise about what the file formats carry, because the gap is often smaller than people assume and the fix is often cheaper than a new tool.
WebVTT is the most useful target. The specification describes it as a format for captions and subtitles, but it also states that “beyond captioning and subtitling, WebVTT can be used for time-aligned metadata, typically in use for delivering name-value pairs in cues” and that it can deliver chapters for navigation around an audio or video file (W3C WebVTT). Two features matter here:
- Voice spans. A cue can name its speaker with a tag like
<v Nadia Rook>. The specification’s own example notes that “each cue says who is talking using voice spans,” and style sheets can target those spans by voice name.
- NOTE blocks. Comments in a WebVTT file are blocks that start with
NOTE. The EmacsConf captioning guide uses this directly: “An easy way to mark chapters is to add a NOTE Chapter heading before the subtitle that starts the chapter” (EmacsConf captioning tips). The same guide suggests that with multiple speakers, a captioning workflow can indicate switches with a [speaker-name]: tag.
So a single WebVTT file can carry both per-cue speaker attribution and chapter-level navigation. That is enough to make an archive searchable by person and navigable by topic without inventing a second format.
What the formats do not give you is the identity itself. An HLS Media Playlist declares media segments with EXTINF durations and a target duration (RFC 8216); it has no speaker field. The YouTube Data API’s video resource has a snippet object with title, description, category, and a tags list (YouTube Data API), but tags are free-text keywords, not a structured speaker record. None of these formats will recover a name you did not capture.
The failure mode of retrofitting
The common retrofit is to run diarization after the event and then ask someone to listen through and assign names. That works, slowly, and it produces a result that is hard to audit.
A worked example of the separation is worth studying. The PA Webinar project’s privacy documentation describes its post-production pipeline this way: “pyannote diarization groups the voices of a single recording into anonymous clusters (SPEAKER_00, SPEAKER_01, …). The embeddings it computes stay in the worker’s memory and scratch volume, and they are gone when the job’s pod ends. The output is only a label per transcript segment. Names come from people, who map labels in the administration area, or from the speaking timeline. They never come from the voice” (PA Webinar recordings and AI documentation).
That is the right architecture, and it is also the argument for moving the naming step upstream. If names come from people, then the people should be named before the recording exists, not reconstructed from memory weeks later. The diarization step becomes a clustering step that matches anonymous segments to an existing speaker record. The human work shrinks from “listen and identify” to “confirm the mapping.”
The same documentation shows a second failure mode worth naming. In its per-participant recording path, each audio file is labeled with the display name the person used in the conference, and the recorder writes a manifest next to the audio files that “holds the same names in plain text.” The database field is encrypted; the sidecar manifest is not. If your speaker names live only in a plain-text sidecar, you have an integrity and privacy problem the moment that directory is copied, backed up, or shared with an editor. The schema below treats the encrypted store as authoritative and the manifest as a derived, purgeable artifact.
The speaker-identity table
The artifact is a table with one row per speaker per session. It is small enough to maintain by hand and structured enough to drive emission. The fields below are a proposal, not a standard; they are chosen because each one answers a question that comes up during post-event work.
| Field |
Type |
Why it exists |
speaker_id |
stable string, e.g. spk_2026_014 |
Keys the record. Never reuse a display name as a key; display names change mid-event. |
display_name |
string as shown in the room |
What appears in the caption. May differ from legal name. |
role |
moderator / speaker / panelist / host |
Drives chapter labels and archive filtering. |
session_id |
string |
Binds the speaker to one recording. A person can appear in several sessions with different roles. |
segment_start, segment_end |
seconds, media time |
The time range in which this speaker is attributed. Empty means “present but not yet attributed.” |
attribution_consent |
yes / no / anonymous |
Separate from recording consent. A person may consent to being recorded but not to being named. |
consent_text_version |
string or date |
Which version of the consent language the person agreed to. Matters when the language changes. |
source |
registration / waiting-room / manual |
Where the name came from. Makes the record auditable. |
captured_at |
ISO 8601 timestamp |
When the row was created or last confirmed. |
Two design choices are worth defending. First, speaker_id is separate from display_name because display names are not stable. A hypothetical but common case: a moderator joins the rehearsal as “N. Rook” and the live session as “Nadia Rook (she/her).” If the schema keys on the display name, you get two speaker records and a broken chapter. If it keys on speaker_id, you get one record with a name change you can log.
Second, attribution_consent is separate from recording consent. The PA Webinar documentation is explicit that its transcription consent is “separate from the recording consent, optional, and asked of everyone, moderators and speakers included; refusing it does not keep anyone out.” That separation is good practice regardless of your jurisdiction, because it lets a speaker appear in the archive as “Panelist 2” without blocking the recording.
Load-in checklist
The table is only useful if it is populated before the room opens. The archiving guidance from the Archiving for the Future course is blunt about timing: “The best practice is to create the metadata about each file at the same time that you create the file; this is the most reliable way to ensure that the metadata gets documented” (Archiving for the Future, Step 5). The same lesson lists the descriptive fields that matter for a recording session: participants and their roles, date and location, languages spoken, and the context and content of the file.
Translated to a conference load-in, the sequence is:
- Export the roster 24 hours before doors. Include registration name, display name, role, and session assignments. This is the seed for the table.
- Assign
speaker_id values. One per person per event, not per session. A person speaking in two sessions gets one ID and two rows.
- Capture attribution consent separately. Record the answer and the version of the consent text. If the answer is “anonymous,” the row stays but the emission step will use a role label instead of a name.
- Reconcile at the waiting room. Display names often change between registration and the live room. Update
display_name and log the change; do not create a new row.
- Timestamp the mapping. Set
captured_at when each row is confirmed. This is what lets you answer “when did we know this?” during an audit.
- Freeze the table before the session starts. Additions during the session are allowed, but they should be marked as manual and reviewed before emission.
If your event has a room-to-chat handoff, the same roster should drive the chat moderation list. The handoff failure mode is covered in Why Hybrid Events Fall Apart at the Room-to-Chat Handoff, and the roster is the shared artifact that prevents both problems.
The emission rule
With the table populated, emission is a deterministic transformation. The rule has three parts.
1. Attributed speech becomes a voice span
For every transcript segment whose time range falls inside a speaker’s segment_start–segment_end and whose attribution_consent is yes, emit a cue with a voice span:
WEBVTT
00:12:04.000 --> 00:12:11.500
<v Nadia Rook>The migration budget question is the one I want to answer first.
The name in the span is the display_name, not the speaker_id. The ID stays in the table and in any sidecar index; the caption shows what the person is called.
2. Unattributed speech stays anonymous
If attribution_consent is no or anonymous, emit the cue without a voice span, or with a role label if your player supports it:
00:14:02.000 --> 00:14:09.000
Panelist 2: I'd add that the vendor contract has a 30-day notice clause.
Do not guess. A wrong name in a caption is worse than no name, because it is quotable and hard to retract.
3. Chapters become NOTE blocks
For each topic change or agenda item, emit a NOTE block before the first cue of the chapter:
NOTE Chapter: Budget and vendor contracts
00:11:58.000 --> 00:12:04.000
Moderator: Let's move to the budget question.
The EmacsConf guide uses exactly this pattern and notes that chapter markers can be extracted programmatically from the NOTE comments. That means the same file serves playback captions and archive navigation, and a downstream indexer can parse the chapters without a second data source.
Why this is reversible
The table is the source of truth; the WebVTT file is a derived output. That ordering is what makes the fix reversible.
If a name is corrected, or a consent changes from anonymous to attributed, you re-run the emission step against the same table and the same transcript. You do not re-record, re-transcribe, or re-edit the video. The cost is compute and a review pass, not a new capture.
Two things break that property, and both are avoidable:
- Names stored only in a sidecar manifest. If the plain-text manifest is the only place a name exists, a correction means editing files in place, and a purge means losing the name entirely. Keep the authoritative name in the encrypted store and treat the manifest as a derived artifact with its own retention rule.
- Names baked into the transcript. If the transcript is edited by hand to insert names, re-emission means re-editing. Keep the transcript name-free and let the table supply attribution at emission time.
The PA Webinar documentation shows the retention pattern that supports this: per-participant audio is deleted after transcription by default, while the transcript’s attribution refers to a database row that stays. The audio is intermediate; the attribution is the durable record.
Copyable artifact: speaker-identity template and emission rule sheet
For the post-event content producer. Copy the CSV below into your archive’s working directory and fill it at load-in. The JSON block is the same schema for teams that prefer a structured file.
speaker_id,display_name,role,session_id,segment_start,segment_end,attribution_consent,consent_text_version,source,captured_at
spk_2026_001,Nadia Rook,moderator,sess_014,0,2700,yes,2026-03-01,registration,2026-04-10T09:00:00Z
spk_2026_002,Sam Okafor,speaker,sess_014,312,1480,yes,2026-03-01,registration,2026-04-10T09:02:00Z
spk_2026_003,,panelist,sess_014,1500,2400,anonymous,2026-03-01,waiting-room,2026-04-10T09:14:00Z
{
"schema_version": "1.0",
"event_id": "evt_2026_spring",
"speakers": [
{
"speaker_id": "spk_2026_001",
"display_name": "Nadia Rook",
"role": "moderator",
"session_id": "sess_014",
"segments": [{"start": 0, "end": 2700}],
"attribution_consent": "yes",
"consent_text_version": "2026-03-01",
"source": "registration",
"captured_at": "2026-04-10T09:00:00Z"
}
]
}
Emission rule sheet. Apply in order; stop and review if any step produces a name that is not in the table.
- Load the speaker table and the transcript. Reject any row with an empty
speaker_id or a captured_at later than the session start unless source is manual.
- For each transcript segment, find the row where
segment_start ≤ segment start and segment_end ≥ segment end. If no row matches, emit the cue without a voice span.
- If the matched row has
attribution_consent = yes, emit <v {display_name}>. If anonymous, emit the role label. If no, emit no label.
- For each chapter boundary in the agenda, emit a
NOTE Chapter: {title} block before the first cue of the chapter.
- Validate the output: every voice span name must appear in the table; every chapter NOTE must have a following cue; no cue may contain two voice spans.
- Store the emitted WebVTT alongside the table and record the table’s version hash in the archive metadata.
The validation step is the one that catches the common error: a display name that was updated in the room but not in the table. If the validator fails, fix the table and re-emit. That is the whole point of keeping the table upstream.
Questions that come up
Can we just use the platform’s automatic speaker labels?
Use them as clustering input, not as identity. The PA Webinar documentation is a useful reference here: its diarization output is anonymous clusters, and names are supplied by people. If your platform produces labels like “Speaker 1,” treat them as cluster IDs and map them to your table before emission.
What if a speaker changes their display name mid-session?
Update display_name on the existing row and log the change in captured_at. Do not create a second row. The voice span will use the current name for the whole session, which is usually what you want for an archive; if you need the historical name, keep a change log alongside the table.
Does this work for hybrid events with a room and a remote audience?
Yes, and the roster is the shared artifact. The room-to-chat handoff problem described in Why Hybrid Events Fall Apart at the Room-to-Chat Handoff is easier to diagnose when the same speaker table drives both the caption emission and the chat moderation list.
What about speakers who decline attribution?
Keep the row, set attribution_consent to anonymous, and emit a role label. The recording and transcript remain usable; the archive simply does not name the person. This is the same separation the PA Webinar documentation describes between recording consent and transcription consent.
How long should the table be kept?
At least as long as the derived outputs it produced. If the WebVTT file is retained for the archive’s lifetime, the table that generated it should be retained too, or the attribution becomes unverifiable. The PA Webinar pattern of deleting intermediate audio while keeping the attribution row is a reasonable model.
What to do this week
Pick one recording from your last event. Open its transcript and count how many segments have a speaker name. If the answer is zero, you have the symptom. The fix is not a new tool; it is a table with nine columns and a load-in step that takes about 15 minutes per session.
Start with the next event’s roster export. Assign IDs, capture attribution consent separately, and freeze the table before doors. Then emit one WebVTT file with voice spans and NOTE chapters, validate it against the table, and compare it to what you would have produced by hand. If the validator passes and the chapters line up, you have a reversible pipeline. If it fails, you have found the missing field before the archive did.