Storing and Indexing Meeting Transcripts for Semantic Search

Storing and Indexing Meeting Transcripts for Semantic Search.

Why meeting transcripts are hard to search

Building semantic search over a meeting archive is a chain of decisions, chunking strategy, embedding model choice, storage design, and hybrid retrieval, and each link in that chain changes what the final system can and can't answer. Get one of them wrong, especially early, and no amount of engineering downstream fixes it.

The pressure behind this problem is not abstract. Microsoft's Work Trend Index found that 80% of employees and leaders say they don't have enough time or energy to get their actual work done, and executives report spending up to 23 hours a week sitting in meetings Microsoft 2025 Work Trend Index azeusconvene.com. A Harvard Business Review analysis found that 71% of senior managers call meetings unproductive and inefficient Microsoft 2025 Work Trend Index azeusconvene.com. That's a workforce telling researchers, repeatedly, that a huge chunk of the workweek produces record but not progress. It's a workforce telling researchers, repeatedly, that a huge chunk of the workweek produces record but not progress.

So companies record everything, and now there's more of it than anyone can watch or read. Google said its "Take Notes for Me" feature was used by more than 110 million attendees in a single month Metrigy. Metrigy polled 1,100 companies and found 42% plan to roll out AI meeting assistants within the next year. Every one of those rollouts produces transcripts, and transcripts pile up into archives that keyword search simply cannot navigate. A plain text search for "database migration" finds neither "we're moving off the legacy system" nor "the schema refactor" Metrigy.

Semantic search is the fix for that gap, but calling it a single tool undersells what's actually required. It's a pipeline: audio gets captured, structured, chunked, embedded, indexed, and only then retrieved. What follows walks through that pipeline stage by stage, with the tradeoffs stated rather than smoothed over, because "production-ready" is a specific, checkable standard, not a marketing phrase.

What comes out of a speech-to-text pipeline

Everything downstream depends on the accuracy of the transcription itself, and accuracy is not uniform across platforms. Zoom's transcription leads at 99.05% accuracy against Webex's 98.71%, which sounds like a rounding error until you notice it's a 27% gap in the error rate, and that gap compounds across a corpus of thousands of meetings azeusconvene.com LEANN: A Low-Storage Vector Index. A small accuracy difference at the word level becomes a real difference in how many queries return the right answer once you're searching tens of thousands of hours of recordings.

Beyond getting the words right, a transcript needs diarization, the process of figuring out who spoke when. Without it, there's no way to ask a system to filter by participant, and speaker-scoped questions become impossible. Two current approaches show what "good" looks like at a technical level. MOSS-Transcribe-Diarize, released in July 2026, runs single-pass inference on recordings up to 90 minutes long, covers more than 50 languages, and assigns anonymous speaker labels; it took first place in the 2nd MLC-SLM Challenge at INTERSPEECH 2026, a competition spanning 14 languages huggingface.co dev.to LEANN: A Low-Storage Vector Index.

Diarization on its own only gets you "Speaker 2 said this huggingface.co dev.to LEANN: A Low-Storage Vector Index." Turning that into "Alice said this" is a separate job, one that needs an identity-linking layer, whether that's enrollment audio recorded ahead of time, a roster pulled from the meeting invite, or somebody manually mapping labels to names. Skip that step and every downstream query about a specific person has to work around anonymous labels instead of names.

STT APIs make this concrete. Deepgram, for instance, returns utterances, diarization, and formatting as structured JSON. That output is raw material. It still has to be normalized, enriched, and pushed into a schema before it's useful for search. And this is really where the ceiling on the whole system gets set: a transcript with weak ASR accuracy or missing diarization drags down every layer built on top of it, no matter how well those later layers are designed. IBM Granite Speech 4.1 Plus offers native diarization rather than post-processing, word-level timestamps for every token, multi-speaker support with accuracy degrading beyond roughly 4–7 speakers, and long-form audio handling.

Designing the transcript schema: what to store beyond the words

The basic unit to build around is the speaker turn, the utterance. Before anything else happens, raw ASR output should get normalized into discrete turns, because that's the structure every later stage (chunking, embedding, retrieval) will assume exists.

Each utterance needs more than just its text attached to it.

Two different storage layers do two different jobs, and conflating them causes problems later. Metadata (who spoke, when, what topic) belongs in a relational store, or in relational columns inside a hybrid database, while the embedding of the utterance text belongs in the vector index; a real query combines both.

None of this is a new idea, even if vector databases are relatively new tools for it. Cisco's own patents in this space (US 10,942,953 and US 10,860,797) describe indexing terms along with their locations and confidence scores, then mapping those terms into categories like Action Items, Commitments, Points of Contention, and Dates and Timeframes. That same categorical thinking, indexing terms into structured buckets rather than treating a transcript as one long undifferentiated blob of text, applies just as well when you're deciding what metadata fields belong in a modern vector-based system. If enterprise compliance is part of the mandate, plan for it early: current-generation transcription tools increasingly need structured fields for tracking whether action items got resolved and for producing audit trails that satisfy compliance requirements, and retrofitting a schema to support that after the fact is a lot more painful than building it in from the start.

Done well, this schema work pays off in a very specific way. A question like "what did Alice say about the budget" turns into a metadata filter (speaker equals Alice) combined with a much narrower semantic search rather than a search across the entire corpus, making it both faster and considerably more precise. Among the NLP enrichment fields worth storing are NER output covering people, organizations, dates, and dollar amounts mentioned, topic model labels, action item flags, and sentiment or contention scores.

Chunking strategies for transcript content

It's tempting to assume that with long-context language models now handling hundreds of thousands of tokens, chunking is a solved problem you can skip. The evidence says otherwise. Chroma's context rot research tested 18 models, including GPT-4.1, Claude 4, and Gemini 2.5, and found that retrieval performance degrades as context length grows, even on tasks that should be straightforward. Dumping an entire transcript into a prompt is inefficient. It's distracting to the model, and the results show it.

So chunking still matters, and the tradeoff at its center is simple to state and hard to solve. Large chunks risk pulling in irrelevant content that muddies the model's answer, while small chunks risk stripping away the context a model needs to answer correctly, sometimes causing outright hallucination. Every chunking strategy is an attempt to sit somewhere sensible between those two failure modes.

The benchmark evidence is more scattered than any single vendor's blog post would suggest, and it's worth looking at closely rather than picking a favorite. A peer-reviewed clinical decision support study published in MDPI Bioengineering found adaptive, topic-boundary-aware chunking hit 87% accuracy against a fixed-size baseline that only reached 13%, a statistically significant result (p = 0.001) and the strongest single number in favor of content-aware chunking MDPI Bioengineering, November 2025. A separate study found Paragraph Group Chunking came out on top for overall accuracy, at a mean nDCG@5 of roughly 59% arxiv.org. Vecta ran a benchmark in February 2026 across 50 academic papers and found recursive 512-token splitting came out first, at 69% accuracy, while semantic chunking, in that same benchmark, actually ranked lower at 54%, producing fragments that averaged only 43 tokens, too small to carry the context a query needs huggingface.co dev.to LEANN: A Low-Storage Vector Index. And a NAACL 2025 Findings paper pushed back on the whole premise of semantic chunking's added value, concluding its computational cost isn't justified by consistent gains, since fixed 200-word chunks matched or beat it across both retrieval and answer generation tasks.

Read across those results honestly, no single method wins in every setting. What decides the outcome is the type of document and the pattern of queries being asked against it.

Meeting transcripts, though, have something most benchmarked document types don't: a natural chunking signal already built into the content. Speaker-turn boundaries are semantically coherent on their own; a fixed-size splitter throws that structure away, and a semantic chunker has to spend effort rediscovering something that was already sitting there in the diarization output. The practical approach, then, is to use speaker turns as the primary chunk boundary, group short consecutive turns that share a topic segment into larger chunks when the query calls for complex reasoning, and fall back to fixed-size splitting only in the rare case where turn structure isn't available. Once the baseline pipeline is stable, adding a reranking step, cross-encoder models like bge-reranker-large or ms-marco-MiniLM-L-6-v2, can reorder the top candidates and squeeze out meaningfully better precision. Chunk size guidance from sources recommends 64–128 tokens for fact-based queries and 512–1024 tokens for complex reasoning tasks. Similarity threshold tuning calls for 0.7–0.8 for technical content and 0.5–0.6 for narrative or conversational content.

Choosing an embedding model for conversational, multilingual, and technical transcript content

Embedding model choice gets less attention than it deserves. Most write-ups treat it as a default setting you accept and move past, but it shapes retrieval quality just as much as chunking does.

For general conversational text, all-MiniLM-L6-v2 from the sentence-transformers family is fast and light on memory, a reasonable default for straightforward meeting content. Once the transcripts start carrying specialized terminology, engineering standups, legal reviews, anything with jargon, all-mpnet-base-v2 tends to hold up better. For corpora that span multiple languages, BGE-M3 was chosen in hybrid retrieval research specifically for its strength across multilingual benchmarks, and it's a sound default whenever a transcript archive isn't confined to one language.

A model trained mostly on general web text can underperform badly on domain-specific vocabulary, whether that's medical terminology, legal phrasing, or engineering jargon, and if queries keep missing specialized terms, that's the first place to look. The only reliable way to catch this before it becomes a production problem is to run a small offline evaluation against a sample of actual meeting queries before committing to a model, because swapping embedding models later means re-indexing the entire corpus from scratch, which is expensive at any real scale. And the model choice doesn't stop mattering after that decision: it fixes the vector dimensionality for the whole system, which in turn drives storage cost and index build time, the exact tradeoffs that shape the database decision covered next.

Vector database options and their storage and performance tradeoffs

Vector databases handle scale through a combination of specialized index structures, HNSW graphs and IVF among them, that cut down the complexity of a similarity search, along with quantization techniques that compress vectors and can cut storage costs by 75% or more while keeping search quality intact dev.to LEANN: A Low-Storage Vector Index. The choice of which database to run is about which tradeoffs match the workload." It's about which tradeoffs match the workload.

Pinecone takes the opposite philosophy: fully managed, zero operational overhead, with built-in inference covering both embeddings and reranking, full-text hybrid search, and BYOC deployment options for teams that need infrastructure control without running the servers themselves. Weaviate offers hybrid search out of the box, combining vector similarity, BM25, and metadata filters through BlockMax WAND and RSF, with modular embedding support and documentation that holds up well in practice. Milvus, and its managed counterpart Zilliz Cloud, is built for the billions-of-vectors range at lower cost per vector, though it asks for more engineering effort to run well. ChromaDB, by contrast, is an in-memory HNSW index persisted to disk, genuinely useful for prototyping, but it has no horizontal scaling, no native hybrid search, and no filtered HNSW; teams tend to hit a performance wall somewhere around 5 to 10 million vectors and migrate off it. TiDB takes a broader systems approach, combining distributed SQL, vector search, HTAP (so OLTP and OLAP workloads run together), and ACID transactions in one system, and it supports all four layers of agent memory with horizontal scalability, which matters if the meeting search system is one component inside a larger AI agent architecture.

Put simply, the decision usually comes down to what's already in the stack and what scale is actually expected. A small to mid-size corpus with Postgres already running points to pgvector plus pgvectorscale. A team that wants zero operational burden and has budget for a managed service should look at Pinecone. An open-source preference paired with a need for hybrid sparse-and-dense search points toward Qdrant. Billions of vectors on a tight cost budget favor Milvus. And anything still in the prototyping phase can start on ChromaDB, as long as there's a migration plan sitting on the shelf for when it's outgrown.

Storage cost deserves one more note here. Research on the LEANN index (arXiv:2506.08276) shows that aggressive quantization can shrink vector storage overhead dramatically, which matters a great deal when planning the storage budget for a meeting archive that's only going to grow huggingface.co dev.to LEANN: A Low-Storage Vector Index. And this isn't purely theoretical: a GitHub issue from an AI-Based-Meeting-Transcript project, published August 17, 2026, shows real teams building on pgvector infrastructure naturally reaching for hybrid keyword-plus-vector search as their next step, because the infrastructure to support it is usually already sitting there. The actual gap is in the retrieval design. With pgvector and pgvectorscale, PostgreSQL stores embeddings, documents, and metadata in one database queryable with SQL joins, and pgvectorscale (Timescale's PostgreSQL extension) enables PostgreSQL to deliver 471 QPS at 99% recall on 50 million vectors (11.4x better than Qdrant by one benchmark), making it the strongest fit for teams already running Postgres who want to avoid a separate vector store huggingface.co dev.to LEANN: A Low-Storage Vector Index. Qdrant v1.9+ is open-source, written in Rust, available for self-hosting or via Qdrant Cloud, offers native sparse vector support (SPLADE, miniCOIL) and ColBERT multi-vector, provides the best free tier, and delivers high-throughput vector search with good metadata filtering.

Hybrid retrieval: combining vector similarity with keyword and metadata filters

Semantic search alone has a specific, predictable blind spot: it's weak on exact matches. Product names, version numbers, proper nouns, and alphanumeric identifiers like ticket numbers or contract IDs appear constantly in meetings, and embedding similarity handles them poorly because these tokens don't carry semantic meaning the way conversational language does. Keyword search has the opposite blind spot: it misses paraphrase entirely. A search for "database migration" won't surface a meeting where the same idea got described as "moving off the legacy system" or "the schema refactor project."

Hybrid retrieval solves this by running both signals together rather than picking one. BM25 supplies lexical precision, dense vector similarity supplies semantic recall, and metadata filters, speaker, date range, meeting series, topic label, get applied either before or after the search to narrow the candidate pool.

The InsightToast system, presented at UIST '26 (November 2–5, 2026, Detroit, MI, arXiv:2608.31115), demonstrates hybrid knowledge sourcing in a meeting context: a multi-agent LLM pipeline formulates multiple complementary search queries from detected knowledge gaps and retrieves from an internal vector knowledge base, a production pattern worth studying for cross-meeting search design LEANN: A Low-Storage Vector Index.

Once hybrid retrieval is returning a reasonable candidate set, the final stage to add is reranking, where a cross-encoder model, bge-reranker-large or ms-marco-MiniLM-L-6-v2 among the common choices, scores each candidate directly against the full query. It costs more compute than plain embedding similarity, but the gain in precision at the top of the results list is substantial. And metadata filtering isn't a nice-to-have bolted on at the end. For a query like "what did the legal team say about indemnification in Q2 2026," filtering by speaker separates a useful answer from a haystack. As of the sources, platforms with native hybrid support include Weaviate (BlockMax WAND + RSF), Milvus 2.5+ (Sparse-BM25), Qdrant v1.9+ (named vector hybrid), LanceDB, and Pinecone (proprietary sparse encoding). SOURCE PAGES (what the pages behind the outline's links say).