Why AI Search Needs Hybrid Retrieval, Not Just Embeddings

Teams adding AI search to a content library usually reach for embeddings, get a demo working in an afternoon, and then discover the two failures that vector search alone cannot fix: it is slow to feel instant, and it is confidently wrong about exact terms.

The fix is not a better embedding model. It is running two retrieval systems and letting each do what it is good at.

Why isn't semantic search enough on its own?

Vector search matches meaning. Ask for "how do I handle grief" and it will surface a talk titled "walking through loss," which keyword search would miss entirely. That is the whole reason to use it.

But meaning-matching has two structural weaknesses.

It is bad at exact tokens. Proper nouns, product names, episode numbers, speaker names, scripture references. A user typing an exact title wants that exact item first, and a nearest-neighbour search over a 1,536-dimension space gives you something semantically nearby, which is not the same promise.

It is not instant. Embedding the query, then searching the index, then ranking, is fine for a search results page. It is too slow for the autocomplete dropdown that appears while the user is still typing — and that dropdown is where most search sessions actually end.

What does hybrid retrieval mean in practice?

Two indexes over the same corpus, queried for different jobs.

On Gloo's AI Discovery Widget, an embeddable search product for large podcast, sermon, and article libraries, we ran Weaviate for vector semantics alongside Typesense for instant suggestions. Weaviate answers "what is this person actually looking for." Typesense answers "what should appear in the dropdown in the next 30 milliseconds."

The division of labour is the design:

A user never sees two systems. They see suggestions that keep up with their typing, and results that understand what they meant.

Where does the real work go?

Not the retrieval. The ingestion.

A semantic index is only as good as what you put in it, and publisher content arrives messy: RSS feeds with inconsistent metadata, transcripts of varying quality, episodes with no description, articles with markup noise. Turning that into enriched, searchable content is the unglamorous majority of the project.

On Gloo, ingestion ran as Python workers on SQS and Celery, pulling publisher RSS feeds and enriching them before indexing. Asynchronous, because ingestion is bursty and slow and must never block a search request. This is the part teams underestimate when they scope "add AI search" as a frontend task.

How do you ship this as something a partner can install?

The retrieval architecture is invisible to the customer. What they experience is integration cost.

Gloo's widget installs with a single script tag, and configuration happens in a five-step builder with live preview — layout, data sources, API keys, embedding. That constraint shapes everything upstream: multi-tenant API keys, domain-level security so a key cannot be lifted and reused elsewhere, and per-partner analytics through PostHog with Sentry for errors.

An embeddable product is a distribution decision before it is an engineering one. "One script tag" is a promise that has to survive contact with every partner's CMS.

When is hybrid retrieval overkill?

When the corpus is small or the queries are simple. A few hundred documents and mostly-navigational queries do not need a vector database — good keyword search with typo tolerance will outperform it on both cost and latency.

Hybrid earns its complexity when the library is large enough that users cannot know what is in it, and the questions are open-ended enough that they cannot name what they want. That is when meaning-matching stops being a nice demo and starts being the product.

---

A working note from building semantic search at production scale. The full project detail is in the Gloo case study.