Vector embeddings

What is a vector embedding?

A high-dimensional numerical representation of a piece of text. Dimensions vary widely by model: sentence-transformers MiniLM uses 384 dimensions, BERT-base 768, OpenAI ada-002 1536, OpenAI text-embedding-3-large 3072, Cohere embed-v3 1024. Newer models trend toward higher dimensions with longer context windows; choice involves tradeoffs between retrieval accuracy and storage cost. Generated by an embedding model such as OpenAI's text-embedding-3¹, Cohere's embed-v3², Voyage AI, or BGE, which compresses semantic meaning into a fixed-length vector. The key property: two pieces of text with similar meaning produce vectors that are mathematically close in the embedding space, even when they share no exact words.

Status in 2026

Common infrastructure. Many modern AI search, RAG, and recommendation systems use embeddings for at least part of retrieval, clustering, reranking, or semantic matching. Observable behavior of major commercial AI search engines is consistent with embedding use, but specific architectures (which embedding model, how dense and lexical signals are combined, whether reranking is applied) vary per engine and are generally not vendor-documented. Most production systems are documented to implement hybrid retrieval (combining lexical and semantic signals), as documented across vector-database vendors including Pinecone, Weaviate, Elasticsearch, and Azure AI Search; see the hybrid retrieval entry for the architecture detail. Retrieval evaluation benchmarks such as MTEB³ (Muennighoff et al. 2022) and BEIR (Thakur et al. 2021) document the relative strengths of embedding models across diverse task types. Embedding models continue to improve roughly annually, with each generation extending context window length and multilingual coverage.

Note on this entry's territory (paired with the BM25 entry as the lexical counterpart): vector embeddings as a technical concept are strongly vendor-canonical: OpenAI, Cohere, Voyage AI, BGE, and sentence-transformers all publish detailed model documentation including dimensions, training methodology, and pricing. The specific application to commercial AI search engines (which embedding model Perplexity uses, how Microsoft Copilot's hybrid stack combines BM25 and embeddings) is non-vendor-canonical: engines do not disclose their retrieval architecture details. The content-side application (precise term usage, concept density per chunk) sits in practitioner-discipline territory: writers can directly measure consequences of their choices via the isolation test described in the passage-level optimization entry. Paired with BM25: vector embeddings cover the semantic layer of hybrid retrieval (intent match); BM25 covers the lexical layer (exact-string match). Together they describe the two foundational components hybrid retrieval combines.

How to apply

You do not tune the embedding model (engines do), but you write the text that gets embedded, and that embedding can influence which semantic queries surface your content in systems that use embedding-based retrieval. Three writing-side levers:

• Use the precise terms your audience actually uses, not insider jargon: embeddings cluster semantically, but exact-term matching still happens via hybrid retrieval (BM25 + embedding). If your audience says "AI search," writing "generative engines" loses both the embedding cluster fidelity and the keyword hit.
• Lead each section with its dominant concept: standard embedding models (OpenAI text-embedding-3, sentence-transformers, Cohere embed-v3) do not weight tokens by position; they aggregate all tokens via mean pooling or fixed-position aggregation ([CLS] / last token). What "front-loading helps" actually means in embedding terms: (a) most production RAG systems chunk at fixed token boundaries (~200-1024 tokens; see passage-level optimization), so concepts placed near the start of a section are less likely to be cut off by a chunk boundary; and (b) concept density: a section opening with "FAQPage schema is..." has higher concept-per-token density than one opening with "When considering structured data options...", which raises the semantic centroid's signal-to-noise ratio. The lesson is concept density, not opening position. This matches the same self-aware correction the hybrid retrieval and BM25 entries apply to position-weighting myths.
• Don't dilute concept density with conversational filler: openings like "Let's talk about..." or "It's interesting that..." add tokens without adding concept signal. Tighter prose embeds with sharper semantic centroids.

What to skip: chasing "embedding-optimized" copywriting frameworks. Most are speculative; no engine publishes their embedding model's exact training. Clear, precise prose embeds well across every engine.

How it relates to other concepts

• Semantic-layer counterpart to BM25: vector embeddings handle intent match; BM25 handles exact-string match. Together they describe the two foundational components hybrid retrieval combines.
• Foundation of semantic retrieval inside RAG systems, paired with lexical retrieval in hybrid retrieval production stacks.
• Underlies sub-document retrieval in many implementations. Many RAG systems compute embeddings over chunks or passages, though granularity varies by implementation (document-level, section-level, multi-vector, or late-interaction representations also exist).
• Different optimization layer than GEO: GEO operates at content creation time (writing chunkable passages, choosing entity language); vector-embedding choice and indexing happens at engine architecture time, controlled by engine operators not content publishers. Cross-reference: see the LLM Optimization entry for broader LLMO context (LLMO is non-vendor-canonical practitioner shorthand, while vector embeddings themselves are vendor-canonical with detailed OpenAI / Cohere / Voyage / BGE documentation).