Embeddings: How AI Understands Meaning

How does a computer know that "king" is to "queen" as "man" is to "woman"? It's not magic; it's math. Specifically, it's a concept called embeddings.

An embedding is a way to translate words, sentences, or even entire documents into a list of numbers (a vector). These numbers capture the "meaning" or "vibe" of the text. Think of them as coordinates on a giant map of concepts. On this map, words and phrases with similar meanings are located close to each other.

This is the foundation for many of the "smart" features we see today, like semantic search (searching by meaning, not just keywords) and recommendation engines.

How Embeddings Work

The process is conceptually simple, even if the underlying neural networks are complex.

When you generate an embedding, the model isn't just counting words; it's looking at the context and relationships between them to produce a rich numerical signature.

Generating Embeddings with the Vercel AI SDK

The Vercel AI SDK makes generating embeddings incredibly easy with its embed function.

// lib/embeddings/generate-embedding.ts
import { embed } from 'ai';
import { openai } from '@ai-sdk/openai';
import 'dotenv/config';

async function main() {
  const { embedding } = await embed({
    model: openai.embedding('text-embedding-3-small'),
    value: 'The quick brown fox jumps over the lazy dog',
  });

  console.log(`Generated embedding with ${embedding.length} dimensions.`);
  console.log(embedding.slice(0, 10)); // Log the first 10 numbers
}

main();

You can also embed an array of texts in a single call using embedMany, which is much more efficient.

// lib/embeddings/generate-batch-embeddings.ts
import { embedMany } from 'ai';
import { openai } from '@ai-sdk/openai';
import 'dotenv/config';

async function main() {
  const documents = [
    'The sun rises in the east.',
    'It is a beautiful and sunny day.',
    'The stock market went up by 2% today.',
  ];

  const { embeddings } = await embedMany({
    model: openai.embedding('text-embedding-3-small'),
    values: documents,
  });

  console.log(`Generated ${embeddings.length} embeddings.`);
  console.log(embeddings.map(e => e.slice(0, 5))); // Log the first 5 numbers of each
}

main();

Measuring "Closeness": Similarity and Distance

Once you have these numerical vectors, you can compare them. The two most common methods are Cosine Similarity and Euclidean Distance.

Cosine Similarity

Cosine similarity is the most popular method for comparing text embeddings. It measures the angle between two vectors, which tells us about their orientation (i.e., their meaning) regardless of their magnitude.

• A value of 1 means the texts are semantically identical.
• A value of 0 means they are unrelated.
• A value of -1 means they are semantically opposite.

// lib/embeddings/similarity.ts
function cosineSimilarity(vecA: number[], vecB: number[]): number {
  if (vecA.length !== vecB.length) {
    throw new Error('Embeddings must have the same dimensions');
  }

  let dotProduct = 0;
  let normA = 0;
  let normB = 0;

  for (let i = 0; i < vecA.length; i++) {
    dotProduct += vecA[i] * vecB[i];
    normA += vecA[i] * vecA[i];
    normB += vecB[i] * vecB[i];
  }

  if (normA === 0 || normB === 0) {
    return 0;
  }

  return dotProduct / (Math.sqrt(normA) * Math.sqrt(normB));
}

Euclidean Distance

Euclidean distance measures the straight-line distance between the tips of two vectors. Unlike cosine similarity, it's sensitive to the magnitude of the vectors. A smaller distance means the items are more similar.

This is less common for semantic search but very useful for clustering tasks, where you want to group similar items together and the "length" of the vector (which can represent things like document length or importance) matters.

// lib/embeddings/distance.ts
function euclideanDistance(vecA: number[], vecB: number[]): number {
  if (vecA.length !== vecB.length) {
    throw new Error('Embeddings must have the same dimensions');
  }

  let sumOfSquares = 0;
  for (let i = 0; i < vecA.length; i++) {
    sumOfSquares += (vecA[i] - vecB[i]) ** 2;
  }

  return Math.sqrt(sumOfSquares);
}

Practical Applications

Semantic Search

Let's build a simple semantic search engine. It will find which document in our "database" is most similar to a user's query using cosine similarity.

// lib/embeddings/semantic-search.ts
import { embed, embedMany } from 'ai';
import { openai } from '@ai-sdk/openai';
import 'dotenv/config';

// (Assume cosineSimilarity function is defined here)
function cosineSimilarity(vecA: number[], vecB: number[]): number {
  if (vecA.length !== vecB.length) throw new Error('Mismatched dimensions');
  let dotProduct = 0, normA = 0, normB = 0;
  for (let i = 0; i < vecA.length; i++) {
    dotProduct += vecA[i] * vecB[i];
    normA += vecA[i] ** 2;
    normB += vecB[i] ** 2;
  }
  if (normA === 0 || normB === 0) return 0;
  return dotProduct / (Math.sqrt(normA) * Math.sqrt(normB));
}

async function semanticSearch() {
  const documents = [
    { id: 1, text: 'The new AI model from Google is setting records.' },
    { id: 2, text: 'Yesterday, the stock market saw a significant increase.' },
    { id: 3, text: 'Generative artificial intelligence is a rapidly growing field.' },
    { id: 4, text: 'What are the best recipes for a healthy breakfast?' },
  ];

  const query = 'What are the latest developments in AI?';

  // 1. Embed all documents and the query
  const { embeddings: docEmbeddings } = await embedMany({
    model: openai.embedding('text-embedding-3-small'),
    values: documents.map(doc => doc.text),
  });

  const { embedding: queryEmbedding } = await embed({
    model: openai.embedding('text-embedding-3-small'),
    value: query,
  });

  // 2. Calculate similarities
  const similarities = docEmbeddings.map((docEmbedding, i) => ({
    id: documents[i].id,
    text: documents[i].text,
    similarity: cosineSimilarity(queryEmbedding, docEmbedding),
  }));

  // 3. Sort by similarity
  similarities.sort((a, b) => b.similarity - a.similarity);

  console.log(`Query: "${query}"\n`);
  console.log('Top 3 most similar documents:');
  similarities.slice(0, 3).forEach(result => {
    console.log(`- [${result.similarity.toFixed(3)}] ${result.text}`);
  });
}

semanticSearch();

Even though our query doesn't share many keywords with documents 1 and 3, the embeddings capture the semantic relationship, correctly identifying them as the most relevant.

Content Recommendation

You can use the same principle to recommend content. If a user likes an article, you can find other articles with similar embeddings.

// lib/embeddings/recommendation.ts
async function getRecommendations(likedItemId: number, allItems: { id: number; text: string }[]) {
  // In a real app, embeddings would be pre-calculated and stored.
  const { embeddings } = await embedMany({
    model: openai.embedding('text-embedding-3-small'),
    values: allItems.map(item => item.text),
  });

  const itemEmbeddings = new Map(allItems.map((item, i) => [item.id, embeddings[i]]));
  const likedItemEmbedding = itemEmbeddings.get(likedItemId);

  if (!likedItemEmbedding) {
    throw new Error('Liked item not found');
  }

  const recommendations = allItems
    .filter(item => item.id !== likedItemId) // Exclude the liked item itself
    .map(item => ({
      ...item,
      similarity: cosineSimilarity(likedItemEmbedding, itemEmbeddings.get(item.id)!),
    }))
    .sort((a, b) => b.similarity - a.similarity);

  return recommendations.slice(0, 3);
}

async function main() {
    const articles = [
      { id: 1, text: 'A deep dive into React Server Components.' },
      { id: 2, text: 'How to optimize your Next.js application for performance.' },
      { id: 3, text: 'Getting started with server-side rendering in Vue.js.' },
      { id: 4, text: 'A guide to object-oriented programming in Python.' },
    ];

    const likedArticleId = 1;
    const recommendations = await getRecommendations(likedArticleId, articles);

    console.log(`Because you liked "${articles.find(a => a.id === likedArticleId)!.text}", you might also like:`);
    recommendations.forEach(rec => {
        console.log(`- [${rec.similarity.toFixed(3)}] ${rec.text}`);
    });
}

main();

What About a Real Database?

Doing this in-memory is great for learning, but for real applications, you'd store your embeddings in a vector database (like Pinecone, Weaviate, or pgvector for Postgres). These databases are highly optimized for performing fast and efficient similarity searches on millions or even billions of vectors.

Embeddings are a fundamental building block of modern AI. By turning unstructured text into meaningful numbers, they allow us to build smarter search, create powerful recommendation engines, and power context-aware RAG systems.