Memory Embeddings: Semantic Search for Your Agent

Overview

Memory embeddings unlock semantic search capabilities for your OpenClaw agent, transforming it from a simple keyword matcher into an intelligent system that understands concepts and relationships. This guide explains how embeddings work, why they're essential for advanced workflows, and how to configure them properly.

What Are Memory Embeddings?

The Simple Explanation

Embeddings convert text into numbers so computers can understand meaning.

Example:

• "Pepsi" → [0.23, 0.87, 0.45, 0.12, ...] (vector of numbers)
• "soda" → [0.25, 0.85, 0.43, 0.14, ...] (similar numbers = similar meaning)
• "car" → [0.91, 0.15, 0.08, 0.73, ...] (different numbers = different meaning)

Why it matters:

• Computers are excellent with numbers, poor with words
• Similar concepts have similar vector representations
• Enables semantic search beyond exact keyword matching

The Technical Explanation

Vector embeddings are high-dimensional numerical representations of text that capture semantic meaning. Modern embedding models (like OpenAI's text-embedding-3-small) convert text into vectors of 1,536 dimensions, where:

• Distance between vectors indicates semantic similarity
• Clustering reveals related concepts
• Search finds content by meaning, not just keywords

Why Embeddings Matter

Without Embeddings (Keyword Search Only)

Query: "What's my soda preference?"

Search: Looks for exact word "soda"

Result: ❌ Nothing found (you wrote "Pepsi" not "soda")

With Embeddings (Semantic Search)

Query: "What's my soda preference?"

Search: Understands "soda" relates to "Pepsi", "Coca-Cola", "soft drink"

Result: ✅ Finds "Ron likes Pepsi diet with ice"

Real-World Impact

Scenario 1: Project recall

Scenario 2: Cross-session knowledge

Scenario 3: Concept exploration

How OpenClaw Uses Embeddings

The Two-Layer Memory System

Layer 1: Daily memory files (keyword search)

• Basic text files in ~/.openclaw/memory/
• Fast but limited to exact matches
• Good for recent, explicit information

Layer 2: Vector database (semantic search)

• SQLite database with embeddings
• Slower but finds related concepts
• Essential for long-term, cross-session recall

Hybrid Search Strategy

OpenClaw combines both approaches:

1. Keyword search - Fast, exact matches
2. Semantic search - Slower, concept matches
3. Ranking - Best results from both methods

Result: Optimal balance of speed and accuracy

Enabling Memory Embeddings

Requirements

API Key from:

• OpenAI (recommended)
• Google Gemini (alternative)

Setup Process

Step 1: Check current status

Expected response (if enabled):

Response if NOT enabled:

Step 2: Configure API key

For OpenAI:

For Gemini:

Step 3: Verify setup

Check for database file:

Look for:

• memory.db or sessions.db (SQLite format)
• File size grows as you add memories
• Not human-readable (binary format)

Test semantic search:

Should find results even if you never used exact phrase "project deadlines"

How Embeddings Are Generated

The Embedding Pipeline

Cost Considerations

OpenAI Pricing (text-embedding-3-small):

• $0.02 per 1M tokens
• Extremely cheap compared to LLM calls
• Typical memory entry: 100-500 tokens
• Cost per memory: ~$0.00001-0.00005

Example:

• 10,000 memory entries
• Average 300 tokens each
• Total: 3M tokens
• Cost: $0.06

Verdict: Negligible cost for massive capability boost

Memory Search Workflow

Two-Step Process

Step 1: Search (memory_search)

Step 2: Retrieve (memory_get)

Why two steps?

• Search returns snippets (fast, low token cost)
• Retrieve loads full context (slower, higher token cost)
• Agent only retrieves what's actually relevant

Search Parameters

Similarity threshold:

• 0.0 - 1.0 scale
• Higher = more strict matching
• Lower = more permissive matching

Typical thresholds:

• 0.8+ : Very similar (strict)
• 0.7-0.8 : Related concepts (moderate)
• 0.6-0.7 : Loosely related (permissive)
• <0.6 : Probably not relevant

Organizing Memory for Embeddings

Directory Structure

Recommended layout:

What to Store Where

Daily memory (ephemeral):

• Today's tasks and decisions
• Temporary context
• Quick notes

Project memory (medium-term):

• Project-specific decisions
• Implementation details
• Lessons learned

Knowledge memory (evergreen):

• Reusable patterns
• Standard procedures
• Best practices

Preferences memory (permanent):

• Personal preferences
• Communication style
• Work habits

Keyword vs. Semantic Search

When Keyword Search Works

Good for:

• Exact names: "Project Apollo"
• Specific terms: "API key rotation"
• Recent information: "yesterday's meeting"
• Unique identifiers: "ticket-1234"

Example:

When Semantic Search Shines

Good for:

• Concept queries: "authentication approaches"
• Fuzzy recall: "that pricing thing we discussed"
• Cross-domain: "security best practices"
• Exploratory: "everything about deployments"

Example:

Hybrid Strategy

OpenClaw automatically uses both:

1. Keyword search for exact matches (fast)
2. Semantic search for concept matches (thorough)
3. Merge and rank results by relevance
4. Return top N results

Best of both worlds: Speed + Intelligence

Advanced Configuration

Embedding Model Selection

OpenAI options:

• text-embedding-3-small (default, 1536 dimensions)
• text-embedding-3-large (3072 dimensions, more accurate, more expensive)

Gemini options:

• text-embedding-004 (768 dimensions)

Configuration:

Batch Embedding

For large memory imports:

Benefits:

• Faster than one-by-one
• More efficient API usage
• Progress tracking

Re-indexing

When to re-index:

• Changed embedding model
• Corrupted database
• Major memory reorganization

How to re-index:

Warning: May take time for large memory directories

Integration with External Storage

Obsidian + GitHub Pattern

Architecture:

Workflow:

1. Agent saves to memory directory
2. Obsidian syncs and displays (human-readable)
3. GitHub backs up (version control)
4. Embeddings index (searchable)

Benefits:

• Edit memories in Obsidian (better UX)
• Version control via GitHub
• Backup and sync across devices
• Semantic search via OpenClaw

Setup Guide

Step 1: Configure Obsidian vault

Step 2: Initialize Git

Step 3: Auto-sync script

Step 4: Configure OpenClaw hook

Use Cases and Examples

Use Case 1: Long-Term Project Memory

Scenario: Working on multiple projects over months

Setup:

Query:

Result:

Without embeddings: Would need to remember exact file and search manually

Use Case 2: Cross-Session Learning

Scenario: Agent learns from past mistakes

Memory entry (2 months ago):

Query (today):

Result:

Impact: Agent applies lessons learned without explicit reminders

Use Case 3: News Research Archive

Scenario: Daily news scraping with searchable archive

Setup:

Query:

Result:

Without embeddings: Would need to read all files manually

Troubleshooting

"Embeddings not working"

Symptoms:

• Semantic search returns no results
• Only exact keyword matches work
• No SQLite database file

Diagnosis:

Solutions:

1. Verify API key is set correctly
2. Check API key has embeddings permission
3. Ensure sufficient API credits
4. Restart OpenClaw to reload configuration

"Search returns irrelevant results"

Cause: Similarity threshold too low

Solution:

Or:

"Database file is huge"

Cause: Too many embeddings stored

Solutions:

Option 1: Archive old memories

Option 2: Selective indexing

Option 3: Database cleanup

"Embeddings are expensive"

Reality check:

• Embeddings cost ~$0.02 per 1M tokens
• LLM calls cost $3-15 per 1M tokens
• Embeddings are 150-750x cheaper

If still concerned:

• Use text-embedding-3-small (cheapest)
• Batch embed instead of real-time
• Archive old memories to reduce index size

Best Practices

1. Enable Embeddings Early

Don't wait - Set up embeddings from day one

Why:

• Retroactive indexing is slower
• Lose semantic search benefits during setup
• Harder to organize memory later

2. Write Descriptive Memory Entries

Bad:

Good:

Why: More context = better semantic search results

3. Use Consistent Terminology

Inconsistent:

• "user login" (file 1)
• "authentication" (file 2)
• "sign-in flow" (file 3)

Consistent:

• "authentication" (primary term)
• "Also known as: login, sign-in" (aliases)

Why: Helps embeddings cluster related concepts

4. Regular Memory Maintenance

Monthly tasks:

• Archive old daily memories
• Consolidate related entries
• Remove outdated information
• Update evergreen knowledge

Why: Keeps search results relevant and database size manageable

5. Test Semantic Search

After adding important information:

Ensures:

• Embeddings are working
• Information is findable
• Search quality is good

Related Resources

• Memory Management (4-Layer System) [blocked]
• Context Window Management [blocked]
• Skills Optimization [blocked]

Duration: 11 minutes
Difficulty: Intermediate
Video Reference: OpenClaw Memory Embeddings EXPLAINED