The Tellodb Proxy (Coming Soon)

Overview

The Tellodb Proxy (Memory Router) is a planned feature that will act as a middleware between your application and your LLM provider. It will intercept standard OpenAI-style chat completion requests, retrieve the most relevant memories for the specified user, and inject them into the system prompt before forwarding the request to the upstream model.

This will allow you to add Tellodb's persistent memory to any existing agent or application by simply changing the `base_url`. The feature is currently in development.

How it works

1. Your app sends a request to `/v1/chat/completions` on the Tellodb engine.
2. Tellodb extracts the `user` field from the payload to identify the `entity_id`.
3. It performs a high-precision semantic lookup based on the latest user message.
4. The system prompt is augmented with a structured `[TELLODB PERSISTENT MEMORY]` block.
5. The augmented request is forwarded to OpenAI (or your configured provider).
6. The final response is returned to your application.

Usage Example

To use the proxy, simply point your OpenAI client to your Tellodb instance. The `user` parameter is mapped to Tellodb's `entity_id`.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3000/v1",
    api_key="YOUR_TELLODB_API_KEY"
)

# Tellodb will automatically retrieve memories for 'user-42'
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What was the name of that coffee I liked?"}],
    user="user-42"
)

Configuration

The proxy behavior can be tuned using the following environment variables on the Tellodb engine:

• OPENAI_API_KEY: Your upstream provider key.
• TELLODB_PROXY_TARGET_URL: The upstream endpoint (defaults to OpenAI).
• TELLODB_PORT: The local port Tellodb is running on (for loopback lookups).