Feature Deep-Dive: AI Summaries (Gemini)

What it does

Generates two-sentence summaries on demand using the reader’s Gemini API key (or an optional team fallback) and caches them per item.

Why it matters

We wanted a low-friction “TL;DR” button for the HUD that keeps readers in flow. Summaries help triage long reads without opening new tabs.

Design

sequenceDiagram
  participant UI
  participant API as /api/ai/summarize
  participant DB as items (metadata)
  participant Gemini
  UI->>API: POST { itemId, force? }
  API->>DB: read item + user settings
  API->>Gemini: generateContent(title + snippet)
  Gemini-->>API: summary text
  API->>DB: metadata.ai_summary
  API-->>UI: { summary, cached }

Tech choices & trade-offs

OptionPerfComplexityCostDXWhy we picked/not
Gemini 1.5 FlashHighLow$$GreatChosen — fast + cheap
OpenAI miniHighLow$$$GoodHigher cost

Implementation walkthrough

web/src/app/api/ai/summarize/route.ts orchestrates the flow:

const GEMINI_MODEL = 'gemini-1.5-flash-latest';
const prompt = `Summarize the following news item...`;
  1. Validate the request body with zod.
  2. Authorize the user and fetch the target item (items joined with sources).
  3. Pull the stored Gemini key from profiles.ai_settings or fall back to GEMINI_API_KEY.
  4. Call the Gemini REST endpoint, parse the response, and write the summary into items.metadata along with provider metadata.
  5. Return cached summaries when force isn’t set.

The key management endpoints live next door in /api/ai/key and keep secrets server-side.

Smart bits (gotchas)

  • No key? The route returns 400 with a friendly error so the UI can prompt the reader.
  • Summaries are cached per item; forcing a refresh (force: true) lets users regenerate after major story updates.
  • We log the provider, model, and ai_summary_updated_at timestamp for future audits.

Testing

  • Hit the route twice: first call should hit Gemini, second should return { cached: true }.
  • Delete the key via /api/ai/key and ensure summaries are blocked until a new key is saved.

Operational notes

  • Rate-limit errors bubble up as 502; consider wrapping the fetch in a retry/backoff if you roll this into production.
  • Keep an eye on token usage—Gemini’s quotas reset daily.

What we’d do next

  • Add additional providers (OpenAI, Claude) via an adapter pattern.
  • Let summaries power in-feed highlights or email digests.