ChronoCast Personal AI Memory & Grounded RAG
"You capture. It remembers. You ask." A complete personal knowledge retrieval system that rescues bookmarked video and social posts from closed walled gardens, transcribes audio on-device, turns content into 1,536-dimensional embeddings, and delivers answers backed by verifiable source citations.
Unlocking the Saved Box: From Trapped Bookmarks to Verifiable Memory
The Problem with Saved Content
Every day, users bookmark insightful video reels, recipes, coding tutorials, and research snippets across social networks. But social platforms are walled gardens. There is no export button, no full-text search across spoken dialogue, and no guarantee a video won't be deleted next month.
Worse, the most valuable information in a video is spoken aloud, not written in the brief caption. Without transcription, 90% of saved knowledge remains unsearchable dark data.
The Core Architecture: Look It Up First, Then Talk
ChronoCast operates as a strict, transparent Retrieval-Augmented Generation (RAG) system. It fetches your own saved content, transcribes every spoken sentence locally, maps it into vector space, and establishes a semantic bridge between your natural language questions and the source media.
The overarching promise is not a smarter answer, but a checkable one: every assertion links directly to an ordinal source footnote pointing to the exact timecoded video capture.
Production Technology Stack
Next.js App Router & Tailwind
Modern responsive frontend with server-side streaming, TanStack Query, React Virtual for rendering extensive media lists, and Day.js for scheduling timelines.
Supabase PostgreSQL & pgvector
Relational storage augmented with the pgvector extension for storing and querying 1,536-dimensional float arrays via cosine distance dot products with HNSW indexing.
Embeddings & Claude AI Digest
Uses text-embedding-3-small for unified 1,536-dim document and query embedding, paired with Claude Haiku (@ai-sdk/anthropic) for automated entity extraction.
Local Whisper & yt-dlp Pipeline
Audio is extracted with yt-dlp and transcribed locally using faster-whisper on-device with zero external API fees or privacy leakage.
Interactive Knowledge Graph
Extracted entities (people, tools, concepts, places) are synced into an interactive dynamic graph rendered with react-force-graph-2d, revealing unexpected connections across saved media.
Zero-Credential Browser Addon
Manifest V3 service worker wakes on a 4-to-7-hour alarm, executing a same-origin fetch inside the user's active session tab via chrome.scripting without storing login credentials.
The 10-Stop Pipeline: From Instagram Bookmark to Grounded Answer
How a Saved Post Becomes an Answer
10-stop deterministic RAG and knowledge graph pipeline
You Save It
CaptureTapping the bookmark drops the post into your Instagram saved list. Read-only source of truth; ChronoCast never deletes or mutates the remote state.
The Robot Checks
ExtensionChrome MV3 background service worker wakes every 4–7 hours, injecting a same-origin fetch inside the active authenticated tab. Zero credentials stored; session cookies never leave the browser.
It Stays Dumb
ExtensionDumb sensor, smart sink: the extension re-sends the entire batch without maintaining client-side state. PostgreSQL ON CONFLICT handles idempotency with zero drift.
The Doorman
SecurityBearer token verified before request body is parsed or DB clients are instantiated. User ID is cryptographically resolved from the token, never trusted from payload.
The Copy Machine
Enrichmentyt-dlp archives metadata and media into private Supabase Storage. faster-whisper runs local on-device audio transcription with zero third-party API keys.
Words to Numbers
Vector Spacetext-embedding-3-small transforms title, caption, and full transcript into a 1,536-dimensional unit vector in pgvector. Similarity computed via cosine dot product.
Sticky Notes
Knowledge GraphClaude Haiku generates a 1-sentence summary, up to 8 topic tags, and extracts named entities (people, tools, places) to construct an interactive 2D/3D knowledge graph.
Two Searchers
RetrievalHybrid search executes two parallel retrieval arms: semantic vector search (conceptual meaning) and lexical ILIKE search (exact names), fused via Reciprocal Rank Fusion: 1 / (60 + rank).
The Floor Gate
ReliabilityHard threshold gate at 0.30 cosine similarity. If the top result is below 0.30, zero LLM model calls are made and zero tokens spent. Prevents hallucinations on unanswerable queries.
The Answer
Grounded GenerationModel receives up to 8 surviving captures and 4,000 chars of verified transcript. Citations [1], [2] are generated in a single pass with ordinal source links directly to video timestamps.
Stop 08 Deep Dive: Two Searchers & Reciprocal Rank Fusion (RRF)
When a user asks a question, two distinct searchers run simultaneously against the user's private dataset:
Converts the question into a 1,536-dim vector and computes cosine distance. Finds conceptual equivalents (e.g. searching for "cooking sauces" retrieves "making reductions").
Executes PostgreSQL ILIKE matching across titles, captions, and transcripts. Incomparable at exact handles, software names, and rare technical keywords.
The results are combined via Reciprocal Rank Fusion: each item at rank r contributes a score of 1 / (60 + r). Items supported by both semantic and lexical signals float to the top with high confidence.
Development Process: Empirical Calibration & Fail-Soft Engineering
The 0.30 Cosine Similarity Floor
Calibrated against 54 labeled questions and 5 unanswerable controls to prevent AI hallucination
An LLM presented with weak, irrelevant scraps doesn't say "I don't know" — it fabricates a fluent, convincing hallucination. ChronoCast enforces a hard gate: if the top semantic match falls below 0.30, zero LLM calls are made and zero tokens are spent.
Empirical Gate Calibration
The 0.30 floor was calibrated with 54 labeled questions and 5 genuine unanswerable questions. The unanswerable controls never exceeded 0.242, while answerable queries never scored below 0.413, placing the 0.30 gate squarely in the noise margin.
Fail-Soft Stage Decoupling
If an Instagram video is private or yt-dlp fails to download media, the caption and author metadata are still preserved, embedded, and made searchable. A partial capture always beats a silent failure.
RLS Security Testing
Every database table enforces Supabase Row-Level Security (RLS) with dedicated automated testing (vitest.rls.config.ts). Users can never access or leak another account's captures or vector coordinates.
Walk through the ChronoCast pipeline live
See how one saved Instagram video moves through the 10 stops, from browser tap to timecoded transcript and cited answers.