Where the Data Lives

Key Concepts

Live data - Information your agent retrieves in real time by reaching out to an external source. APIs and web scrapes produce live data. It reflects what exists right now, but has no memory of what came before.

Stored data - Information previously saved to a knowledge base, whether by you, a teammate, or the agent itself on a prior run. Stored data gives an agent context and memory, enabling it to detect what has changed over time.

Knowledge base - Any location where your agent reads pre-existing information. This includes documents your team created (brand guidelines, templates, messaging frameworks) as well as outputs the agent saved from previous runs (e.g., a Google Sheet snapshot of last week's competitor pricing).

Quick Reference

Three Data Access Channels

• API - Requests specific data directly from a platform (HubSpot, Google Ads, Salesforce, Search Console). Returns structured data without needing to open the platform's UI.
• Web scrape - Reads content from any public URL. Competitor sites, review pages, pricing pages. Deep research tools like Perplexity can scrape dozens of pages in a single run.
• Knowledge base read - Reads stored files or documents your agent already has access to. Brand guidelines, templates, previous agent outputs, saved snapshots.

The Change Detection Pattern

1. Gather - Agent collects current data (e.g., scrapes a page)
2. Read - Agent pulls what it saved from its last run out of the knowledge base
3. Compare - Agent sends both to an LLM and asks what changed
4. Overwrite - Agent saves today's data back to the knowledge base, replacing the old version

The baseline slides forward with each run. The infrastructure can be as simple as a single Google Sheet cell that gets read at the start and overwritten at the end.

Building Your Data Inventory

1. List every tool and website you use for work
2. For each tool, identify the specific data points available (not "SEO data," but "keywords, positions, click counts by page, impression trends by day")
3. Label each tool with its access channel: API, web scrape, or knowledge base read
4. Brainstorm workflows that combine live and stored data

An agent with only live data can tell you what is there. An agent that combines live and stored data can tell you what changed - and the change is almost always the thing worth knowing.