Source-of-truth cache (read the real website)

systems/source-of-truth-cache.md · System · enrichment, accuracy, scraping, gigo

unverified: no trust signals recorded yet

A business's official website cached as markdown (Jina Reader) — the accuracy foundation for enrichment; enrich content, never overwrite location.

Source-of-truth cache

86% of our businesses have an official website — the highest-signal reference material for getting hours / services / menus / products RIGHT. Reading it closes the generate-from-Places accuracy gap that produces wrong menus and products. This is GIGO: better signal in, better site out.

How it works

includes/source-cache.php. Fetches the business website as clean markdown via Jina Reader (https://r.jina.ai/<url> — free, no key, renders JS; a hosted GET, nothing to install, which fits the FTP/no-build model). An optional jina api_keys entry (Bearer) raises limits but is not required. Cached in business_sources (business_id PK, url, markdown MEDIUMTEXT ≤80KB, title, bytes, status, fetched_at) — fetch once, refresh on demand (cost discipline). Tool: dz source biz=<id> [refresh=1].

Note: Firecrawl never worked for us — Jina Reader replaces that need.

The enrichment leg (built on top of this)

1. Fact extraction — AI over the cached markdown → structured facts (hours, services,

menu + prices, products, team, service area, booking URL). Accuracy-first: null when the source is silent, NEVER invent.

2. Validation filter — cross-check the generated Doozer site vs. the facts; the source

wins on conflicts; flag low-confidence for review.

3. Never copy prose, extract facts — accuracy + no duplicate-content SEO hit (the

SEO cliff was scaled-content demotion).

4. Deep extractions (full menus, product catalogs) are a Doozer+ feature.

Pairs with postcard targeting: enrich the target sites from their real websites BEFORE mailing, so a scanned card lands on an accurate, better page.

GEO-ANCHOR RULE (load-bearing)

Enrich content (services / products / menu / hours / about / brand). NEVER overwrite the site's LOCATION/address — each Doozer site is already geo-verified per its Places lat/lng/city. Multi-location businesses list several addresses: the homepage header can be an admin/mailing address while the storefront we want sits behind a "View all locations" dropdown. Match the source location to OUR site's city; never take the header address. The validation filter flags conflicts for review — it does NOT auto-clobber. Fix accuracy without introducing errors (the taste filter).

Example: Powell Flooring's homepage shows Oakland NJ (admin); the wanted address is 1811 Union Valley Rd, West Milford (behind the dropdown). An anomaly — most businesses are a clean single-address hit.

See also: content generation, fact guard.