domains.include_in_sitemap gated the sitemap AND every directory listing query. So "do not submit this to Google" and "do not show this to a person browsing NearByMeNow" were the same switch. There is no setting for the state par-create was designed to produce: a real site, worth showing to the human who conjured it, not yet worth asking Google to index.
Now:
| flag | question | posture |
|---|---|---|
include_in_directory |
can a person find this in NBMN? | generous - real content about a real, located place |
include_in_sitemap |
do we ask Google to index it? | conservative - thin content stays out |
siteDirectoryGateSql($alias) is the fragment every listing query uses; it falls back to the old flag until the migration has run, so there is no broken window. sitemap.php and indexnow.php stay on include_in_sitemap deliberately - they are the index side.
A new skeleton must close both gates explicitly. include_in_directory defaults to 1 so that thousands of existing rows keep working; a skeleton with no content would therefore list itself. par-create.php writes 0,0.
The rule "skeletons are promoted on enrichment" was written down, taught to every session, and never implemented. Par-create inserted include_in_sitemap = 0 and the only statement anywhere that set it back to 1 was a manual bulk action in the admin domain list. 662 sites were enriched (for $1.11, in one batch) and stayed invisible.
sitePromoteOnEnrichment($businessId) now runs from saveGeneratedContent() - the single place all generated content is written. Directory visibility is earned by having substantive content; index eligibility is a separate, stricter test (siteIndexEligible: real prose + an image + an address + coordinates).
A documented rule with no code behind it is indistinguishable from a bug, and harder to find, because everyone believes it already works.
1,658 of 2,929 listable businesses appeared on no niche page. Not bad data - the niche_mappings table simply had no row for the type they carry, so no page existed to put them on. The largest group was association_or_organization at 1,627: the lodges, which drive 68% of human traffic, had no category.
dz nichecatalog [seed=1 [live=1]] is the seeder; dz nichecatalog alone is the standing gap report. Two kinds of entry, and the distinction matters:
live-music, theaters, stadiums, golf-courses, sports-clubs, breweries, wineries, caterers, medical-clinics, chiropractors, nonprofits, farms).
mexican_restaurant is a restaurant. It should not get its own page; it should be findable on the restaurants page and rank there as an exact match.
Orphans went 1,658 -> 4.
Google's broadest tags (service, consultant, health, finance) route to the catch-all businesses page - a page beats being unreachable, and a wrong category would be worse than a vague one. locality and food are excluded on purpose: one is a town, the other is a modifier.
backfillBusinessFlatColumns() chose the first non-generic Google type, so ['consultant','accounting','finance'] filed under consultant - no page - while accounting sat right behind it. It now prefers the earliest type that resolves to a known niche, falling back to the old behaviour so a genuinely new category is still recorded rather than dropped. Anything unmapped surfaces in dz nichecatalog instead of vanishing.
The infrastructure-probe denylist (v2.2.563) treated ^[a-z0-9]{12,}$ as a generated hash. That matches any all-letter word of twelve characters or more, so associations (exactly 12), chiropractors, veterinarians, electricians, supermarkets and universities all 404'd on their own subdomains. A real hash mixes in digits; a niche slug never does. A guard written against one shape of garbage will catch real data if the shape is described loosely enough - and it fails silently, because a 404 on a page nobody has linked yet looks like nothing.