# Public Data Completion And Source-Link Audit

This audit is the single evidence ledger for the public-data surface. It documents what is covered today, where the original-source links live, and what is intentionally not exposed.

## Current Evidence Snapshot

| Area | Current evidence | Status | Known limit |
|---|---|---|---|
| Full data universe | `full-data-surface-matrix.tsv` tracks 89 known Homecastr data assets across REST, MCP, OpenAPI, docs, SDK/library, readiness, sensitivity, license status, blocker, and next step. | Covered | The matrix is category-level for some internal and holdback families; it is not a raw-row dump. |
| Original source links | `source-inventory.tsv` tracks 112 source rows and 115 official upstream links. Validator checks every non-exception source row has an HTTP official URL, and `pnpm summarize:public-data` prints both known and unexpected missing-URL lists. | Covered | Two explicit exceptions remain: `source_review.discovery_universe` is a summary of a larger untracked source-review ledger, and `intervention.policy_action` is an umbrella class awaiting exact local rows. |
| Source-to-panel lineage | `panel-lineage.tsv` tracks 140 source-to-panel edges across 38 panels, with join keys, transform IDs, output fields, status, and evidence files. | Covered | Lineage is metadata-public; protected raw rows and storage identifiers are not published. |
| Public API surface | `/api/v1/data/sources`, `/api/v1/data/sources/{source_id}`, `/api/v1/data/panels/{panel_id}/lineage`, `/api/v1/data/coverage`, `/api/v1/data/coverage/sources`, `/api/v1/data/coverage/map`, and artifact downloads expose the source-link, lineage, coverage-index, and declared-coverage records. | Covered | Public endpoints expose source metadata and governed access status, not every underlying raw payload. |
| MCP surface | `list_data_sources`, `get_data_source`, `get_panel_lineage`, `get_data_coverage`, `list_source_coverage_index`, `get_source_coverage_map`, and `get_data_artifact` mirror the REST source, lineage, coverage, map, and artifact surfaces. | Covered | MCP uses the same public/governed access posture as REST. |
| OpenAPI and SDK | OpenAPI documents the data-source, panel-lineage, coverage, source-coverage-index, coverage-map, and artifact operations; `apps/web/src/lib/public-data-client.ts` wraps the same operations. | Covered | The TypeScript wrapper is in-repo; separately published SDK packages are still future work. |
| Rendered docs | `/docs/data` starts with a ranked use-path navigator for forecast consumers, benchmark/eval users, source/provenance auditors, training-panel researchers, and governed-data reviewers, with copy-ready REST, MCP, and SDK examples for each path; `/docs/data` includes a public-safe artifact shelf with direct metadata and download links for the matrix, standardization ledger, source inventory, panel lineage, coverage GeoJSON, reference boundaries, coverage index, schema bundle, release screen, and completion audit; `/docs/data#standardization` shows the raw-to-delivery lifecycle and family implementation map for all matrix-backed assets; `/docs/data#coverage` shows an interactive source-coverage atlas with selectable/deselectable declared geographies, focused-geography controls, source-to-region zoom/isolation, dataset/panel filters, associated sources, an "Original source links for selected map" directory that renders every official URL for the current geography/dataset/search/source slice, source-to-geography index, a searchable panel-to-original-source directory, and downloadable coverage GeoJSON/TSV artifacts; `/docs/data#sources` renders source cards with official upstream links and source JSON/lineage links. | Covered | Country, state, county, city-proxy, and metro-proxy regions use public reference boundaries where available and generalized selection regions elsewhere; source-native boundaries are still gated by source-specific review. |
| Standardization ledger | `standardization-ledger.tsv` and `public-data.standardization_ledger_tsv` join each matrix row to raw retention, canonical contracts, normalization, engineering, synthesis, target REST/MCP/OpenAPI/docs/SDK surfaces, public-readiness, blockers, and next steps. | Covered | This is an implementation and policy ledger; it does not publish governed raw payloads. |
| Public-safety gate | `validate-exposure-registry.mjs` blocks launch assets with high sensitivity or unresolved license posture, requires surface parity, requires source evidence, and allows only named official-URL exceptions. | Covered | License review is metadata-level for holdback source rows; raw redistribution still requires source-specific review. |
| Benchmark namespace | `/api/v1/benchmarks/catalog`, `/api/v1/benchmarks/vintages`, `/api/v1/benchmarks/backtests`, and `/api/v1/benchmarks/sources` remain the benchmark-first compatibility surface. | Covered | Some benchmark panels remain metadata-only until raw source terms and leakage controls are reviewed. |
| External contribution briefs | `docs/github-contrib/ranked-matrix.md` maps adjacent-repo opportunities to concrete Homecastr assets and public surfaces for Nixtla, Chronos, TimesFM, Prior Labs, Earth AI, and related projects. | Covered | Briefs are draft contribution packets; protected-org interactions still require explicit approval before live outreach. |
| Public-records acquisition evidence | `public-records-acquisition-rollup.md` and `public-records-acquisition-rollup.tsv` fold in the parallel public-records/R2 worktree evidence for Florida DOR, Texas CAD, Maricopa, MAG/OEO/Phoenix, Georgia permits, utility easements, water utility capacity, NPMS/EIA, FCC, and PHMSA public-alternative lanes. | Covered | The rollup is metadata and request-gate evidence. It does not publish protected correspondence, raw source payloads, source-native restricted geometry, or send-ready request copy. |

## Where Original Source Links Live

The canonical original-source link surface is `source-inventory.tsv`. The same rows are available through:

| Consumer path | What it returns |
|---|---|
| `GET /api/v1/data/sources` | Filterable source catalog with `official_urls`, source family, geography, license status, training eligibility, panels, and evidence file. |
| `GET /api/v1/data/sources/{source_id}` | One source record with all `official_urls` plus `_links.official` for the primary upstream URL when present. |
| `GET /api/v1/data/panels/{panel_id}/lineage` | Source-to-panel edges that embed each source's `official_urls` and lineage metadata. |
| `GET /api/v1/data/coverage` | Counts for source rows, official links, missing official URLs, panels, and lineage edges. |
| `GET /api/v1/data/coverage/sources` | One-row-per-source coverage/geography index with coverage feature IDs, coverage geometry role, reference-boundary metadata, native-boundary status, official URLs, and redistribution policy. |
| `GET /api/v1/data/coverage/map` | Declared-coverage GeoJSON region layer with public reference boundaries where available, generalized regions elsewhere, source IDs, source names, official URLs, panel IDs, readiness, license metadata, and geometry notes. |
| `GET /api/v1/data/artifacts/public-data.source_coverage_map_geojson/download` | Declared-coverage GeoJSON region layer with public reference boundaries where available, generalized regions elsewhere, source IDs, source names, official URLs, panel IDs, readiness, license metadata, and geometry notes. |
| `GET /api/v1/data/artifacts/public-data.reference_boundaries_geojson/download` | Public Census TIGERweb and Natural Earth reference-boundary cache for coverage regions that can be safely represented with redistributable country, state, county, city-proxy, or metro-proxy geometry. |
| `GET /api/v1/data/artifacts/public-data.source_coverage_index_tsv/download` | Complete TSV artifact mapping every source row to coverage-map features, native geography hints, official URLs, native-boundary review status, and geometry redistribution policy. |
| `GET /api/v1/data/artifacts/public-data.standardization_ledger_tsv/download` | Complete joined TSV showing raw-retention, normalization, engineering, synthesis, canonical contracts, and target REST/MCP/OpenAPI/docs/SDK surfaces for every matrix asset. |
| `GET /api/v1/data/artifacts/public-data.source_inventory_tsv/download` | Complete TSV artifact containing every tracked source row and official URL field. |
| `GET /api/v1/data/artifacts/public-data.panel_lineage_tsv/download` | Complete TSV artifact tying original sources to panels and transforms. |

The rendered docs at `/docs/data#sources` show the same official URLs as outbound links and provide per-row source JSON plus panel-lineage links.

The rendered atlas at `/docs/data#coverage` and `/coverage` also shows an "Original source links for selected map" directory. That panel recomputes from the active map selection, dataset/panel checkboxes, search query, and hidden-source toggles so users can inspect every upstream URL associated with the selected geographies instead of opening source rows one by one.

## What "Everything" Means Here

Everything means every known data family has a documented target path through raw retention, normalization, engineering, panelization, synthesis policy, REST, MCP, OpenAPI, docs, SDK/library, and artifact packaging.

Everything does not mean public raw release of every source payload. Parcel/unit histories, internal analytics, training-cost records, source-unclear extracts, and relationship-sensitive research remain governed or unavailable until source terms and release policy are explicit.

## Current Counts

| Metric | Count |
|---|---:|
| Documented Homecastr data assets | 89 |
| Source inventory rows | 112 |
| Official upstream links | 115 |
| Source rows with official links | 110 |
| Explicit official-link exceptions | 2 |
| Source-to-panel lineage edges | 140 |
| Lineage panels | 38 |
| Source-to-coverage index rows | 112 |
| Public-records acquisition rollup rows | 13 |
| Launch-ready assets | 22 |
| Governed holdback or non-public assets | 25 |

## Not Claimed Yet

- The tracked repo does not contain the master source-review CSV behind the 1,671-source discovery-universe summary, so the docs do not fabricate exact outbound URLs for those untracked mentions.
- Phase 1 does not expose raw parcel/unit histories, source-unclear assessor extracts, internal cost data, user/session analytics, or protected relationship artifacts.
- Phase 3 training panels are candidates only; each requires source-specific license review, leakage controls, provenance manifests, and explicit publication approval.
- The current SDK surface is an in-repo TypeScript wrapper, not a separately versioned public SDK package.
- The Georgia Douglas `PermitShapefile.zip` attachment is not acquired or R2-verified; Georgia records-law follow-up remains closed until connector/manual export, R2 ingest, and exact downstream gap checks are complete.
- The public-records acquisition rollup is not a blanket release approval. R2 source-review bundles prove durable acquisition evidence, not raw redistribution rights or training eligibility.

## Verification Commands

```text
pnpm validate:public-data
pnpm summarize:public-data
pnpm --dir apps/web lint
pnpm --dir apps/web typecheck
pnpm --dir apps/web test
pnpm --dir apps/mcp-server build
```

Use `/docs/data#standardization`, `/docs/data#coverage`, `/coverage`, and `/docs/data#sources` in preview to visually verify the standardization lifecycle, family implementation map, interactive selectable source map, dataset/panel filters, selected-map original-source link directory, source-to-geography index, coverage GeoJSON download, coverage-index TSV download, standardization ledger download, official source links, source TSV download, lineage TSV download, and this audit artifact.

`pnpm summarize:public-data` should report `unexpected_missing_official_url_sources: []`.