Spine — schema design
SLW Data Engine, Phase 1. Written 2026-08-07 against
BRIEF.md, the Future AI plan,
the 8/4 code audit (/root/FUTURE-AI-AUDIT-2026-08-04.md
on slwdev), slw-webster/migrations/0001–0005, slw-ford's intake_links,
and the three live app matchers.
Stop point 1. Nothing is built. No migration has been applied.
One sentence
The spine is the layer that says what a document is about (mentions), what vocabulary the fleet shares (typed tags), how entities relate to each other (edges), and how a new entity gets born (propose, then a human approves) — all of it additive to the registry that already runs, and none of it able to write into any app.
0 · What does not change
Stated first, because every rule below inherits from it.
- The registry links, never merges. Nothing here creates a code path from the engine into LP Flow, DealFlow, Terminal, Podcast, or Ford. Everything the spine knows about those systems arrives through SELECT-only snapshots or through an app calling the engine.
- No name match is ever auto-confirmed. The confidence tiers in
migrations/0001stand. The assessment layer (§6) advises the queue and never writes aconfirmedstatus. - Dewey is untouched. Read-only. The spine points at Dewey pages by id;
it never adds a column to
dewey.*. - Every existing Webster table keeps its shape. New tables and views only.
No
ALTER, noDROP, no backfill that rewrites an existing row.
1 · Where the tables live, and why
Decision: the new tables go in the existing webster schema, in the existing
slw-experiments project (akpdtnhnacxkeehheoqz) — not a new spine schema
and not a new database.
Three reasons, in order of weight:
- The normalization functions are already there and are load-bearing.
webster.norm_name,norm_org_name,norm_domain,norm_linkedin,email_domain,is_free_email_domainback generated columns onentities,src_personandsrc_org. Mentions and tags need exactly the same normalization to join against them. A second schema either duplicates the functions (two definitions that drift) or reaches across (a cross-schema dependency with none of the isolation a separate schema was supposed to buy). - RLS and grants are a per-object surface.
0001and0002end withrevoke all ... from anon, authenticatedandenable row level securityon every table. A second schema doubles the number of places that lockdown has to be re-stated correctly, and the failure mode of getting it wrong is a public read on relationship data. - "Data Engine" is a product concept, not a namespace. The brief is
explicit that the three organs stay separate services. Renaming a
Postgres schema communicates nothing to any consumer — apps talk to
webster.rivent.devover HTTP and never see a schema name.
Object naming carries the concept instead: everything new is prefixed by what
it is (mentions, tags, tag_assignments, entity_edges,
entity_proposals, merge_proposals, link_assessments, matching_policy).
2 · The mention store
The problem Ford already solved, and the part it left open
ford.intake_links records (message_id, entity_kind, name) — a pointer, by
name, because "the entities live in OTHER systems' registries … Ford linking by
name is honest about being a pointer rather than pretending to own the entity."
That honesty is the right instinct and it is also the ceiling. Once the registry exists, "S3" as a string cannot answer which S3, cannot roll up with the LP Flow record for the same firm, and cannot be corrected once. But the fix is not "require an entity id" — a mention is often the first time the fleet has ever seen a name, and refusing to store it until someone resolves it throws away the only evidence that would let anyone resolve it.
The shape
A mention carries a name always, and an entity id when one is known. The name is the durable fact; the entity id is the resolution, added later and reversible.
create table webster.mentions (
id uuid primary key default gen_random_uuid(),
-- WHAT MENTIONS -----------------------------------------------------------
-- The document, addressed the way its own system addresses it. No FK: the
-- documents live in Dewey, Ford, Terminal, Podcast and Magellan, and the
-- registry owns none of them.
source_app text not null, -- ford | terminal | podcast | dewey | magellan | lp_flow | dealflow
doc_kind text not null, -- intake_message | page | literature_post | episode | artifact | note
doc_id text not null,
doc_url text, -- where a human can read it, when derivable
doc_title text, -- so the queue reads without a cross-system fetch
-- WHAT IS MENTIONED -------------------------------------------------------
subject_kind text not null check (subject_kind in ('person','org')),
subject_name text not null,
subject_norm text generated always as (webster.norm_name(subject_name)) stored,
subject_org_norm text generated always as (webster.norm_org_name(subject_name)) stored,
-- Resolution. NULL means "we have not resolved this yet", which is a normal,
-- useful state, not a defect.
entity_id uuid references webster.entities(id) on delete set null,
resolved_by text check (resolved_by in ('exact_key','human','backfill','policy')),
resolved_at timestamptz,
-- HOW IT WAS MENTIONED ----------------------------------------------------
role text, -- publisher | subject | investor | author | speaker | mentioned
passage text, -- verbatim quote, Terminal's source_passage discipline
locator jsonb not null default '{}'::jsonb, -- {page: 4} | {char: [120,340]} | {t: "00:14:22"}
extra jsonb not null default '{}'::jsonb, -- sentiment, stage, fit — app-specific, not promoted
-- PROVENANCE (enforced, not conventional) ---------------------------------
as_of timestamptz not null,
extractor text not null, -- 'ford/classifier@2', 'terminal/literature@7', 'manual'
confidence numeric(4,3) not null default 1.000
check (confidence >= 0 and confidence <= 1),
created_at timestamptz not null default now(),
updated_at timestamptz not null default now(),
-- Re-processing a document must never double its mentions.
constraint mentions_identity unique
(source_app, doc_kind, doc_id, subject_kind, subject_name, role)
);
create index mentions_doc_idx on webster.mentions (source_app, doc_kind, doc_id);
create index mentions_entity_idx on webster.mentions (entity_id, as_of desc) where entity_id is not null;
create index mentions_subject_idx on webster.mentions (subject_kind, subject_norm, as_of desc);
create index mentions_org_idx on webster.mentions (subject_org_norm, as_of desc)
where subject_kind = 'org';
create index mentions_unresolved_idx on webster.mentions (subject_kind, subject_norm)
where entity_id is null;
Why each of the load-bearing choices:
- The unique key includes
role, not just the name. A newsletter can name the same firm as both publisher and subject; those are two facts, not a duplicate. Ford's own key is(message_id, entity_kind, name), which already collapses them — the spine version does not. subject_kindisperson|orgonly, matchingentities.kind. Ford's third kind,sector, is not a mention — it is a tag (§3). Conflating them would put an uncountable noun in a table whose whole purpose is countable subjects, and would makeentity_idpermanently null for a third of the rows.entity_idison delete set null, not cascade. A mention outlives the resolution. Deleting an entity must never delete the evidence that the entity was mentioned.extractoris mandatory and versioned by convention (app/pass@version). It is what makes a re-run auditable: "everything the literature extractor v6 produced" is a query.as_ofis the document's date, notnow()— no default, because the writer knows it and a default would silently record ingestion time as publication time. That is a real class of bug in fleets like this one.
The read model
-- "Everything we've ingested about X", newest first, resolved or not.
create view webster.v_entity_mentions as
select m.*, e.canonical_name, e.kind
from webster.mentions m
left join webster.entities e on e.id = m.entity_id;
The resolve service answers everything about entity E as
entity_id = E OR (entity_id is null AND subject_norm = <E's normalized names>)
— so an unresolved mention still surfaces on the entity page, marked as
unconfirmed. That is the entire point of storing the name.
Visibility
Mentions inherit the per-link visibility rule from 0005 through their
source app: a caller may see a mention iff it may see the source app. The
resolve service applies this in the same one-place predicate as
visibleLinks(). A mention sourced from an app the caller has no relationship
with is invisible and uncountable, exactly as a private link is today.
3 · The typed tag vocabulary
Why the fleet has four vocabularies and needs a fifth
| System | Today | Count |
|---|---|---|
| Terminal | RESEARCH_SECTORS — a frozen 14-value string union in TypeScript |
14 |
| Podcast | INTELLIGENCE_TAXONOMY — slug + label, seeded, clamped to other |
18 |
| Ford | free-text scope.sectors from a model pass, unbounded |
∞ |
| LP Flow | targeting_tags text[] on deals, orgs and people |
free |
| DealFlow | organizations.tag, delimited string; no tags on people |
free |
The engine does not get to replace any of them. Terminal's slugs are baked into a prompt and into historical rows; Podcast's own file says renaming a slug orphans history. So:
Decision: the spine holds a canonical vocabulary plus a crosswalk. Apps keep their own strings, unchanged, forever.
create table webster.tags (
id uuid primary key default gen_random_uuid(),
kind text not null check (kind in ('sector','topic','theme','stage','region')),
slug text not null,
label text not null,
parent_id uuid references webster.tags(id), -- shallow hierarchy: ai > ai-infra
description text,
status text not null default 'active' check (status in ('active','deprecated')),
merged_into uuid references webster.tags(id),
created_at timestamptz not null default now(),
constraint tags_slug_unique unique (kind, slug),
constraint tags_deprecated_requires_target
check ((status = 'deprecated') = (merged_into is not null))
);
-- Every string any app has ever used, pointing at the canonical tag.
create table webster.tag_aliases (
id uuid primary key default gen_random_uuid(),
tag_id uuid not null references webster.tags(id) on delete cascade,
alias text not null,
alias_norm text generated always as (webster.norm_name(alias)) stored,
source_app text not null, -- which book spells it this way
as_of timestamptz not null default now(),
constraint tag_aliases_unique unique (source_app, alias_norm)
);
create index tag_aliases_norm_idx on webster.tag_aliases (alias_norm);
tag_aliases_unique on (source_app, alias_norm) is the important one: within
one app a string means exactly one thing, but "Infrastructure" may legitimately
mean different tags in Terminal and in LP Flow. The crosswalk is per-app by
construction.
Applying a tag
A tag attaches to a document or to an entity. One table, discriminated:
create table webster.tag_assignments (
id uuid primary key default gen_random_uuid(),
tag_id uuid not null references webster.tags(id) on delete cascade,
subject_type text not null check (subject_type in ('doc','entity')),
entity_id uuid references webster.entities(id) on delete cascade,
source_app text not null,
doc_kind text,
doc_id text,
salience int check (salience between 1 and 5), -- Podcast already models this
as_of timestamptz not null,
extractor text not null,
confidence numeric(4,3) not null default 1.000
check (confidence >= 0 and confidence <= 1),
created_at timestamptz not null default now(),
constraint tag_assignments_subject_shape check (
(subject_type = 'entity' and entity_id is not null and doc_id is null) or
(subject_type = 'doc' and doc_id is not null and entity_id is null)
)
);
create unique index tag_assignments_doc_unique
on webster.tag_assignments (source_app, doc_kind, doc_id, tag_id)
where subject_type = 'doc';
create unique index tag_assignments_entity_unique
on webster.tag_assignments (entity_id, tag_id, source_app)
where subject_type = 'entity';
create index tag_assignments_tag_idx on webster.tag_assignments (tag_id, as_of desc);
Seeding
Migration 0007 seeds the vocabulary from what already exists, in this order:
- Podcast's 18 slugs become the
topicspine — it is the only vocabulary in the fleet that was deliberately designed with slugs, labels and a clamp-to-otherrule, and it is already normalized. - Terminal's 14
RESEARCH_SECTORSbecomesectortags, each with atag_aliasrow for its exact display string ("AI/ML","Deep Tech", …) undersource_app = 'terminal'. Terminal's prompt and its historical rows are untouched. - Ford's observed
scope.sectorsstrings are aliased where they match and queued as vocabulary proposals where they do not — a free-text extractor pointed at a controlled vocabulary is exactly the case that produces a long tail, and inventing 200 tags from one model's phrasing would poison the vocabulary on day one. - LP Flow
targeting_tagsand DealFloworganizations.tagare read from the existing snapshot tables, counted, and the top values aliased. The tail stays un-aliased and visible, not silently dropped.
Nothing in this step writes to Terminal, Podcast, Ford, LP Flow or DealFlow. The crosswalk lives on the spine side only.
4 · Relationship edges
Typed, dated, directional, and never asserted by a model without a human on the other end of the tier bar.
-- The controlled predicate list. Domain/range are enforced, so "a person
-- invested in a person" is impossible rather than merely wrong.
create table webster.edge_predicates (
slug text primary key, -- investor_in | partner_at | founder_of | board_member_of |
-- led_round | advisor_to | lp_in | acquired | spun_out_of
label text not null,
from_kind text not null check (from_kind in ('person','org')),
to_kind text not null check (to_kind in ('person','org')),
inverse_of text references webster.edge_predicates(slug),
is_dated boolean not null default true,
description text
);
create table webster.entity_edges (
id uuid primary key default gen_random_uuid(),
from_entity uuid not null references webster.entities(id) on delete cascade,
to_entity uuid not null references webster.entities(id) on delete cascade,
predicate text not null references webster.edge_predicates(slug),
-- dated, because "was a partner at" and "is a partner at" are different facts
valid_from date,
valid_to date,
detail jsonb not null default '{}'::jsonb, -- {title:"Partner"}, {round:"Series B", amount_usd:…}
status text not null default 'suggested'
check (status in ('confirmed','suggested','rejected')),
confidence numeric(4,3) not null default 0
check (confidence >= 0 and confidence <= 1),
derived_by text not null, -- 'crm_snapshot' | 'terminal/deal@3' | 'manual'
evidence jsonb not null default '{}'::jsonb,
mention_id uuid references webster.mentions(id) on delete set null,
source_app text not null,
as_of timestamptz not null,
decided_by text,
decided_at timestamptz,
created_at timestamptz not null default now(),
updated_at timestamptz not null default now(),
constraint entity_edges_identity unique
(from_entity, to_entity, predicate, valid_from, source_app),
constraint entity_edges_no_self check (from_entity <> to_entity),
constraint entity_edges_dates check (valid_to is null or valid_from is null or valid_to >= valid_from),
constraint entity_edges_decided check ((status = 'suggested') or (decided_at is not null))
);
create index entity_edges_from_idx on webster.entity_edges (from_entity, predicate, status);
create index entity_edges_to_idx on webster.entity_edges (to_entity, predicate, status);
A trigger mirrors entity_links_kind_guard from 0001: the two endpoints'
entities.kind must satisfy the predicate's from_kind/to_kind, checked in
the database and not in the service.
Edge status follows the same bar as links. An edge derived from a CRM field
the books already agree on (LP Flow's local_persons.local_org_id, DealFlow's
investor lists) lands confirmed — it is a copy of a fact a human already
entered. An edge extracted from prose lands suggested and joins the review
queue. There is no third tier and no model-only confirm.
Audit: link_events is extended by an additive migration to accept an
edge_id column (nullable, no FK, same as link_id), so one append-only table
remains the audit for every decision the engine records. This is the one place
an existing table is touched, and it is an ADD COLUMN ... NULL — no rewrite,
no default, no lock beyond the catalog update.
5 · Entity creation — propose, then approve
Webster has never created an entity. It must now, because a mention of a firm nobody has ever filed is the common case, not the edge case. The rule that protects the registry is that creation is a proposal until a human accepts it.
create table webster.entity_proposals (
id uuid primary key default gen_random_uuid(),
kind text not null check (kind in ('person','org')),
proposed_name text not null,
name_norm text generated always as (webster.norm_name(proposed_name)) stored,
org_name_norm text generated always as (webster.norm_org_name(proposed_name)) stored,
emails text[] not null default '{}',
domains text[] not null default '{}',
linkedin text,
title text,
firm text,
fields jsonb not null default '{}'::jsonb, -- everything else the proposer knows
-- why we think this is new
reason text not null, -- 'no_candidate' | 'all_candidates_rejected' | 'requested'
near_misses jsonb not null default '[]'::jsonb, -- [{entity_id, score, matched_by}] — shown in the queue
mention_id uuid references webster.mentions(id) on delete set null,
status text not null default 'pending'
check (status in ('pending','approved','rejected','superseded')),
entity_id uuid references webster.entities(id), -- set on approval
decided_by text,
decided_at timestamptz,
decision_note text,
source_app text not null,
as_of timestamptz not null,
proposed_by text not null, -- extractor or actor
policy_version int, -- §7
created_at timestamptz not null default now(),
constraint entity_proposals_decided check ((status = 'pending') or (decided_at is not null)),
constraint entity_proposals_approved_has_entity
check ((status = 'approved') = (entity_id is not null))
);
create unique index entity_proposals_open_unique
on webster.entity_proposals (kind, coalesce(org_name_norm, name_norm))
where status = 'pending';
entity_proposals_open_unique is what stops the queue from filling with 40
copies of the same unfiled firm from 40 documents: the second proposal for the
same normalized name attaches its evidence to the open one instead of creating
a row. That is a service-side upsert, and the index is the guarantee it holds
under concurrency.
Approval is a service transaction, never a trigger: insert the entity,
insert the field values with their provenance, resolve every mention whose
subject_norm matches and whose entity_id is null, write the link_events
row with the actor. Reversible: rejecting later archives the entity rather than
deleting it, so the audit survives.
Duplicates get proposed too, never merged. Entity creation makes in-registry duplicates possible for the first time, so the same propose-then-approve rail covers merges:
create table webster.merge_proposals (
id uuid primary key default gen_random_uuid(),
keep_entity uuid not null references webster.entities(id) on delete cascade,
merge_entity uuid not null references webster.entities(id) on delete cascade,
reason text not null, -- 'shared_key' | 'near_name_same_kind' | 'requested'
evidence jsonb not null default '{}'::jsonb,
score numeric(4,3) not null default 0,
status text not null default 'pending'
check (status in ('pending','approved','rejected')),
decided_by text, decided_at timestamptz, decision_note text,
as_of timestamptz not null,
created_at timestamptz not null default now(),
constraint merge_proposals_distinct check (keep_entity <> merge_entity),
constraint merge_proposals_pair unique (keep_entity, merge_entity)
);
Approving a merge uses the lifecycle 0001 already built for it —
entities.status = 'merged' + merged_into — and never deletes a row.
6 · Match assessment (AI on the queue, never past it)
The 1,413 open suggestions break down as: 736 org name matches at 0.65, 503 person name matches at 0.55, 149 single-token org matches at 0.45, 25 name-and-firm at 0.75. The 0.55–0.75 band is 1,264 of them — the band the brief targets.
create table webster.link_assessments (
id bigserial primary key,
link_id uuid not null, -- entity_links.id; no FK, assessments outlive links
edge_id uuid, -- same layer, reused for edge suggestions
verdict text not null check (verdict in ('same','different','unsure')),
rationale text not null, -- one line, shown in the queue
model_confidence numeric(4,3) not null
check (model_confidence >= 0 and model_confidence <= 1),
-- Reproducibility. An assessment that cannot say what produced it
-- does not get stored — hence NOT NULL on all four.
model text not null, -- 'claude-haiku-4-5'
prompt_version int not null,
policy_version int not null references webster.matching_policy(version),
input_digest text not null, -- sha256 of the exact rendered comparison
cost_usd numeric(10,6),
latency_ms int,
created_at timestamptz not null default now(),
constraint link_assessments_rerun unique (link_id, model, prompt_version, policy_version)
);
create index link_assessments_link_idx on webster.link_assessments (link_id, created_at desc);
create index link_assessments_verdict_idx on webster.link_assessments (verdict, model_confidence desc);
input_digestis what makes "re-runnable" true rather than aspirational. If the snapshot refreshes and the two records now read differently, the digest changes and the stored verdict is visibly stale rather than quietly wrong.- The unique key is the re-run key. Same link, same model, same prompt, same policy → one row. Change any of the three versions and you get a new row alongside the old one, which is how precision-over-time (§9) is measured at all.
- Fail closed, folded in from Terminal.
aiMatch.tsreturns{same:false}on any parse or API failure and logs. In the engine, a failure stores no row and the suggestion stays unassessed — an absent assessment and adifferentverdict must not look alike to the reviewer.
The queue sorts and groups by (verdict, model_confidence desc) so a reviewer
can bulk-accept a block of same at high confidence, and unsure sinks to
where it belongs. decide() is unchanged — same endpoint, same actor,
same link_events audit. The model's only power is ordering.
The review-queue UI change (verdict grouping, bulk-accept-by-verdict) is a real visual change and gets a published HTML mockup before it is built. Not part of this stop point.
7 · Matching policy as configuration
Today the tier weights live as literals in service/lib/registry.ts
(1.000 email, 0.980 linkedin, 0.950 domain, 0.750 name_and_firm,
0.650/0.550 name) and in scripts/seed.sql (0.45 single-token org).
Terminal has its own copy in matching.ts CONFIG (acceptThreshold: 3,
highConfidence: 6, reviewFloor: 1). LP Flow has a third in
intel-matching.ts (0.99 / 0.95 / 0.55 / 0.7). Three matchers, three
constants files, three deploys to change a threshold.
create table webster.matching_policy (
version int primary key,
weights jsonb not null, -- {"email":1.0,"linkedin":0.98,"domain":0.95,
-- "name_and_firm":0.75,"name_org":0.65,
-- "name_person":0.55,"name_org_single_token":0.45}
thresholds jsonb not null, -- {"auto_link":1.0,"ambiguous":[0.55,0.75],"floor":0.40}
options jsonb not null default '{}'::jsonb, -- free-mail list overrides, token rules
note text,
is_active boolean not null default false,
created_by text not null,
created_at timestamptz not null default now()
);
create unique index matching_policy_one_active on webster.matching_policy (is_active)
where is_active;
Version 1 is seeded with exactly today's literals, so activating it is a
no-op and the first migration proves the mechanism without moving a single
score. Every link_assessments row and every auto-link records the
policy_version that produced it, so changing a threshold is a settings edit
plus an optional re-run — not a deploy, and not a silent re-interpretation of
history.
The one thing policy may not express is a bar that auto-confirms a name
match. auto_link is validated at write time against the exact-key tiers only.
That rule is in the schema, not in a code review.
8 · How Ford's intake_links maps on
Ford keeps its table. It is Ford's own ledger of what its intake attached to, it is already live, and the brief's rule is that nothing in a consumer app gets rewritten. The backfill copies forward.
ford.intake_links |
Spine | Notes |
|---|---|---|
message_id |
mentions.doc_id, with source_app='ford', doc_kind='intake_message' |
joined to ford.intake_messages for doc_title and the real as_of |
entity_kind = 'firm' |
mentions.subject_kind='org', role='publisher' |
Ford's firm is always the publisher of a newsletter |
entity_kind = 'company' |
mentions.subject_kind='org', role='subject' |
|
entity_kind = 'sector' |
tag_assignments, subject_type='doc', kind='sector' |
not a mention — see §2 |
name |
mentions.subject_name |
entity_id left null; resolution is a separate, reviewable step |
created_at |
mentions.created_at |
as_of comes from the message, not from this |
| — | extractor = 'ford/classifier@1' |
so a re-extraction is distinguishable |
Volume today: 11 links across 3 messages. This backfill is the cheap one and goes first precisely because it is small enough to eyeball end to end.
Ford's forward path (Phase 2, not this session): spineLinks() in
packages/intake/src/pipeline.ts gains one additional call that POSTs the same
links to the engine. It keeps writing intake_links — dual-write, because a
door that depends on a second service being up is a worse door.
9 · How the three matchers call it
The audit's headline risk is matcher fragmentation: "the spine must become the one matcher everyone calls." Concretely, per app:
Terminal — artifacts/api-server/src/lib/{matching,aiMatch,dealflow}.ts
Today: a weighted signal vote (name 3, domain 6, investors ≤8, sector 2, with
negative signals) → accept | review | reject; the review band goes to a
Haiku tiebreaker that fails closed; results cached in dealflow_links with a
manual override.
matching.tsstays. It is the best-designed of the three — pure, I/O-free, a signal registry with declared weights — and it scores a newsletter deal against DealFlow search results, which is a DealFlow-shaped question the registry has no better answer to.- What changes is where the entity comes from. Before scoring, Terminal
calls
POST /api/resolve {kind:'org', name, domain, ...}. A confirmed registry link short-circuits the whole vote — that is the "registry joins" gap in the audit closed for Terminal. aiMatch.tsretires into the engine. Its system prompt, its fail-closed posture and itstrackedCreatecost accounting become the assessment layer's first prompt version. Terminal callsPOST /api/assessinstead of holding its own Anthropic client for this. One assessment brain.- Literature mentions become spine mentions. Each
literature_company_mentionsrow is a mention withrole='mentioned',passage=quote, andextra={sector,stage,sentiment,fit,vertical,region};sectoralso becomes atag_assignment. Terminal's table is unchanged.
LP Flow — artifacts/api-server/src/lib/intel-matching.ts
Today: five tiers (exact normalized name 0.99 → proposal alias 0.95 → LP
profile alias 0.95 → trigram Jaccard ≥0.55 → contact-email domain 0.70), plus
org_match_exclusions (learned negatives) and person_name_variants (learned
name variants). Nothing auto-applies.
- The tiers stay local — they resolve to a LP Flow
local_organizationsrow, which is LP Flow's own question. - The Workbench gains a registry tier, inserted above the fuzzy tier: if
the extracted org resolves to a registry entity that has a confirmed LP
Flow link, that local id is the answer at registry confidence, with the
match reason
registry. org_match_exclusionsbecomes a spine signal, not just a local one. A reviewer's "this is NOT that org" is the highest-value negative in the fleet and today it is trapped in one app. The engine reads it (read-only, from the snapshot) as a rejection prior for the same normalized name. LP Flow's table keeps working exactly as it does.person_name_variantsseedsentity_field_valuesas additionalnamerows withsource_app='lp_flow'— which is what that table already is, in the registry's own vocabulary.
DealFlow — the thinner Workbench clone
Least code, so it gets the registry tier first and keeps the least local matching. Same shape as LP Flow: registry tier above fuzzy, learned negatives read as priors, nothing auto-applied.
Podcast — the private registry
Podcast1's entities table (kind, normalized_key, alias accumulation,
mention_count) is a well-built registry that no other app can see, and
entity_mentions is already a mention store in everything but name.
- Podcast's entities become
entity_proposals, one per row, carryingaliasesas near-miss evidence. Approving one creates a registry entity and a confirmedpodcastlink. Nothing is written back into Podcast. entity_mentionsbecome spine mentions —doc_kind='episode',rolefrom the existingrolecolumn,passagefromcontext,extra={sentiment},locatorreserved for a timestamp when the pipeline starts emitting one.- Podcast's own tables, cron, and normalizer are untouched. The brief's "do not touch" list covers its ingestion pipeline, and this respects it: the spine reads a snapshot, exactly as it does for the CRMs.
10 · Migrations
Additive only, numbered after 0005, pg_dump immediately before each apply,
backup path recorded in STATUS.md. Each ships with a read-back script
asserting the new tables exist and that row counts on all five existing
tables are unchanged.
| # | File | Contents |
|---|---|---|
| 0006 | spine_mentions.sql |
mentions, indexes, v_entity_mentions, RLS + revoke |
| 0007 | spine_tags.sql |
tags, tag_aliases, tag_assignments, vocabulary seed |
| 0008 | spine_edges.sql |
edge_predicates + seed, entity_edges, kind-guard trigger, link_events.edge_id |
| 0009 | spine_proposals.sql |
entity_proposals, merge_proposals |
| 0010 | matching_policy.sql |
matching_policy + version 1 seeded to today's literals |
| 0011 | link_assessments.sql |
link_assessments, queue view with verdict grouping |
0010 precedes 0011 because link_assessments.policy_version references it.
11 · API surface (Phase 1)
Same service, same two credential tiers, same conventions as
resolve/entity/suggestions/decide/peek.
| Method | Path | Scope | Notes |
|---|---|---|---|
POST |
/api/mentions |
write | batch upsert, idempotent on the identity key; returns resolved/unresolved counts |
GET |
/api/mentions |
read | by entity_id, or by app+doc_id; visibility-filtered |
GET |
/api/tags |
read | vocabulary + aliases; the crosswalk any app can read |
POST |
/api/tags/resolve |
read | free string → canonical tag, or null (never invents one) |
POST |
/api/edges |
write | suggested by default; confirmed only for exact-key derivations |
GET |
/api/edges |
read | by entity, predicate, date range |
POST |
/api/entity-proposals |
write | dedupes onto an open proposal |
POST |
/api/entity-proposals/:id/decide |
decide | approve creates the entity + link + audit |
POST |
/api/merge-proposals/:id/decide |
decide | approve sets merged, never deletes |
POST |
/api/assess |
decide | runs or returns a cached assessment; cannot change a status |
GET |
/api/policy · POST /api/policy |
read / decide | read the active policy; publish a new version |
Every write endpoint requires source_app, as_of and extractor, and 400s
without them. Provenance is enforced at the schema level and refused at the
door, because a nullable column nobody fills is not provenance.
12 · What this design deliberately does not do
- No
dewey.*change and no Dewey code. The spine points at Dewey pages by(document_id, idx); Dewey never learns the spine exists. - No writes into any app database. Not LP Flow, not DealFlow, not Terminal, not Podcast, not Ford's own tables beyond its existing ledger.
- No auto-merge, ever. Duplicates surface as proposals.
- No model-confirmed link and no model-confirmed edge. The assessment layer sorts a queue.
- No embedding-based matching. The audit found matcher fragmentation, not matcher weakness; adding a fourth scoring method before the three existing ones agree would make it worse. Deferred until the eval gate says the deterministic tiers are the limit.
- No consumer-visible change. Everything downstream ships behind a flag that defaults off, after the eval gate.
13 · Open questions for Shawn
Three, and only the first blocks the migrations.
Ford's unmatched sector strings. The classifier emits free text. My recommendation: alias what matches the seeded vocabulary, and hold the rest as vocabulary proposals in the queue rather than auto-creating tags — a controlled vocabulary that any model can extend is not controlled. That means Ford's tag coverage starts partial and improves as you approve. The alternative is auto-create-and-prune later, which is faster now and messier forever. Blocks
0007.Podcast entity volume. Podcast's private registry is the largest source of
entity_proposalsand every one needs a human. If that turns out to be hundreds of rows, do you want them queued in full, or gated to entities with a mention count above some floor (say, mentioned in ≥2 episodes) with the long tail held? Answerable after the 50-item sample at stop point 2 — not blocking now.Who may approve an entity proposal.
decide()today takes anactorstring and the review-queue token is single-tier. Entity creation is a heavier action than confirming a link. Same token, or a separate one? My recommendation: same token for now — the audit trail already records the actor, and a second credential tier is real overhead for a one-reviewer registry. Revisit when someone other than you reviews. Not blocking.
14 · Decisions recorded here
Copied into slw-webster/docs/DECISIONS.md when the first migration lands.
- 2026-08-07 — Spine tables live in the
websterschema, not a new one. The normalization functions that back generated columns are there, and RLS is a per-object surface worth stating once. "Data Engine" is a product concept; no consumer sees a schema name. - 2026-08-07 — A mention stores a name always and an entity id when known. Generalizing Ford's names-not-FKs decision. An unresolved mention is a normal state and still answers "everything about X"; requiring resolution at write time would discard the evidence needed to resolve it.
- 2026-08-07 — Sectors are tags, not mentions. Ford's third
entity_kindmoves totag_assignments. A mention's subject must be a countable person-or-org that can carry anentity_id. - 2026-08-07 — Apps keep their own tag strings forever; the spine holds a
crosswalk. Terminal's slugs are in a prompt and in history; Podcast's file
says renaming orphans rows.
tag_aliasesis unique per(source_app, alias)because the same word means different things in different books. - 2026-08-07 — Entity creation is propose-then-approve, with one open proposal per normalized name. The partial unique index is what keeps 40 documents naming the same unfiled firm from becoming 40 queue rows.
- **2026-08-07 — An assessment records model + prompt version + policy version
- input digest, or it is not stored.** A verdict that cannot say what produced it is not evidence. A failed assessment stores nothing, so "unassessed" and "judged different" never look alike.
- 2026-08-07 —
matching_policyv1 is seeded to today's exact literals. The first migration proves the mechanism without moving a score. - 2026-08-07 — Edges are dated and their endpoint kinds are enforced in the database. "Was a partner at" and "is a partner at" are different facts, and a person cannot be an investor in a person.