1. What the system produces
The system maintains a versioned, evidence-backed model of a brand's material requirements and relates those requirements to a mill's documented capabilities. It answers a defined set of business questions, shows the limits of each answer, and identifies the next evidence that would improve it.
The core loop follows the shared design conversation ↗: raw evidence → atomic observations → bounded Jev judgments → provisional states → maturity evaluation → preserved states → higher-level states → mill opportunities. New evidence can strengthen, narrow, challenge, or supersede any conclusion.
Preserving a state means saving a reproducible conclusion and its justification. It never means freezing that conclusion as permanent truth. Every state version, including provisional and rejected versions, is retained. Maturity determines eligibility for use, not whether a record deserves storage.
This document specifies the target design. The current state-next app is a published bilingual design reader; its state engine, database, workers, and operational screens are not implemented. Numerical policies below are proposed starting values, not measured performance or findings about Mammut.
Related specifications:
- 64 business questions: the questions that determine the ontology and work priorities.
- Contracts and policies: record shapes, decision packs, lifecycle rules, and worked example.
2. The main design decisions
- Business questions drive collection and aggregation. Every maintained claim serves at least one registered question and a defined scope.
- Code handles arithmetic and hard constraints. Jev handles semantic judgments that cannot be answered reliably by parsing or database queries.
- Separate four concepts: observed facts, inferred claims, forecasts, and commercial recommendations. Each has a different evidence requirement.
- Separate model output from validated reliability. Store raw Jev distributions, evidence coverage, policy outcomes, and evaluated error rates as different fields.
- Use independent evidence roots. A retailer's copied description and a brand page may be one source of a claim. Ten child states derived from one source do not create ten confirmations.
- Treat unknown as a first-class result. Missing composition is not zero polyester; missing orders are not zero demand; unobserved products are not discontinued products.
- Update through events and immutable versions. A changed source invalidates dependent conclusions, including already generated answers.
- Begin with a relational evidence graph. Tables and explicit dependency edges support audit and recomputation; a separate graph database is unnecessary for the first version.
- Keep the decision engine replaceable. A provider adapter separates Jev's transport contract from the domain model.
- Start with one complete question path. Prove source → state → answer → correction before expanding the question catalog.
3. System map
- 01
Collect evidence
ERP · product history · reviewsKeep immutable sources and collection gaps.
- 02
Resolve observations
Identity · component · timeNormalize facts without losing their origin.
- 03
Make bounded judgments
Jev · Choice / Noul / ScoreEvaluate meaning; compute quantities in code.
- 04
Evaluate maturity
Coverage · conflicts · validationApply explicit policies and preserve unknowns.
- 05
Preserve & compose
Product → family → category → brandVersion conclusions and carry their lineage.
- 06
Find mill opportunities
Demand × verified capabilityAnswer with evidence and a concrete next step.
Full dependency diagram source
flowchart TD
Q[Business question registry] --> PLAN[Scoped evidence plan]
ERP[ERP records and purchase documents] --> ING[Source adapters and immutable snapshots]
CAT[Retail products and historical snapshots] --> ING
REV[Consumer reviews] --> ING
MILL[Mill capabilities and verification documents] --> ING
PLAN --> ING
ING --> NORM[Normalize units, entities, time and material components]
NORM --> OBS[Atomic observations and lineage]
OBS --> METRIC[Deterministic cohorts, counts and coverage]
OBS --> PACK[Bounded evidence packets]
PACK --> JEV[Jev decision packs]
JEV --> VAL[Validate typed responses and retain uncertainty]
METRIC --> CAND[Candidate claim versions]
VAL --> CAND
CAND --> GATE[Versioned maturity policies]
GATE --> STORE[State versions and dependency graph]
GATE --> GAP[Evidence gaps and analyst review]
GAP --> PLAN
STORE --> HIGH[Family, category and brand states]
HIGH --> GATE
STORE --> MATCH[Mill capability matching]
MATCH --> ANSWER[Scoped answers and opportunities with citations]
STORE --> ANSWER
ING --> CHANGE[Change detection and invalidation]
CHANGE --> STORE
ANSWER --> OUTCOME[Verified outcomes and corrections]
OUTCOME --> EVAL[Evaluation and calibration]
EVAL --> GATEThe drawing describes a feedback loop over time. Within a single computation, derivation dependencies must form a directed acyclic graph. A parent conclusion cannot be used as independent evidence to prove one of its own children.
4. Business scope and boundaries
The first release supports descriptive material mix, historical assortment trends, review-derived requirement hypotheses, observed purchasing relationships, capability matching, and ranked investigation opportunities. Each answer declares whether it describes observed retail assortment, observed ERP activity, an inference, or a forecast.
A brand's retail product mix does not measure its purchase volume. Retail price does not identify fabric cost. A compatible ERP fabric does not establish that it was used in a particular product. Recycled content does not establish certification. A review complaint does not establish which manufacturing stage caused it. These are separate claims with separate proof requirements.
The system improves its stored knowledge through corrections and independent observations. Updating Jev's weights is outside the first release. A later small classifier may learn from verified outcomes and Jev-derived features, but its quality must be evaluated separately.
5. Source inventory and integration strategy
Repository inspection found the following integration points. The adjacent project documentation describes their contents; this design has not re-audited the underlying corpus or confirmed its completeness.
| Existing component | Proposed use | Boundary |
|---|---|---|
labs/prod_jev/data/MANIFEST.json | Starting inventory of brand, catalog, reviews, supply, market, ERP, and factory datasets | Verify actual files, hashes, dates, access classification and coverage on import |
labs/prod_jev/out | Historical Jev decision runs and replay examples | Model output is a derived artifact, never a primary source |
../jev-mammut-next/scripts/build-data.mjs | Reference for locating and interpreting existing source datasets | Its aggregated canvas snapshot is a display projection, not the canonical evidence store |
../jev-mammut-next/lib/loop.ts | Candidate → judgment → independent confirmation loop and pair types | Existing grade labels require review for each intended task |
../mcq-mammut-next/lib/jev.ts | Reference for server-side Jev request construction | Add schema validation, durable jobs, version tracking and abstention |
../mcq-mammut-next/content/questions.ts | Reading-comprehension regression cases | A small MCQ result does not validate maturity judgments or future predictions |
| Mammut history, review corpus and purchase-document sources referenced by the canvas builder | Raw product versions, review links and documented fabric relationships | Ingest through adapters that preserve record-level lineage |
The adjacent canvas README reports overlapping catalog populations, unresolved retailer-to-brand joins, and incomplete certification evidence. Treat these as initial audit tasks rather than silently importing the display totals into one population.
Each adapter emits a source manifest: dataset ID, owner, origin, source type, schema version, content hash, snapshot time, declared time coverage, known collection gaps, record count, root-origin identity, and access restrictions. An import can finish successfully while coverage remains incomplete; these are different statuses.
Import raw data into private storage. Keep source collection separate from reasoning. Start by importing existing authorized snapshots; designing this framework does not require live Jev calls or transmitting ERP data.
6. Ontology: what we need to identify
| Entity | Identity and important distinctions |
|---|---|
| Brand | Canonical brand ID; legal and trading entities remain separate related entities |
| Product family | A longitudinal design family; name similarity alone cannot establish continuity |
| Product model/version | Style or article ID plus materially distinct revision; revision can change fabric |
| SKU/variant | Size, color and market variations; counting unit is explicit for each metric |
| Product snapshot | Source × product identity × market × capture time; observed date differs from release date |
| Material component | Face, backer, membrane, lining, insulation, reinforcement, trim; distinguish blended and layered constructions |
| Fabric specification | Fiber composition, weave/knit, yarn, denier, weight, width, stretch mechanism, finishing; values carry units and basis |
| ERP fabric item/version | Supplier, trading code, customer code and effective dates; preserve suffixes until equivalence is proven |
| Business relationship | Buyer, maker, mill, brand programme and documented relation; no transitive inference of purchase by default |
| Review | Original review identity, source, product reference, language, date and duplicate group |
| Capability | Mill × process × specification range × verification date × evidence |
| Requirement | Scoped product/material need; distinguish stated requirement from inferred consumer preference |
| Claim/state | A proposition with scope, dependency set, quality assessment and immutable versions |
| Opportunity | A proposed mill action backed by demand-side and capability-side states, assumptions and validation steps |
Identity resolution follows: authoritative IDs → normalized exact mappings → constrained candidate generation → bounded semantic evaluation → explicit unresolved/review outcome. Retain proposed links separately from accepted links. A relation is typed: same_product, successor_of, variant_of, compatible_with, purchased_as, and used_in are not interchangeable.
Do not import a loose match threshold from an exploratory visualization as the production identity policy. Entity links need their own held-out precision tests and a way to reverse merges without losing original records.
7. Evidence model
Atomic observations
An observation expresses one source-backed statement: subject, predicate, value, component, units, valid time, observed time, source locator, extraction method, and uncertainty. For example, a parsed source field can assert that a particular model's face fabric is 100% polyester. The same record does not automatically assert that it is recycled, woven, or sourced from a specific mill.
Store the original wording beside the normalized value. A locator must resolve to a field path, page and bounding region, or text span in an immutable source snapshot. OCR and generated extractions are provisional until their field quality passes the applicable policy.
Four evidence roles
| Role | Meaning | Permitted use |
|---|---|---|
| Primary record | Product specification, order line, test report, original review, original commercial document | Supports the specific fact stated, within its scope |
| Deterministic derivation | Count, unit conversion, filtered cohort, trend estimate | Supports a reproducible calculation over a named population |
| Semantic inference | Jev classification or relation assessment | Supports an inference with task-specific validation |
| Independent outcome label | Verified match, checked document, lab result, completed commercial outcome | Used for evaluation only when label provenance is independent of the prediction |
A source's authority is claim-specific. An official composition field may establish advertised composition; it does not independently establish tested performance. A purchase order supports a purchase relationship, while a quotation supports quoted availability.
Dependence and duplication
Maintain both originGroupId for shared source ancestry and samplingUnitId for the statistical unit, such as product family. Repeated seasonal observations can demonstrate persistence, but must not be counted as independent corroboration of an unchanged specification. Syndicated reviews and copied product descriptions share origin groups.
When combining child states, take the union of leaf evidence IDs and dependence groups. Keep child versions as dependencies, but do not count both the child and its underlying observations in an evidence denominator. Unknown source dependence is reported and handled conservatively.
Coverage and time
Maintain a coverage matrix by source, market, category, year/season, component, and key field. Preserve missing cells and failed collections. Seven calendar years of files do not necessarily mean seven comparable product cohorts.
Use two times: validTime for when a fact applies and recordedAt for when the system knew it. Support both "what do we now believe about 2023?" and "what could we have answered using only evidence known in 2023?" Late-arriving evidence may revise the former without leaking into the latter.
Normalize units with the measured basis: garment weight is not fabric gsm; finished gsm is not greige gsm; width affects yield; composition percentages belong to a specific component. Missing basis prevents conversion or matching.
8. State hierarchy
| Level | Stored object | Example proposition | How it is created |
|---|---|---|---|
| L0 | Atomic observation | A source reports polyester for this model's face fabric | Source extraction and validation |
| L1 | Product/material state | This product version has a polyester-dominant face fabric | Resolve component and conflicting specifications |
| L2 | Product-family state | A design family retains the same face-fabric platform across observed seasons | Version links plus longitudinal observations |
| L3 | Category state | Polyester-dominant face fabrics increased in an observed insulated-jacket cohort | Deterministic comparable-cohort analysis |
| L4 | Brand strategy hypothesis | Material choices differ systematically by end-use demand | Multiple relevant category states and alternative explanations |
| L5 | Mill fit state | A mill has documented capability adjacent to a scoped requirement | Requirement constraints intersect verified capability |
| L6 | Opportunity state | Develop or sample a specified platform for a stated use case | Demand, fit, feasibility, economics where known, and human judgment |
These examples are illustrative, not Mammut findings. A mature L3 descriptive state can support a provisional L4 inference. The parent's maturity never follows automatically from its children's maturity. The evidence graph permits direct dependencies across levels where needed; it is not a mandatory seven-step ladder for every question.
9. The Jev decision layer
TypeSafe documents three primitives: Choice, Score and Noul. Questions in one request share a state and are evaluated in isolation; a question cannot consume another answer from that same request. Batch independent questions over a bounded evidence packet, and use a later request only for a real dependency. TypeSafe primitives ↗
The API's state is a request payload. Our durable StateVersion is a database record. An evidence packet is the versioned adapter output connecting these two meanings.
| Decision pack | Input | Semantic decisions | Code responsibilities |
|---|---|---|---|
| Identity | Candidate identities and distinguishing fields | Same version / related / different / insufficient | Candidate restriction, ID checks, link policy |
| Material interpretation | Component-scoped source spans and ontology | Fabric family, stated vs inferred attribute, applicability | Parsing, units, value constraints |
| Review interpretation | One original review and known product context | Performance theme, explicitness, possible responsible stage | Deduplication, denominators, join restrictions |
| Evidence relation | A claim and an individual evidence item | Supports / contradicts / qualifies / irrelevant / insufficient | Scope intersection and lineage |
| Claim interpretation | Computed metrics, definitions and counterevidence | Plausible alternative explanation; whether wording overstates support | Counts, coverage, trend estimation, maturity |
| Capability fit | Requirement and verified candidate specs | Qualitative adjacency where a numeric rule is unavailable | Required limits, exclusions, missing mandatory fields |
| Next investigation | Enumerated available actions and known gaps | Which gap a document or review could resolve | Eligibility, utility, cost, queue ordering and budget |
Choice lists include insufficient_evidence and not_applicable where meaningful; candidate matching also includes none_of_candidates. These are different outcomes. For Noul, use a separate applicability/data-availability check so a low probability does not conflate missing evidence with a negative fact.
Score uses anchored ordinal criteria, retaining the distribution and legend. Do not turn an arbitrary ordinal average into a measured physical quantity or business probability.
Input packets contain selected source spans, scope, ontology definitions, explicit missingness, deterministic metrics, and relevant counterevidence. They exclude answer keys, final analyst verdicts and unrelated ERP/customer fields. Retrieval records candidate counts, selection rules, omitted data, and any context truncation.
The provider adapter validates response type, expected IDs, option membership, finite numbers, probability ranges and normalization tolerance. It records requested and resolved model IDs, input hash, question-pack version, provider request ID if available, timestamps, attempts, tokens, and cost basis. Missing or malformed results produce decision_failed, not negative evidence.
10. Maturity and confidence
A mature state passes a policy for a particular claim type and scope. It is sufficiently supported for a declared use at a particular time; it is not guaranteed correct.
Display a quality vector instead of a single invented confidence number:
- Evidence quantity and independent origin coverage.
- Population, field and temporal coverage, with explicit denominators.
- Identity-link quality and component specificity.
- Consistency, counterevidence and unresolved critical contradictions.
- Freshness and dependency health.
- Task-specific held-out evaluation status.
- Sensitivity to deduplication, missing values and cohort choices.
Jev's Choice/Score confidence summarizes the shape of its output distribution; Noul has no separate confidence field. Keep it separate from the selected option's probability and from empirical accuracy on our tasks. TypeSafe confidence ↗
Code computes measurable maturity criteria. Jev can flag ambiguity, relevance, semantic conflict or alternative explanations; it should not estimate counts already known to the database. Do not multiply correlated Jev judgments or average child confidences into a supposedly calibrated probability of truth.
Policies differ by claim family:
| Policy | What it requires |
|---|---|
| F — Scoped fact | Traceable authoritative field/document, accepted extraction and identity, resolved material contradiction |
| D — Descriptive aggregate | Defined observed population, reproducible denominator, field coverage and uncertainty bounds |
| T — Historical trend | Comparable cohorts, minimum time coverage, effect size, missingness and dependence sensitivity |
| R — Review pattern | Original review groups, theme denominator, product-link limits, sampling caveat and no unsupported causality |
| L — Commercial link | Exact document-backed relationship, commercial stage and effective date |
| C — Capability | Verified process/specification range and required current documents |
| O — Opportunity | Usable demand and capability premises, explicit gaps and analyst approval for recommended action |
| P — Prediction | Defined horizon/target, leakage-safe backtests, baseline comparison and empirical calibration |
The first deployment uses shadow evaluation and analyst release for model-dependent states. Automatic maturity is enabled per task only after its evaluation gate passes. A sparse authoritative fact may mature with one document; a broad trend needs a defensible population. Neither uses one universal evidence-count threshold.
11. State lifecycle
Separate assessment, freshness, and version relationship:
assessment: candidate → provisional → mature; contested and rejected are explicit alternatives.freshness: current / stale / invalidated, independent of assessment.supersedesVersionId: creates an immutable version chain; a current-head pointer selects the latest applicable version.
Every transition records its trigger, policy version, gate results, evidence changes, actor, and prior version. A mature-but-stale historical conclusion can still be shown as historical, while being ineligible for a current recommendation.
| Trigger | Result |
|---|---|
| First claim proposal | Persist candidate with scope and supporting sources |
| Valid evidence but incomplete gates | Persist provisional with concrete gaps |
| All applicable gates pass | Persist mature under the policy version |
| Material unresolved contradiction | Persist contested; block dependent recommendations |
| Disproven or invalid proposition | Persist rejected; preserve reason and lineage |
| New evidence changes scope/value | Create new version; rerun affected gates |
| Freshness deadline expires | Mark current use stale; request refresh |
| Source retracted or identity mapping corrected | Invalidate affected versions for serving immediately; rebuild |
| Old period and new period differ | Preserve both time-scoped claims; do not treat genuine change as logical contradiction |
State splitting is explicit: if a claim only holds for women's alpine shells in one market, create the narrower child claim and record why the broader claim failed. A narrower claim must not inherit maturity without reevaluation.
12. Runtime loop and stopping rules
- Select a question and scope. Resolve its metric, required sources, applicable policy and decision use.
- Build a coverage plan. Identify available records, missing mandatory fields and unresolved links.
- Ingest and normalize changed evidence. Preserve original records and produce versioned observations.
- Compute deterministic facts. Cohorts, counts, ratios, time series and exact constraint matches.
- Generate bounded claim candidates. Prefer question templates and computed values. Optional generative extraction may propose structured candidates with citations, never self-certify them.
- Evaluate semantic decisions. Build reproducible packets, call Jev through the adapter and validate the results.
- Evaluate maturity. Apply the claim-specific policy; persist complete gate outcomes, including unknowns.
- Propagate changes. Recompute dependent states in topological order over the affected subgraph.
- Assemble an answer or abstain. Return the supported portion, limitations, and next useful evidence.
- Queue further work only if it can change an unresolved decision. Record why the action is worth its cost.
Stop when the answer is usable, when no independent evidence action remains, when required evidence is inaccessible, when a run budget is reached, or when two consecutive passes make no material evidence/gate change. Rephrasing a question or repeating a call on the same evidence is an experiment, not additional corroboration.
For an initial local pilot, propose at most three evidence-acquisition passes per question and a configurable 100-call ceiling per run, plus a separately configured currency budget. These are runaway-prevention defaults, not capacity estimates. Budget exhaustion returns a partial answer and resume cursor.
Prioritize work using a transparent utility score: business importance × estimated chance of resolving a blocking gate × expected decision impact, divided by estimated cost. Before outcomes exist, label these values as analyst heuristics. A later learned scheduler must be evaluated against the heuristic baseline.
13. Answer and opportunity contract
An answer contains: question ID/version, brand and scope, as-of time, knowledge cutoff, status, concise conclusion, measured result where applicable, cited state versions and leaf sources, coverage, material counterevidence, assumptions, limitations, and the next evidence needed.
Statuses are supported, partial, contested, and insufficient_evidence. supported means the applicable policy passed for the exact wording and scope. It does not mean every aspect of the broad business question is known.
Generate the factual answer from structured results/templates first. Optional language generation can improve readability, but cannot add numbers, sources, relationships or certainty. Validate each sentence's citations and use an analyst review path for new commercial interpretation.
Opportunity matching has two stages:
- Eligibility: enforce mandatory construction, material, width, weight basis, processing and current-document constraints. Separate pass, fail and unknown. An unknown mandatory constraint cannot receive a fully qualified match.
- Ranking: compare eligible candidates on documented material adjacency, end-use evidence, persistence, capability readiness, process dependencies and commercial evidence. Store component scores and configurable weights. A ranking score is a priority measure, not a probability of winning an order.
Greige mills control yarn/construction and base-cloth characteristics. DWR, lamination, dyeing, testing and certification may require downstream partners. An opportunity card names which requirement the mill can address and which depends on a partner or trial.
Every opportunity includes required validation: sample weaving, finished-fabric trials, test methods/targets, width and shrinkage checks, partner availability, and economics when actual values exist. Do not derive a purchase volume from assortment counts. No automatic purchase, outreach or production commitment belongs in the knowledge loop.
14. Application structure
The primary workflow is Question → Answer → State → Evidence, with a secondary Gap → Investigation → Revised answer path.
| Screen | Main job |
|---|---|
| Overview | Show active business questions, supported answers, changes and blocking gaps; avoid a misleading single brand-confidence score |
| Question library | Browse the 64-question catalog; select brand, scope, period, metric and intended use |
| Answer workspace | Read conclusions alongside coverage, counterevidence and links to exact state versions |
| State explorer | Inspect hierarchy, dependencies, lifecycle history and comparison between versions |
| Evidence browser | Inspect original snapshots/spans, normalized observations, duplicate groups and entity links |
| Maturity review | Inspect pass/fail/unknown gates and resolve evidence/identity issues with a reason |
| Opportunity board | Compare mill candidates, hard requirements, unknowns and trial next steps |
| Run inspector | Inspect evidence packets, question packs, raw distributions, retries, usage and failures |
| Evaluation | Inspect verified labels, calibration, acceptance error, coverage and model/policy comparisons |
Use accessible labels in addition to status colors. Make dates, scope, units and missing values visible. Provide English and Taiwan Traditional Chinese display labels, while keeping stable internal IDs and original source wording. The F(SYNC) header mark links to https://fabric-sync.com if the app is added to the platform landing.
15. Technical architecture
Use state-next as the analyst application and server API boundary. Run ingestion, Jev evaluation and graph recomputation in a separate durable worker process. Long historical backfills must not depend on a browser request staying open.
| Component | Recommended initial responsibility |
|---|---|
| Next.js app | Authenticated screens, validated commands, answer queries, run progress |
| PostgreSQL | Canonical entities, observations, claims, versions, dependencies, gate runs, jobs and audit events |
| Private object store | Immutable raw snapshots, documents, full request/response artifacts; local filesystem adapter for development |
| Durable job worker | Import, extract, resolve, compute, evaluate, propagate and materialize answers |
| Transactional outbox | Publish work only after the corresponding state change commits |
| Provider adapter | Jev calls, schema validation, budget/rate limiting, redaction and telemetry |
| Search projection | Text and structured filters first; optional embeddings for candidate retrieval only |
| Evaluation runner | Frozen fixtures, model/policy comparisons, calibration and regression reports |
This is the proposed engine deployment design. Only the bilingual design reader is hosted; engine infrastructure has not been provisioned. Hosting choices can follow the existing platform when implementation starts without changing domain contracts.
Proposed module boundaries:
app/ analyst routes and API handlers
domain/ entities, observations, claims, policies, answers
adapters/sources/ corpus, history, reviews, ERP, documents, capabilities
adapters/decisions/ provider-neutral interface and Jev adapter
engine/ planning, metrics, aggregation, maturity, invalidation
worker/ job handlers, leases, retries and outbox processing
storage/ repositories, migrations and object-store adapters
question-packs/ versioned semantic decisions and rubrics
evaluation/ labels, splits, fixtures, calibration and reports
docs/ architecture and implementation contractsProposed commands and query routes:
| Route | Contract |
|---|---|
GET /api/questions | Versioned registry and supported scope dimensions |
POST /api/runs | Question IDs, scope, cutoff, budget, policy and idempotency key → durable run ID |
GET /api/runs/:id | Progress, cursor, blocking errors, usage and artifacts |
GET /api/answers/:id | Exact answer snapshot; dependencies checked before current serving |
GET /api/states/:id | Current head plus access to immutable versions |
GET /api/evidence/:id | Authorized source metadata and cited span |
POST /api/reviews | Evidence correction/link resolution with rationale and expected revision |
POST /api/recompute | Scoped reevaluation request, reason, expected head and budget |
Do not expose a route that simply sets mature=true. Analyst actions supply a verified fact or explicit signed override; the policy engine produces the resulting state. Overrides retain expiration and excluded automatic-use modes.
16. Reliability, access and operations
Use at-least-once job delivery with idempotent writes. Derive evaluation identity from packet hash, question-pack version, requested/resolved model, ontology version and provider configuration. A model alias may change its resolved version; do not reuse cached decisions across that change without a recorded policy.
Jobs use leases, attempt counters, bounded retries, jitter and a dead-letter/review queue. Retry transient timeouts/rate limits within budget; do not retry invalid credentials or malformed requests indefinitely. An ambiguous provider timeout may have incurred cost; record it rather than claiming exactly-once billing.
Each state update uses an expected prior-head revision and transaction. If another worker has advanced the state, reload and recompute rather than overwriting. Commit new version, dependencies, gate results and outbox event atomically.
Use explicit brand/organization access scope on records and API queries. Keep ERP amounts, customer identities and private documents out of public bundles and logs. Apply a provider-boundary field allowlist; store redaction metadata. Sending private source material requires the project's data-sharing policy to permit that provider and purpose. Public source text is treated as data, never instructions for tools or state changes.
Immutable logical history still respects source withdrawal and retention rules. Revoked or deleted source payloads leave a minimal audit tombstone where permitted and invalidate affected claims; do not keep prohibited raw data merely to preserve reproducibility.
Track ingestion lag, coverage gaps, unresolved identity links, candidate and mature counts, invalidation backlog, acceptance error, abstention, token usage, cost per usable answer, queue age, retries and p95 processing latency. Compare model and policy versions in shadow mode before changing active behavior. Retain previous versions for rollback; never restore an invalidated source as part of rollback.
Back up database and private artifacts, test a restore and replay an import before production release. Separate development fixtures from real commercial data.
17. Evaluation and learning
Maintain different evaluations for extraction, entity resolution, semantic classification, claim maturity, answer fidelity and opportunity usefulness. High MCQ accuracy on a fact sheet is evidence only for that narrow reading task.
Build verified labels with original provenance and reviewer identity. Two Jev calls agreeing is a reproducibility result. Agreement with an existing model-generated atlas is a comparison to that atlas. Neither alone is independent ground truth. Where reviewers disagree, adjudicate or preserve the disputed label instead of forcing agreement.
Split by product family, shared source lineage, and time to prevent leakage. For future-facing tasks, train/tune on earlier knowledge and evaluate on later outcomes; exclude documents recorded after the prediction cutoff even when they describe an earlier period. Keep a frozen holdout separate from threshold tuning. Add another brand only after the Mammut path works, then measure transfer directly.
Report precision/recall and abstention for accepted identity links; Brier score and reliability curves for probabilities where labels support them; false-mature rate and answerable coverage for policies; citation and numerical fidelity for answers. Report sample counts and uncertainty intervals, with resampling clustered by the relevant family/origin unit. Model distributions are not substitutes for empirical validation.
Initial proposed release gates are deliberately task-specific: at least 98% accepted-link precision for identity and no more than 5% false-mature rate for descriptive claims, with one-sided 95% uncertainty bounds satisfying the targets. These are design targets for review, not achieved metrics. Small or unrepresentative samples keep the task in analyst-reviewed mode even if point estimates look good. Higher-impact commercial action remains analyst-led.
Compare code-only, Jev-assisted, and reviewed baselines. Test missing inputs, contradictory evidence, copied descriptions, option-order changes, multilingual reviews, changed identities, source retraction and stale dependencies. Calibration or a meta-classifier can be added only when a held-out improvement is demonstrated. Features may include Jev judgments; labels must come from independently checked outcomes.
18. Delivery sequence and acceptance
| Phase | Deliverable | Exit condition |
|---|---|---|
| 0 — Data and question audit | Source manifests, coverage matrix, entity keys, prioritized 10 questions | Each pilot question has a defined population, answer contract, source path and explicit gaps |
| 1 — One complete path | Q09 material mix: source observations → aggregate → state → answer → correction | Replaying the same import creates no duplicate facts; every number traces to source records; changing one source updates the answer |
| 2 — Decision and review loop | Jev adapter, relation pack, task evaluation, provisional/mature/contested policies | Unknown or malformed output cannot mature a claim; analyst-reviewed acceptance is measured |
| 3 — Historical and review states | Q12 trend, Q25 complaints, family identity and longitudinal coverage | Duplicates do not inflate evidence; scope/time conflicts are handled; cohort sensitivity is shown |
| 4 — ERP and mill fit | Q41 purchased vs quoted, Q49 capability, Q57 opportunity | Compatibility is never displayed as confirmed use; unknown mandatory capability blocks qualification |
| 5 — Incremental operation | Durable jobs, budgets, invalidation, replay, access controls and restore | Source retraction immediately blocks affected current recommendations; failed runs resume safely |
| 6 — Broader knowledge and learning | Remaining catalog, cross-brand trial, verified-outcome learning | Held-out improvement over the frozen baseline; no loss of provenance or increase in unsupported assertions |
The first pilot should answer a small number of questions well before running all 64. Suggested order: Q09, Q10, Q12, Q25, Q30, Q41, Q42, Q49, Q57 and Q62. Forecasts and programme attribution remain constrained by the evidence they require.
19. Design assumptions to validate during phase 0
- Historical product files contain enough stable identifiers and comparable captures for family-level longitudinal analysis.
- ERP access can distinguish quote, sample, bulk order, shipment and return, with units and programme evidence.
- Raw purchase-document references can be retained privately and retrieved by authorized analysts.
- A textile reviewer can adjudicate the initial identity, material and contradiction samples.
- Mill capability documents distinguish greige production from downstream finishing and current certification scope.
- Jev usage, retention and data-sharing arrangements are suitable for the fields selected for evaluation.
- Required refresh frequency and business tolerances are chosen per question rather than globally.
Failure of an assumption narrows what the system can answer; it does not justify filling a gap with model certainty.
20. References and attribution
The state hierarchy and evidence-accumulation objective come from the shared ChatGPT conversation ↗. The database design, policies, question catalog, operational rules and delivery plan here are proposed engineering decisions.
Provider behavior was checked against TypeSafe introduction ↗, primitives ↗, and confidence ↗ on 2026-09-22. No vendor performance or calibration claim is used as an achieved system guarantee.
Repository context: canvas README ../../jev-mammut-next/README.md, canvas data builder ../../jev-mammut-next/scripts/build-data.mjs, existing loop ../../jev-mammut-next/lib/loop.ts, MCQ README ../../mcq-mammut-next/README.md, and Jev adapter reference ../../mcq-mammut-next/lib/jev.ts.