CM/CSM Product Vision: Client Schema Management and Delivery
**MANDATORY AGENT READ.** Any agent working on CM, CSM, `/client-schema`, ADAC,
mapfiles, fieldcodes, MetaConnex, Attribute Manipulator, macros, or schema
mapping must read this document before doing any work. It is the single authoritative
answer to "what am I trying to do here?" Do not guess. Do not ask Abdullah to
re-explain the mapping chain. The answer is in Sections 3, 4, and 9 of this document.
> **New to this? Start visual.** Open `docs/CSM_Macro_Vision_Explainer.html` in a browser —
> it walks the whole pipeline and the 12d macro panel in plain language with real data.
> For the macro specifically (one fixed macro, two file boxes, one tabbed panel), the
> authoritative spec is `docs/MACRO_ARCHITECTURE_AND_PLAN.md` — read its **§00 "at a glance"**
> first. This vision doc remains the source of truth for the *why* and the mapping chain.
For the relationship between this file and the older
`Client_Attr_Mapper\VISION_WORKSHOP.md` / `VISION_QA_LOCKED.md` files, see
`docs/vision_file_map.md`. Short version: this CSM vision doc is current and
authoritative; the CAM vision files are historical source material.
Pipeline in Plain English (30-second read — for every agent)
> Confirmed by Abdullah 2026-05-30. The full vision has more detail below, but this is the shape of the whole pipeline; read it before touching the app.
- **CSM is the LINK and the BRAINS.** It links each **client code** (e.g. "manhole cover") → a **Veris code** → the **client's attributes + choices**.
- **CSM produces a FILTERED field-code list** — only the attributes a surveyor actually captures **on site** (NOT the full list, which would overwhelm them). This filtered set is exported to the survey instruments as `.12dfieldcodes` / Leica / Trimble.
- **The surveyor captures only those on-site attributes** → the picked-up data flows into **12d**.
- **The 12d MACRO fills everything the surveyor did NOT capture** — the **formulas** and **fixed/default values**.
- **Per-project profiles:** the same client has many projects (e.g. 34), each with **different fixed values**. The macro lets the user **enter the per-project values** and run the formulas — one profile per project.
- **CSM defines all the formula/logic (the brains);** the 12d macro executes that logic and collects the per-project entries.
- **Net result:** survey data, started from CSM's filtered code list, **enriched to fully match the client's attribute spec** → compliant deliverable.
Easy to under-weight but CENTRAL: (a) the **filtering** (surveyor sees only on-site attributes — reducing overwhelm is a core CSM value), and (b) the **per-project profiles** (user enters different fixed values per project via the macro). Only 3 field-capture export formats matter: `.12dfieldcodes`, Leica XML, Trimble FXL.
---
Current Phase (updated 2026-05-28)
**Phase 1 — ACTIVE:** Perfect the CSM app. All workflow gaps resolved, onboarding
wizard working, Sync & Share panel working, bulk import working. App is distributable
to multiple users with a clear file-based collaboration model.
**Phase 2 — NEXT:** Vida full pipeline end-to-end, including the fixed 12d macro UI
and logic. Only after Phase 1 is complete. UnityWater follows after Vida is proven.
**Active client focus:** Vida first, UnityWater second. MRBC and WAADAC remain in
backlog until the Vida pipeline is proven end-to-end. The "all CSM schema
clients" goal is aspirational — do not treat it as the current build target.
**A-SPEC / GIS export:** Explicitly NOT in the CSM vision. Do not build aSpec export
paths, GIS dataset generators, or A-SPEC XML emitters. If an audit or agent flags
A-SPEC as a gap, discard it.
**Only three field capture export formats matter:**
`.12dfieldcodes`, Leica XML, Trimble FXL. All other export formats are secondary.
Generated sample `.12da` files are not production evidence by themselves. Treat
`Output\<Client>\<Client>_sample_survey.12da` as regression fixtures until re-audited.
A real extracted `.12da` (e.g. `.planning\vida_12daz_extract_20260522_0216`) carries
stronger evidentiary value but still needs client/schema relevance and mapping-lineage
proof before it can support acceptance.
This document is Abdullah's durable product vision for the combined Codelist
Manager (CM) and Client Schema Manager (CSM) system. Future agents working on any
of the above must preserve this direction.
---
North Star
Client asset data delivered correctly, first time, every time.
CM/CSM is not just a code-list editor and not just a UI over tables. It is the
controlled production system that helps Veris capture, enrich, validate, and
deliver compliant as-constructed infrastructure asset data for councils and
asset managers — under whatever delivery standard the client requires (ADAC,
Vida, council-specific, or other).
The system must reduce rework by making the required asset classes, field codes,
attributes, choices, rules, validation evidence, and export blockers visible
before delivery.
---
Domain Framing
ADAC and A-SPEC are asset data specifications used across Australia to deliver
as-constructed infrastructure data in a consistent, GIS-ready format.
They cover infrastructure assets such as:
- roads and transport assets
- drainage and stormwater assets
- water and sewer assets
- lighting and electrical assets
- open space assets
- cadastre and supplementary/context assets
The aim is consistent asset data capture and delivery so councils and asset
owners can reduce errors, improve data quality, and use the final data in GIS
and asset-management systems.
---
Workflow CM/CSM Must Support
0. System Boundary Between CM And CSM
Codelist Manager is the upstream source of truth for the Veris master codelist.
CSM is downstream and client-delivery focused. They serve two different client archetypes
(see Section 0D). The pipeline is strictly one-way:
`Codelist Manager → Client Schema Manager`
Do not silently write CSM decisions back into Codelist Manager. If a CSM finding
should become a permanent upstream CM change, create a safe proposal/review item
with evidence.
**CM owns:**
- Veris master library: codes, attributes, choices, styles, field capture definitions
- Veris National subset and state/client variants
- Veris-native client handling end-to-end (clients who use Veris attributes as-is)
- client code/model/layer to Veris code assignment for Veris-native clients
- `client_mappings` and `client_extra_entries`
- `attribute_match` / `att_key` qualifiers used to make broad Veris codes specific
- mapfile generation/patching while preserving supplementary sections
- the `master_library.db` — the single source of truth for all Veris code/attr/choice definitions
**CSM owns (custom attribute clients only — see Section 0D):**
- client requirement schema (features, attributes, choices with client vocabulary)
- client dictionary intake: ADAC XLS, client Excel workbooks, council PDFs, 12daz
- side-by-side comparison of Veris code/attributes vs client code/schema attributes
- client attribute mapping and review (the four-step mapping chain)
- capture-gap detection
- field capture outputs: `.12dfieldcodes`, Leica XML, Trimble FXL
- delivery rules for Attribute Manipulator, MetaConnex, custom 12dPL, and exporters
- validation, evidence, and final client-delivery readiness
- local cache of Veris master data (`veris_code_attrs` table) synced from CM's `master_library.db`
**CSM works from a distributed copy of CM data.** Abdullah copies the current CM
database/package into CSM and sends it to app users. CSM users may edit that
distributed CM copy to propose client code, attribute, choice, sort, and export
changes, then export/send the changed package back to Abdullah. Abdullah remains
the CM authority: he reviews the returned package and imports accepted changes
back into CM.
For routine lookup, CSM still syncs `veris_code_attrs` and `veris_attr_choices`
from `master_library.db` via `csm/importers/cm_master.py`. The CM MCP server is
used only by the `/client-schema` AI skill for code discovery during onboarding.
0A. The Chicken-And-Egg Constraint — No Real Survey Data Exists Yet
This is the single most-missed point by agents working on this system. Read it
before assuming anything about test data.
The CSM app produces the field code file (`.12dfieldcodes`, Leica XML, Trimble
FXL) that surveyors **will** use on site to capture data. That file cannot be
released to real surveyors until the whole CSM-to-12d-runtime pipeline is
proven end-to-end. **Real survey data therefore does not exist yet by design**
— it cannot exist until the system is proven, because the system is what tells
surveyors how to capture.
The validation loop is:
1. CSM generates the field code file (defines what surveyors capture)
2. Agent generates a synthetic .12da that REPLICATES what a surveyor would
have captured using that field code file as their spec
3. CSM mapping pipeline runs against the synthetic survey
4. Output must be a compliant client deliverable
5. Once step 4 is green end-to-end, the field code file is released to
surveyors → real survey data starts to existWhat this means for agents:
- Do **not** wait for "a real survey" to validate the system. There is no real
survey and there will not be one until the system itself ships.
- The synthetic `.12da` test input is the canonical fixture. It must
faithfully replicate the surveyor's expected capture using the
CSM-generated field code file.
- Generated regression fixtures at `Output/<Client>/<Client>_sample_survey.12da`
served as smoke tests. The full shippable validation is a full-coverage
synthetic survey (every client code present, every required attribute
populated as the surveyor would have done) running through the pipeline
end-to-end.
- "Real-job validation against a previous manual deliverable" is not the
shippable gate. It cannot be — the previous manual process is what this
system replaces.
- Once the full-coverage synthetic run is green, the field code file ships,
surveys start, and post-deployment real-survey audits become possible. That
is a post-shipping verification, not a pre-shipping gate.
0B. Two-Stage Architecture And Canonical Terms
Use these terms consistently. Agents have repeatedly gone in circles because
docs and conversations described the same architecture with different phrasing.
This section locks the vocabulary.
The system has exactly two architectural stages.
**Stage 1 — Structural Mapping**
- Owner: **Codelist Manager (CM)**
- Output: the **Map File** (`.mapfile`)
- Job: translate Veris field codes into the client's structural language. The
surveyor captures using Veris codes; the Map File converts those codes to
the client's expected code/model/layer names at delivery.
- Example: client model name `"Stormwater.Pit"` ↔ Veris code `DRDPC`.
**Stage 2 — Attribute Logic and Enrichment**
- Design owner: **CSM** (writes the rules)
- Execution owner: **12d Macro** (runs the rules)
- Output (design): the **Guide Template** — the package of files at
`exports/<Client>/current/` containing `csm_runtime_actions.csv`,
`csm_value_maps.csv`, `csm_rule_manifest.csv`, the macro-readable sidecar
`project_runtime_profile.12d_runtime.txt`, and the
`project_runtime_profile.csm.json` / `.locked.json` profile pair.
- Output (execution): the populated client deliverable (a 12d project with
every required client attribute populated, plus exported `.12dfieldcodes`,
Leica XML, Trimble FXL, `.12dattmf`, `.12dMetaConnex`, ADAC/A-SPEC XML
where the client requires them).
**Role split — one sentence to remember:**
> CSM is a template generator. The 12d macro is the execution engine.
CSM does not perform calculations, spatial lookups, or value translations at
design time. CSM writes instructions describing what to compute; the macro
reads those instructions and runs the math at runtime. This is the only
correct mental model.
**Canonical attribute source categories** (used in
`csm_runtime_actions.csv` under the `source_kind` column):
| `source_kind` | Meaning |
|---|---|
| `veris_attr` | Copy from a captured Veris attribute. May apply a `value_map`. |
| `capture_new` | Use the directly captured VT_UTL_* (or equivalent client) attribute. |
| `office_ingestion` | Value entered in office, not captured by the surveyor. Builder names, document numbers, asset owners, project defaults. |
| `project_default` | Value supplied by the Project Runtime Profile (client-wide or job-specific). |
| `reference` | Resolved via a selected 12d reference — TIN, alignment, polygon, string, cadastral parcel. |
| `derived_formula` | Computed from element geometry or other attributes (e.g. Z, concatenation, distance, bearing, surface-minus-Z). |
| `conditional_rule` | Excel/Power Automate-style IF/THEN logic over other attributes. |
| `value_map` | Veris choice value translated to client choice value (e.g. QL-A → A). |
| `ignore` | No mapping required for this client. |
| `blocked_manual` | No executable source path — needs human review before delivery. |
**`office_ingestion` is the canonical term.** Do not use alternative phrasings
like `office_added`, `manual_office`, `office_default`, or `office_only`.
**Constrained predefined function environment.** Users build rules only from
a fixed catalogue the macro can execute. Free-text logic is not supported.
This is what enforces "every rule is executable" — the rule builder UI cannot
emit a rule the macro cannot run.
---
0C. Worked End-to-End Example — One Attribute, All Layers
Use this example to test whether a fresh agent (or fresh chat) understands
the pipeline. If the agent cannot trace one attribute through all five
layers below, they have not understood the system yet.
**Target:** populate `VT_UTL_DEPTH = 1.25` (metres) on a Vida side-entry pit.
| Layer | What happens | Where |
|---|---|---|
| **1. CM Map File (Stage 1)** | Client model `"308 Side entry pit"` is linked to Veris code `DRDPC`. The Map File defines this code mapping. | `cm_features` table — pre-existing CM data |
| **2. CSM Guide Template (Stage 2 design)** | Office user picks: "Depth comes from design TIN minus element Z." CSM writes one row in the Guide Template: `feature=308 Side entry pit, attr=VT_UTL_DEPTH, source_kind=reference, source_ref=TIN_A, derived_fn=surface_minus_z`. | `exports/Vida/current/csm_runtime_actions.csv` |
| **3. Project Runtime Profile (Stage 2 design)** | The abstract reference `TIN_A` is resolved to a real 12d TIN name — say `Design Surface 2026-05-24`. The Veris job number, document number, builder, and asset owner are also captured here. | `exports/Vida/current/project_runtime_profile.locked.json` |
| **4. Field capture (synthetic survey while system is being proven)** | Surveyor (or synthetic fixture) captures a side-entry-pit point at `(x=502000, y=6959000, z=10.0)` using the CSM-generated field code file. | `Output/Vida/Vida_simulated_survey.12da` |
| **5. 12d Macro execution (Stage 2 execution)** | Macro reads the runtime action, looks up `Design Surface 2026-05-24`, computes `surface_at(502000, 6959000) − 10.0 = 1.25`. Writes `VT_UTL_DEPTH=1.25` onto the element. Logs hash, paths, and value to evidence. | Macro writes `exports/Vida/current/CSM_runtime_result_<timestamp>.txt` + evidence CSV |
| **6. Deliverable export** | All pits in the output 12d project carry their correctly computed `VT_UTL_DEPTH`. Export sees the populated attribute. | Final `.12dattmf` / `.12dMetaConnex` / Vida XML |
That single attribute touches every architectural layer: Map File for the
code link, Guide Template for the rule, Project Runtime Profile for the
reference resolution, synthetic survey for capture (until real surveys
exist), macro for execution, and deliverable for output. If any agent
proposes a change to one layer, they should be able to state how it affects
the other five.
0D. Two Client Archetypes — Which System Handles Them
This split is non-negotiable. Do not blur the boundary.
**Type A — Veris-native client (handled entirely by CM):**
- The client captures and delivers data using Veris attribute names and Veris choice values exactly as defined in the master codelist.
- No custom attribute naming, no choice translation, no value maps needed.
- CM generates the mapfile, fieldcode list, and exports directly.
- CSM is not involved. Do not create a CSM schema for a Veris-native client.
- Example: internal Veris survey jobs where field crews use Veris vocabulary directly.
**Type B — Custom attribute client (handled by CSM):**
- The client has their own attribute names, labels, and choice vocabulary.
- Example: a council that uses "Pipe Material" instead of `mat_type`, and "HDPE Pipe" instead of `HDPE`.
- CSM manages the full four-step mapping chain: Veris code → client attribute name → client choice values → conditional rules.
- CSM generates the field capture outputs (fieldcodes, Leica, Trimble) using client vocabulary.
- Example clients: Vida, UnityWater, MRBC, WAADAC, Gold Coast, etc.
**How to tell which type:**
If the client's field data uses client-specific attribute names or choice values → Type B (CSM).
If the client is happy using raw Veris attribute names and Veris choice values → Type A (CM).
When in doubt, ask Abdullah.
**Locked current client split (confirmed with Abdullah, 2026-05-28):**
This split is a one-time fixed classification. Do not infer ownership from
whichever database currently has a row, and do not run a recurring "split"
process that moves clients between CM and CSM. Use this locked list unless
Abdullah explicitly changes it.
CM-owned controller field-code clients:
- `BHP`
- `DoD`
- `Master` (Veris National)
- `MRWA`
- `RMS` (this is the CM owner for the `TFNSW` controller field-code package)
- `TMR`
- `VicRoads`
CSM-owned custom schema clients:
- `BrisbaneCC`
- `GoldCoast`
- `MRBC`
- `NswAdac`
- `QLDADAC`
- `Rspec`
- `SunshineCoast`
- `TFNSWMETA`
- `TFNSWUT`
- `UnityWater`
- `Vida`
- `WAADAC`
Controller Field Codes folder mapping:
| `C:\Veris\Controller Field Codes` folder | CM owner |
|---|---|
| `01_Veris_National` | `Master` |
| `02_BHP` | `BHP` |
| `03_DoD` | `DoD` |
| `04_MRWA` | `MRWA` |
| `05_TFNSW` | `RMS` |
| `06_TMR` | `TMR` |
| `07_VicRoads` | `VicRoads` |
Implementation rule:
- CM-owned clients may appear in CSM as distributed CM copy/proposal rows that
users can edit and send back to Abdullah for review/import into CM.
- CM-owned clients must not appear in the CSM client picker as schema clients.
- Do not create CSM schemas for CM-owned controller field-code clients.
- CSM variants such as `TFNSWMETA` and `TFNSWUT` are downstream schema clients and must link back to the CM/RMS controller owner where controller field-code truth is required.
0E. Distributed Collaboration Model (Hub-and-Spoke)
CSM is designed for distributed use: multiple staff each run a local copy, work
on one client's schema and/or a copied CM client database package, then exchange
database files with Abdullah who holds the central hub.
**The model:**
Abdullah (central hub)
├── master_library.db ← CM's master codelist, distributed to all users
├── CM client DB copies ← editable proposal packages distributed to users
└── workspace/
├── Vida/client_mapping.db ← receives from User A, redistributes
└── UnityWater/client_mapping.db ← receives from User B, redistributes
Each user (local spoke)
├── master_library.db ← imported from Abdullah via "Import Veris Master DB"
├── copied CM package ← edits proposed CM changes here
└── workspace/<Client>/client_mapping.db ← works here, sends back to Abdullah**Sync & Share panel — three operations:**
| Operation | Who uses it | What it does |
|-----------|-------------|-------------|
| **Import Veris Master DB / CM Package** | Every user | File picker -> select Abdullah's copied CM database/package -> syncs reference data and, where enabled, loads editable CM proposal data. Shows "Last synced: YYYY-MM-DD HH:MM" |
| **Export Client DB** | Every user | File-save dialog → copies `workspace/<Client>/client_mapping.db` to a location of their choice. This is the file they send to Abdullah |
| **Export CM Change Proposal** | Every user | File-save dialog -> writes the edited CM copy/proposal package so Abdullah can review and merge accepted changes into CM |
| **Import Client DB** | Abdullah only | File picker → validates it's a valid CSM DB → copies to `workspace/<ClientName>/`. Used when Abdullah receives a DB from a user |
**Rules:**
- One client DB per user at a time. No two users work on the same client simultaneously.
- Never write directly to Abdullah's authoritative CM database from a user's CSM
session. Users edit a copied CM package/proposal and send it back.
- Abdullah reviews returned CM proposal packages and imports accepted changes
into the real CM database.
- Auto-sync the master DB on CSM startup when the master DB path is configured in settings.
- The sync is non-destructive to Abdullah's source CM file: it updates local CSM
reference/proposal data and never mutates the original file in place.
- Show "Last synced" timestamp prominently in the app header or settings screen.
- The master DB path is stored in user settings (not in the client DB) so it persists across sessions.
**No server required.** The SQLite file IS the unit of exchange. Email, shared drive,
or Teams file share — Abdullah decides the distribution channel.
0F. Bulk Client Schema Import (CSV Format)
New clients can be seeded in bulk via a CSV file rather than manual feature-by-feature
entry. This is the format Abdullah (or any user) fills in and uploads through the
new-client wizard or the Import button.
**Format — one row per attribute:**
feature_code,feature_description,attr_name,attr_label,data_type,is_required,field_visible,sort_order,choices
UTIL_WATER,Water Utilities,PIPE_MAT,Pipe Material,choice,1,1,1,HDPE=HDPE Pipe|PVC=PVC Pipe|STEEL=Steel Pipe
UTIL_WATER,Water Utilities,PIPE_DIA,Pipe Diameter (mm),integer,1,1,2,
UTIL_SEWER,Sewer Infrastructure,PIPE_MAT,Pipe Material,choice,1,1,1,PVC=PVC Pipe|CONC=Concrete Pipe**Column rules:**
- `feature_code` — required. Used as `client_feature_code` in DB.
- `feature_description` — first occurrence wins if the feature appears on multiple rows.
- `attr_name` — required. Used as `client_attr_name`.
- `attr_label` — stored as `notes`; optional human-readable label.
- `data_type` — one of: `text`, `choice`, `integer`, `real`, `date`.
- `is_required` — `1` or `0`. Default `1`.
- `field_visible` — `1` or `0`. Controls field capture visibility. Default `1`. Applies to all choices for this attr.
- `sort_order` — integer. Controls order attrs appear in export. Default `0`.
- `choices` — pipe-delimited `value=label` pairs. Leave blank for non-choice attrs.
- `HDPE=HDPE Pipe|PVC=PVC Pipe` — value and label differ
- `HDPE|PVC|STEEL` — value used as label (bare tokens, no `=`)
- 20 choices = 20 pipe-delimited entries in one cell — fill in Excel comfortably
**Behaviour:**
- Import is idempotent: re-uploading the same file causes no duplicates.
- Bad rows (missing `feature_code`, invalid `data_type`) are collected as errors; the rest still import.
- Template file: `csm/importers/client_schema_import_template.csv`
- Backend: `csm/importers/bulk_import.py` → `import_client_schema(client_name, schema_id, csv_path)`
1. Client Checklist And Scope
The user must be able to define which delivery standard and asset groups apply:
- ADAC Queensland
- Vida (Victorian utility standard)
- Council/client-specific standards
- Drainage / Stormwater
- Roads / Transport
- Water
- Sewer
- Open Space
- Other client-specific asset classes
**Out of scope for CSM:** A-SPEC GIS export, A-SPEC XML, GIS-ready datasets.
Do not build these. Do not flag their absence as a gap. The delivery standard
determines validation rules and required attributes, but GIS/A-SPEC output is
not a CSM responsibility.
This is not a passive note. It drives the schema subset, required attributes,
lookup values, validation rules, and final export requirements.
1A. Project Runtime Profile And Job-Specific Values
A client/council schema is reusable. A Veris project/job under that client is
runtime-specific. Do not confuse the two.
Example hierarchy:
Client / council:
Brisbane City Council
Client schema:
BrisbaneCC ADAC delivery schema
Project runtime profiles:
Builder A - Road Upgrade - Stage 1
Builder B - Sewer Main Extension
Developer C - Subdivision Package 4**Real-world example — Vida (VIDA Transport Digital Engineering):**
Vida is the governing body that defines the utility data standard for Victorian
infrastructure projects. Vida does not receive data directly — multiple contractors,
councils, and developers deliver to the Vida standard on behalf of different clients.
Governing standard:
Vida (VIDA Transport Digital Engineering Data Specification)
Client schema (defined once, reused for all Vida jobs):
Vida utility schema — 40 feature codes, 28 VT_UTL_* attributes, choice lists,
derivation rules for VT_UTL_FEATURE, VT_UTL_ABS_HEIGHT, etc.
Project runtime profiles (one per job, filled in CSM by office staff):
Council A - Road Upgrade Package 3 - Stage 1
→ Document number: DOC-VIDA-2026-0341
→ Contractor: XYZ Civil
→ Asset owner: Council A Infrastructure
→ Survey datum: AHD, MGA2020 Zone 55
Council B - Stormwater Renewal - Stage 2
→ Document number: DOC-VIDA-2026-0412
→ Contractor: ABC Surveys
→ Asset owner: Council B Drainage Authority
Developer D - Subdivision Estate - Stage 4
→ Document number: DEV-VIDA-2026-0089
→ Contractor: DEF Consulting
→ Asset owner: TBC pending handoverThe Vida schema defines the rules (how VT_UTL_FEATURE is generated from model name
and attribute concatenation, what choices are valid, what is mandatory). Each project
runtime profile supplies the job-specific values that feed those rules. CSM generates
and validates the profile; the 12d fixed macro reads the saved profile file and
executes it against the survey data. The office staff fills in the profile in CSM —
the 12d operator runs the macro.
The same council/client can receive many Veris jobs from different builders,
developers, contractors, stages, packages, and asset owners. Those jobs can use
the same client schema but require different project values such as:
- Veris job number
- client project number
- contractor / builder / developer name
- package, stage, estate, development, or subdivision name
- asset owner / receiving authority
- project status, survey status, construction status, and acceptance status
- coordinate system, datum, level datum, survey control reference, and capture
quality defaults
- construction dates, install dates, inspection dates, or handover dates
- project-wide ADAC/A-SPEC defaults such as Owner, Status, Level, DataQuality,
AssetCategory, capture method, or validation contact
- reference models, TINs, polygons, alignments, boundaries, and lookup zones
used by calculations or spatial assignment rules
CSM therefore needs a first-class **Project Runtime Profile** under the selected
client/schema. The profile stores the job-specific values and runtime references
that are not permanent client-schema facts. Users must be able to create,
duplicate, review, edit, approve, and select the active profile before running
the 12d macro or generating final exports.
Every real project must also have a saved **Project Runtime Profile file**. This
must be a portable, versioned handoff artifact, not only hidden UI state and not
only rows inside the CSM database. The database can index, cache, validate, and
history-track profiles, but the macro/runtime handoff must be an explicit saved
file that can travel with the project evidence package.
Recommended artifact shape:
exports/<Client>/<SchemaVersion>/<ProjectProfileSlug>/
project_runtime_profile.csm.json
project_runtime_profile.locked.json (optional frozen run copy)
project_runtime_profile.12d_runtime.txt (macro-readable key/value sidecar)
CSM_evidence_<timestamp>.csv
CSM_runtime_result_<timestamp>.jsonThe saved Project Runtime Profile file must include at minimum:
- profile schema version
- client ID/name and client schema ID/version
- project profile ID/name/version
- Veris job number and client project number
- builder, contractor, developer, package, stage, estate, subdivision, and
asset-owner values where applicable
- approved project defaults, with source, status, approver, timestamp, and
whether the value is client-wide or project-only
- office-ingestion values required before export
- selected 12d runtime references: models, strings, points, TINs, alignments,
boundaries, polygons, lookup zones, and reference attributes
- spatial/polygon lookup definitions and the reference geometry/model they use
- rule/function export manifest for the active mapping/rule set
- output paths for fieldcodes, ATTMF, MetaConnex, macro outputs, ADAC/A-SPEC
deliverables, and evidence
- validation state and any manual/blocking decisions
The 12d macro is responsible for applying the saved profile to the active 12d
project. The canonical profile is JSON, but the macro does not need to parse
JSON directly when a simpler generated handoff is safer. CSM may emit a locked
macro-readable line-pair sidecar such as
`project_runtime_profile.12d_runtime.txt`; the macro reads that sidecar with
12d file IO, uses the values to populate the Project Profile tab, allows an
operator override where the generated macro explicitly permits it, resolves the
selected 12d references, applies project defaults and office values, runs
spatial/polygon lookups, executes approved mappings/rules, and writes
evidence/results back out. The macro may prompt for missing runtime selections
only when the profile explicitly allows a manual runtime prompt. It must not
silently invent project values or treat a missing profile file as success.
Project defaults and fixed values must not be hardcoded into the client schema
unless they are genuinely client-wide. A value such as `"AS5488 Level A"` may be
a project default for one job, a client-wide default for another client, or a
manual office decision for a third job. The UI must make that source explicit.
The 12d macro is the runtime execution engine. It must read a generated project
configuration/template that includes:
- path to the saved Project Runtime Profile file
- path to the locked macro-readable runtime sidecar when used
- active client and schema version
- active project runtime profile ID/name
- approved project default values
- office-ingestion values
- selected reference models/TINs/polygons/alignments
- approved mapping, value-map, formula, spatial lookup, and conditional-rule
definitions
- export paths and evidence output paths
The evidence written back from 12d must record which project runtime profile and
which project values were used. A successful schema import is not enough; final
acceptance depends on the correct project profile being applied to the correct
job data.
1B. Fixed 12d Runtime Macro And CSM Instruction Package
> **Macro design is fully specified in `docs/MACRO_ARCHITECTURE_AND_PLAN.md` — read it
> before any 12d macro work.** Summary: ONE fixed macro (`CSM_Apply_Profile`), any
> client/profile. CSM generates **converter settings (data), NOT 12dPL**. The macro IS the
> UI and builds a **dynamic panel** (input/model/TIN/choice boxes) per client from a CSM
> settings file (file box 2 = "which fields to input + how to calc"), and reads/writes a
> per-project **client sub-profile** (file box 1 = the user's saved values, reload for easy
> re-runs). Which fields appear on the panel is determined by each attribute's `source_kind`
> (§0B/§1C). `.12dattmf`/`.12dMetaConnex` are last-resort adapters, never first.
>
> **Interactive UI is TREE-FIRST (IMPLEMENTED 2026-06-02).** The operator panel is a
> left `Tree_Box` of **group nodes** (one per seeded element/group name) with **element
> child nodes**; the right pane is native per node — group node = counts + element grid +
> group-safe bulk apply, element node = attribute grid + per-element override. Grouping is
> by seeded element/group name. It is **not** a flat grid and **not** resolver-first. Full
> contract: `docs/MACRO_ARCHITECTURE_AND_PLAN.md` §0, `docs/TWO_FILE_CONVERTER_SETTINGS_ARCHITECTURE.md`,
> `_planning/CSM_UI_UX_WORKFLOW_REBUILD_PLAN.md`.
The production architecture is **not** "CSM creates a new macro for every
client or every job." That pattern was useful during prototyping because
client-specific generated macros were easy to compile, inspect, and test in
isolation. It is not the target operating model.
The target operating model is:
CSM app
-> exports a project instruction package
- project_runtime_profile.csm.json
- project_runtime_profile.locked.json
- project_runtime_profile.12d_runtime.txt
- csm_runtime_actions.csv/jsonl
- csm_value_maps.csv/jsonl
- csm_rule_manifest.csv/jsonl
- expected evidence schema/output paths
Fixed 12d macro/runtime engine
-> user selects or points to the exported package
-> macro reads the profile and instruction files
-> macro applies mappings, defaults, value maps, fallback chains, formulas,
spatial/reference/TIN/polygon/alignment lookups, and validation
-> macro writes evidence/results back to the packageThe fixed runtime macro may be named something like:
CSM_Apply_Profile.4dm / CSM_Apply_Profile.4doThe fixed macro should remain stable and heavily tested. CSM should normally
export **data/configuration**, not macro source. The same compiled macro should
be able to run MRBC, UnityWater, Vida, WAADAC, or a future client by loading a
different Project Runtime Profile/instruction package.
The operator-facing 12d macro still needs a real panel. That panel is driven by
the Project Runtime Profile: if the profile declares one TIN, expose one TIN
binding; if it declares ten TINs, expose ten; likewise source models, boundary
models, strings, polygons, alignments, fixed values, office values, and output
model/prefix controls. Agent/headless testing must not use a separate fake
engine. It should pre-fill those same inputs from baked command-line/profile
values and auto-run without waiting for the Run button.
12d model/layer names generated by CSM or entered as output prefixes must be
12d-safe. Do not generate names with underscores, dashes, or dots for runtime
models. Prefer readable space-separated names such as `CSM Vida CMCPC`, and log
the final output model name in the macro evidence. Because the macro moves
processed elements to the output model, repeatable tests must rebuild/copy fresh
fixture data or deliberately use the previous output model as the next source.
If CSM continues to emit per-client `.4dm` files during transition, classify
them as generated debug/test artifacts or compatibility shims. Do not treat
per-client generated macros as the final production architecture unless a future
design review explicitly reverses this decision.
The macro-readable sidecar should be deliberately simple because 12dPL is safer
with line-pair or CSV-style input than with complex nested JSON. The canonical
CSM profile can remain JSON, but the runtime package should include simple
files the macro can parse reliably, for example:
client=MRBC
schema_version=current
profile_json=C:\...\project_runtime_profile.csm.json
locked_profile_json=C:\...\project_runtime_profile.locked.json
evidence_csv=C:\...\CSM_evidence.csv
action_file=C:\...\csm_runtime_actions.csv
value_map_file=C:\...\csm_value_maps.csv
rule_manifest=C:\...\csm_rule_manifest.csv
project_default.DataQuality=AS5488 Level A (surveyed)
reference.design_tin=Design Surface
reference.sewer_main=Runtime Sewer Main A
force_project_default.SurveyedBy=falseThe fixed macro must log the exact package path, locked profile path, profile
hash, client, schema version, selected 12d references, and action/rule files it
used. A macro run without those values is not production evidence.
This design keeps ownership clear:
- CM owns Veris field-capture/code-list truth.
- CSM owns client schema, mappings, rules, profiles, validation, and instruction
package export.
- The fixed 12d macro owns runtime application against the active 12d project.
- The Project Runtime Profile/instruction package is the contract between CSM
and 12d.
1C. Field Code List Visibility Control
Not every schema attribute should appear in the field-capture tool. Surveyors
get overwhelmed when the code list dumps every attribute the spec defines.
The CSM app exposes explicit, per-client, per-feature, per-attribute visibility
controls:
- **`field_visible` toggle** per `client_schema_attributes` row — three-state:
- `NULL` (default): use the source_kind default
- `1`: force visible to surveyor
- `0`: force hidden from surveyor
Source_kind-based defaults when `field_visible` is `NULL`:
| `source_kind` | Default `field_visible` | Why |
|---|---:|---|
| `capture_new` | 1 (visible) | Surveyor enters it directly |
| `veris_attr` | 1 (visible) | Legacy Veris-side capture path (see 1D) |
| `office_ingestion` | 0 (hidden) | Office enters post-survey |
| `project_default` | 0 (hidden) | From Project Runtime Profile, not field |
| `reference` | 0 (hidden) | Computed via selected 12d reference at runtime |
| `derived_formula` | 0 (hidden) | Computed by macro |
| `conditional_rule` | 0 (hidden) | Computed by macro |
| `value_map` | 0 (hidden) | Translation, not separate capture |
| `ignore` | 0 (hidden) | Excluded entirely |
| `blocked_manual` | 0 (hidden) | Not executable yet |
The user can override the default per attribute. Override always wins.
**Field-tool choice-list trim:** for any choice-backed attribute that IS
field-visible, the user can also trim which `client_attr_choices` rows appear
in the surveyor's picker. Full choice list is preserved for value-map /
validation purposes; only the surveyor view is trimmed.
**Export contract:** the field-code exporters (`.12dfieldcodes`, Leica XML,
Trimble FXL, plus the per-client PDF code-list document — see Section 12)
MUST filter by `field_visible` and by the choice-trim subset. The full
runtime schema is unaffected.
**Validation:** if a `capture_new` attribute is forced hidden, the surveyor
literally cannot capture it. The validation centre (Section 11) MUST flag
this case as a soft warning the user has to acknowledge.
1D. Client-Side Field Capture is the Canonical Path
Surveyors capture using **Veris codes** (Stage 1 routing) plus **client
attribute names with client choice vocabulary** (Stage 2 in client-side form).
They do NOT capture using Veris attribute names + Veris choice values
expecting CSM to translate later.
Reasons:
- **Capture-time error prevention.** If the surveyor only sees the client's
valid choices in the field, they cannot pick a value that has no
client-side translation. Value-map gaps cannot happen in the field —
they're impossible by construction.
- **Clearer surveyor workflow.** The picker shows the client's actual
vocabulary (e.g. "Wing Wall", "Gravity sewer") instead of generic Veris
options translated post-hoc.
- **Cleaner macro path.** The macro reads client attribute names directly,
no value-map translation step. Already proven by Vida (40 elements / 120
data rows / `status=pass` 2026-05-24).
**Veris-code broadness — att_key qualifier:**
Veris codes are deliberately broad (e.g. `DRDPC` covers all drainage pits;
`SWSUG` covers all sewer underground). When a broad Veris code maps to
multiple client features, CSM uses the **att_key qualifier** (existing CM
concept — `cm_features.attribute_match`) to narrow:
Veris code DRDPC + Sub Type = "Side Entry" → Vida feature "308 Side entry pit"
Veris code DRDPC + Sub Type = "Grated" → Vida feature "309 Grated pit"
Veris code SWSUG + Sub Type = "Gravity" → UnityWater feature "Sewer.NonPressurePipe"
Veris code SWSUG + Sub Type = "Rising" → UnityWater feature "Sewer.PressurePipe"The surveyor captures `Sub Type` as a regular field attribute; CSM resolves
the (Veris code + Sub Type) tuple to the client feature code at macro apply
time.
**Source_kind status under this canonical path:**
| `source_kind` | Status |
|---|---|
| `capture_new` | **Canonical** — client attribute captured directly with client vocabulary |
| `veris_attr` | **Legacy migration path** — used only for clients whose schema intake stored Veris-side attribute names. Should be migrated to `capture_new` with the client attribute name. New schemas should not use `veris_attr` |
| `value_map` | **Deprecated at capture layer** — keep only as a runtime migration shim for legacy Veris-side captures. New designs should never need it |
| All other source_kinds | Unaffected |
**Migration policy:** when a new client schema is imported or an existing
schema is reviewed, the intake step should convert any `veris_attr` mapping
to `capture_new` with the client attribute name and the client's choice
vocabulary, unless the user explicitly opts to keep the Veris-side capture
path for that attribute. The validation centre should surface remaining
`veris_attr` mappings as "legacy capture path — consider migrating."
2. Source Dictionary And Client Schema Intake
CSM must ingest and understand source material such as:
- raw ADAC V501/V600 `.xls` data dictionaries
- A-SPEC/client Excel workbooks
- council PDFs/specifications
- GIS schemas
- 12d sample projects or `.12daz`/`.12da` files
- CM mapfiles and fieldcode exports
- CM `client_mappings`
- CM `client_extra_entries`
- CM `attribute_match` / `att_key` mapping qualifiers
For raw ADAC dictionaries, use:
Get-ChildItem "..\To_Do" -Recurse -Filter "ADAC*Data Dictionary.xls"
python -m csm.importers.adac_dictionary "<path-to-adac-xls>" --json .planning\<name>_dictionary_extract.jsonThe ADAC parser preserves:
`asset element -> feature group -> feature -> attribute/container -> enum/type/child detail`
with mandatory flags, choices, type strings, constraints, source sheet, and row
provenance.
3. The Four-Step Mapping Chain (The Law — Read This Every Time)
The complete mapping pipeline runs in four steps, in this order, always.
Do not skip steps. Do not run steps out of order.
STEP 1 — CODE LINKING (Codelist Manager)
Client code / element / layer name
↓
Veris various code(s)
e.g. "Telco Conduit UG" → CMCUG
STEP 2 — ATTRIBUTE NAME LINKING (CSM)
Veris attributes that exist on the linked Veris code(s)
↓
Client attribute names matched to those Veris attribute names
e.g. Veris "Sub Type" → Client "SUBTYPE"
Veris "Depth(m)" → Client "DEPTH_M"
STEP 3 — ATTRIBUTE VALUE LINKING (CSM)
Veris attribute choice values
↓
Client attribute choice values (the client's predefined dropdown list)
e.g. Veris Material "Concrete" → Client Material "CONC"
Veris Material "Steel" → Client Material "STL"
STEP 4 — RULE / CONDITIONAL LOGIC (CSM Rule Builder)
When value linking is not a flat 1:1 translation —
build conditional rules that combine multiple attributes
e.g. IF Material="Concrete" AND Diameter>300 THEN PipeClass="RC3"**Why this order is non-negotiable:**
- Step 2 (attribute names) is only valid after Step 1 (code linking). A Veris
attribute mapping is fake if the client feature is not yet linked to a Veris
code that carries that attribute.
- Step 3 (value linking) is only valid after Step 2 (attribute names). You
cannot map "Concrete → CONC" until you know which attribute pair that applies to.
- Step 4 (conditionals) builds on top of Steps 2 and 3. It references already-
linked attribute names and already-defined choice lists.
**Client code fallback rule:**
If the client schema has no separate feature code, use the raw model/layer/element
name as `client_feature_code`. Keep the raw value separate from the 12d-safe export
name.
**CM match-key rule:**
Existing CM `attribute_match` / `att_key` values are real mapping evidence. They
often explain why one broad Veris code becomes a specific client code/model row.
Examples:
- UnityWater `LotParcel -> SSBOU` with `Sub Type=Lot Parcel`
- MRBC `SurveyMark` fan-out to `SSCTL`, `SSPIL`, `SSBMK` using `Sub Type`
match values
CSM must carry those qualifiers into the schema comparison and rule/mapping
workflow. They are not comments.
4. Attribute And Choice Mapping
For each client-required attribute, the system must classify the source honestly:
- captured Veris attribute (direct name match)
- value map (Veris choice → client choice translation table)
- conditional rule output (IF/THEN based on one or more attribute values)
- client default (same value for all projects for this client/schema)
- project runtime default (job-specific value read from the active project
runtime profile)
- fixed feature default (same value for all instances of this feature within the
approved scope)
- office ingestion value (added in office, not captured in field)
- spatial / polygon lookup value (derived from containing boundary, selected
polygon, zone, package, estate, asset-owner area, or other reference geometry)
- runtime reference value (read from a selected 12d model/string/point/TIN/
alignment/reference surface)
- derived geometry value (calculated from Z, bearing, etc.)
- derived API/runtime value (MetaConnex, TIN surface, etc.)
- capture-new field requirement (attribute has no Veris equivalent — must be added to fieldcodes)
- manual review / blocked
Allowed values and lookup tables are first-class data, not comments. They must
feed pickers, rule validation, value-map validation, and export blockers.
The UI source-kind chooser must not flatten these into one vague "fixed value"
bucket. A human reviewer needs to know whether the value is captured in field,
copied from Veris, translated from a Veris choice, set once for the client, set
once for the project, entered in the office, calculated from references, looked
up from a polygon/zone, or blocked for manual review.
5. Three-Tier Value Mapping System
Translating Veris choice values to client choice values operates on three tiers.
The tiers apply in priority order: conditional overrides per-feature, per-feature
overrides global.
**Tier 1 — Global default value map (client-wide)**
Defined once per client, applies to all features that have the matching attribute.
Example: for this client, wherever Veris `Material = "Concrete"` appears, output
`Material = "CONC"`. This covers the bulk of all codes without manual work per code.
Global (Client: Vida)
Material:
"Concrete" → "CONC"
"Steel" → "STL"
"PVC" → "PVC-U"
"Ductile Iron" → "DI"**Tier 2 — Per-feature override**
For a specific Veris code / client feature where the global rule does not apply,
define an override. Only the exception cases need entries here. Everything else
inherits the global.
Per-feature override (Client: Vida, Feature: DRDUG)
Material:
"Concrete" → "RC" ← overrides global "CONC" for this feature only
(all others inherit global)**Tier 3 — Conditional rule**
When the correct output depends on more than one attribute value, use a conditional
rule. This overrides both tiers above when the condition matches.
Conditional (Client: Vida, Feature: DRDUG)
IF Material = "Concrete" AND Diameter > 300
THEN PipeClass = "RC3"
IF Material = "Concrete" AND Diameter <= 300
THEN PipeClass = "RC2"**Resolution order at export time:**
- Check for a matching Conditional rule → use it if matched
- Check for a Per-feature override → use it if found
- Fall back to Global default value map
- If no match in any tier → flag as unresolved (do not export a blank)
6. AI-Assisted Bulk Mapping With Human Review
The `/client-schema` skill and CSM do not require the user to manually assign
every attribute and every value one by one. That would take too long with 200+
codes and 20 attributes each.
**AI proposes, human approves:**
- The skill ingests the client's source documents (Excel, PDF, 12daz, mapfile).
- It runs all four steps of the mapping chain automatically:
- Suggests Veris code(s) for each client feature (Step 1)
- Suggests Veris attribute name matches for each client attribute (Step 2)
- Proposes global value maps by comparing Veris choices to client choices (Step 3)
- Flags attributes that need conditional logic based on choice combinations (Step 4)
- The user reviews the proposals in the Workstation. The app shows:
- Auto-matched items (green, one click to approve)
- Uncertain items (amber, need a decision)
- Unresolved items (red, must be handled before export)
- The user approves the bulk in one pass, adjusts exceptions, and adds conditionals
where the flat map is not enough.
The user's job is reviewing and correcting exceptions — not building from scratch.
The AI handles 80–90% of the work. The human handles the 10–20% that needs
business judgment.
**Per-client independence:**
Each client's value maps, overrides, and conditionals are stored independently.
"Concrete → CONC" for Client A does not affect Client B who might use "CON" or
"C25". The global tier is global *within* one client's schema, not across clients.
7. Self-Improving Skill Loop
The `/client-schema` skill improves over time through a documented correction loop.
**How the loop works:**
- The skill runs on a new client and proposes mappings (Steps 1–4).
- The user reviews in the Workstation and makes corrections:
- Changes a suggested Veris code to a better one
- Corrects a value map translation
- Rejects an attribute name match and picks the right one
- Adds a conditional the AI missed
- After review, a dedicated **skill improvement agent** reads the approved +
corrected state from the DB for this client and:
- Compares what the skill proposed vs what the user approved
- Extracts patterns: which corrections were made, why, what the right answer was
- Writes those corrections as new training examples in
`references/examples/<ClientName>_schema_guide.md`
- Updates `references/cm-training/MASTER_AI_TRAINING_GUIDE.md` with
cross-client patterns (e.g., "for Victorian infrastructure clients, 'RC' is
more common than 'CONC' for reinforced concrete drainage")
- The next client run starts with that accumulated knowledge.
**What the skill learns from:**
- Which Veris codes were accepted vs corrected for each feature type
- Which attribute name matches were accepted vs rejected
- Which value map translations were accepted vs corrected
- Which conditional rules were added that the AI missed
- Which global defaults held vs needed per-feature overrides
**What stays per-client vs what becomes cross-client knowledge:**
- Per-client: the final approved maps, overrides, and conditionals (stored in DB)
- Cross-client (skill training): patterns about asset types, naming conventions,
common value translations, and rule structures that appear across multiple clients
Current active focus: **Vida** (Phase 2 primary), then **UnityWater**. All other
clients (MRBC, WAADAC, Rspec, GoldCoast, SunshineCoast, QLDADAC, BrisbaneCC,
NswAdac, TFNSWUT, TFNSWMETA) are backlog. Do not target them in active commands until Vida
and UnityWater pipelines are proven. As more clients are processed and corrected,
the skill's first-pass accuracy improves and the human review burden shrinks.
7A. Truth Cases — Active Priority Order (updated 2026-05-28)
**Phase 2 primary target: Vida**
- Full end-to-end pipeline: schema → mapping → fieldcodes → simulated survey → 12d macro → deliverable.
- 40 feature codes, 28 VT_UTL_* attributes, choice lists, derivation rules.
- Must be proven before any other client pipeline is started.
- The `.planning/vida_12daz_extract_20260522_0216` extract is the primary reference fixture.
**Phase 2 secondary target: UnityWater**
- Starts after Vida pipeline is proven end-to-end.
- Must preserve examples such as `LotParcel -> SSBOU` and CM match-key driven mappings.
- Real ADAC mapping case.
**Backlog (after Vida + UnityWater proven):**
- **MRBC**: main ADAC proof case. Real ADAC schema, mapping, rules, XML/GML delivery.
- **WAADAC**: WA ADAC-facing proof case.
- **Rspec, GoldCoast, SunshineCoast, QLDADAC, BrisbaneCC, NswAdac, TFNSWUT**: council/client
cases. Their sample-data evidence must be re-audited against source and runtime evidence
before any production acceptance is claimed.
- **TFNSWMETA**: metadata/profile-only case. It must remain a CSM schema client,
but it has no physical feature-code mappings or field-capture export.
**The "all CSM schema clients" goal is aspirational, not the current build target.**
Do not treat it as active scope. Use Vida and UnityWater as the primary proof vehicles.
8. Rule Builder: Excel / Power Automate Style
The user should be able to build logic without writing free-text code.
The rule builder should feel closer to Excel formulas and Power Automate flows:
- guided function search
- required argument hints
- attribute/value pickers (always backed by the client's predefined choice list)
- structured IF/THEN/ELSE blocks
- value-map tables (the three-tier system in Section 5)
- fixed/default actions
- calculated expressions
- conditional cross-attribute logic
- runtime parameter selection such as TIN surfaces or model names
Examples:
- If `{Material} == "PVC"` then set `{PipeClass}` from a lookup table.
- If `{SurfaceLevel_m}` and `{InvertLevel_m}` exist, calculate
`{Depth_m} = {SurfaceLevel_m} - {InvertLevel_m}`.
- If the client requires an owner/department field, set a fixed default based on
the selected client/council.
- If a client attribute depends on another attribute's allowed value, use a
picker-backed condition and validated choice list.
- If the value changes per Veris job, read it from the active project runtime
profile instead of hardcoding it into the client schema.
- If the value depends on where the asset sits, use a spatial/polygon lookup
from a project-selected reference model or boundary.
Rules may reference the active Project Runtime Profile as a first-class input.
Examples:
Client attribute Owner
= PROJECT_DEFAULT("AssetOwner")
Client attribute ProjectNumber
= PROJECT_DEFAULT("ClientProjectNumber")
Client attribute Package
= POLYGON_LOOKUP("PackageBoundary", element_xy)
Client attribute Depth
= SURFACE_LEVEL(PROJECT_REF("DesignTIN"), element_xy) - element_z
Client attribute Responsibility
IF PROJECT_DEFAULT("Builder") = "Builder A"
AND Material = "Concrete"
THEN "Council"
ELSE "Developer"If a rule depends on a project value, the app must prevent export until the
active project profile supplies that value or explicitly marks it as manual.
If a rule depends on a runtime 12d reference, the profile must name the required
reference type and the macro must confirm it was supplied.
Every rule the user can build must be checked against actual execution
capability. If CSM cannot export/run it through 12dPL, Attribute Manipulator,
MetaConnex, ADAC/A-SPEC export, or another implemented runtime path, the UI must
mark it as blocked/manual with a reason. It must not pretend unsupported logic
is production-ready.
This safety contract applies to project/runtime values as well. The app must not
allow users to create a rule that looks valid in the UI but has no way for the
12d macro to receive the required project default, office value, polygon, TIN,
string, point, alignment, or other runtime reference.
**Client dropdowns are always predefined — never free text:**
Whenever a rule or value map sets a client attribute that has a defined choice
list, the value picker must only show choices from that list. Free-text entry for
a choice-backed attribute is not permitted. The app knows the client's choices
because they were imported in Step 2 of the mapping chain.
9. Workstation UI For The Mapping Chain
The Workstation is the human-facing interface for reviewing and adjusting AI
proposals across all four mapping steps. It must support:
**Feature-level view (Step 1 review):**
- Show each client feature code / element name
- Show the AI-suggested Veris code(s) with confidence indicator
- Allow the user to approve, change, or add additional Veris codes
- Show the bridge status (approved / pending / unresolved)
**Attribute name view (Step 2 review):**
- Side-by-side: Veris attributes on the left, client attributes on the right
- Show AI-suggested name matches with a match-strength indicator
- Allow approve / reject / remap per attribute
- Show unmatched client attributes as gaps (capture_new, fixed_default, etc.)
**Attribute Choices view (per-code choice configuration):**
The attribute is shared across client codes, but its valid choice list can differ
per code. "Pipe Diameter" is one attribute, yet a communication pipe and an
electrical pipe each need their own diameter choice list. The Workstation must
expose a dedicated screen to configure this directly — not by hunting feature by
feature.
- Left-hand panel lists every client attribute (e.g. "Comments", "Pipe Diameter",
"Material"). Selecting one is the entry point.
- Main area shows that attribute's full/default choice list **by default** — the
complete set of choices defined for the attribute.
- For each client code that carries the selected attribute, the user can **modify
the choices for that code**: trim the default list down, or set a code-specific
subset. The change is scoped to that (attribute, code) pair only and does not
touch the attribute's default list or any other code.
- A code with no explicit configuration **inherits the attribute's full default
choice list**. Only the exception codes need a per-code entry — the same
default / per-code-override shape as the value-map tiers in Section 5.
- Non-choice attributes (free text such as "Comments", or numeric/date) carry no
choice list; the screen shows them as having nothing to configure.
- The user does all of this through the UI. No manual DB editing, no per-code CSV
surgery.
Resolution: the choice list for an attribute on a given code is **the per-code
override if one exists, otherwise the attribute's default choice list.** This is
the configuration counterpart to the field-tool choice-list trim in Section 1C —
Section 1C trims what the *surveyor* sees in the field, whereas this view defines
what the valid choice list *is* per code for mapping, value maps, and validation.
**Value map editor (Step 3 — inline in attribute row expander):**
- For each mapped attribute pair where both sides have choice lists:
show a side-by-side translation table
- Left column: Veris choice values (read-only, from veris_attr_choices)
- Right column: client choice values (dropdown, from client_attr_choices)
- AI pre-fills the right column; user corrects any wrong suggestions
- "Apply as global default" button — promotes this map to Tier 1
- "This feature only" toggle — keeps it as Tier 2 per-feature override
- Unmapped Veris values are flagged red (cannot export until resolved)
**Conditional rule editor (Step 4 — accessible from attribute row):**
- Triggered by "Add condition" button on any attribute row
- IF block: attribute picker + operator + value picker (always choice-backed)
supports AND/OR for multi-attribute conditions
- THEN block: target client attribute + value picker (client choices only)
- Multiple conditions shown as stacked cards (Power Automate style)
- Each condition shows its tier (global / per-feature / conditional) and priority
**Batch operations:**
- "Approve all AI suggestions" button for bulk acceptance when the AI got it right
- "Apply global value map to all features" — propagates a confirmed Tier 1 map
to every feature that has the matching attribute
- Per-feature overrides are set by editing a single feature's row —
the global is not affected
10. 12d Execution And Export Reality
The system must produce artifacts that can run in the real 12d workflow.
**CSM is independent of CM at export time (updated 2026-05-29).** Earlier
revisions of this section treated CM as the canonical source of mapfile,
fieldcodes, Leica XML, and Trimble FXL outputs, with CSM "enriching" CM's
base files. That coupling is removed. CSM users will not have CM running on
their machine — they only receive a read-only copy of the CM master DB per
Section 0E. CSM must therefore generate the entire export stack itself,
from its own database and the synced `veris_code_attrs` / `veris_attr_choices`
reference data. The exporter code that previously lived only in CM is to be
ported into `csm/exporters/` so a CSM user can produce every delivery
artifact without invoking CM.
CM remains the canonical authority for the **master codelist** (Veris codes,
Veris attributes, Veris choices, Veris-native client mappings). CSM never
mutates `master_library.db`. But CSM owns the **export generators** that
read CSM's own client schema + the synced reference tables and produce all
shipping artifacts.
**The four shipping artifacts that matter (mandatory for every CSM client):**
- **`<Client>.mapfile`** — the 12d mapfile (Veris code ↔ client code/model/layer
linking). CSM generates the canonical mapfile from its own data, not a
stop-gap. The "CSM stop-gap mapfile" language from earlier revisions is
obsolete; the file CSM emits IS the canonical mapfile for that client.
- **`<Client>.12dfieldcodes`** — the 12d field-code list (UTF-16 LE, 12d
schema). CSM generates this directly from its `client_schema_features` +
`client_schema_attributes` + synced `veris_code_attrs`. No CM-side base
file is required.
- **`<Client>_Leica.xml`** — LandXML 1.2 field-code library for Leica
controllers. Must use namespace
`http://www.landxml.org/schema/LandXML-1.2` and contain non-zero
`<FeatureCode>` children. CSM-generated, not CM-enriched.
- **`<Client>_Trimble.fxl`** — Trimble Feature Coding Definitions (FXL
SchemaVersion 8, namespace `http://trimble.com/schema/fxl`). Must
contain non-zero `<FeatureDefinition>` children. CSM-generated.
**The three shipping PDFs (mandatory for every CSM client):**
- **`<Client>_field_code_list.pdf`** — surveyor-facing reference. One row
per visible field code with the codes/attributes the surveyor must
capture. Filtered by `field_visible` per Section 1C.
- **`<Client>_attributes_breakdown.pdf`** — per-feature attribute roll-up.
For each client feature: every attribute with its `source_kind` (per
Section 0B), its choice list (where applicable), its mandatory flag,
and a note where it has no executable source path
(`blocked_manual`). This is the "where does this value come from?"
document referenced in the UI/UX section.
- **`<Client>_schema_summary.pdf`** — office-handoff overview. Schema
version, feature/attribute/choice counts, mapping coverage percentage,
unresolved Tier-1/2/3 value-map gaps, capture-action backlog count,
project runtime profile readiness. One-page status snapshot per
client per schema version.
**Additional outputs (where applicable):**
- project runtime configuration/template for the 12d macro
- `.12dattmf`
- `.12dMetaConnex`
- `.4dm` macro pack
- evidence CSV
- ADAC XML/GML for ADAC-standard clients only
**Explicitly out of scope:**
- A-SPEC GIS-ready tables or datasets — do not build.
- Calling out to a running CM instance at export time — CSM must export
from its own DB without CM as a live dependency.
**Export correctness contract.** A file that exists with the right magic
bytes but zero feature codes is FAIL, not PASS. Every CSM exporter must be
validated at three depths before being shipped:
- EXISTS — file is created, non-empty.
- WELL-FORMED — parses (XML / PDF magic / UTF-16 BOM).
- CORRECT — schema version matches expected (LandXML 1.2, FXL
SchemaVersion 8), feature/definition count > 0 and matches the CSM DB
count for that client within a defined tolerance, freshness (mtime
after the most recent export invocation), and CM-parity-or-better
where comparable (CSM-only feature codes that CM does not carry must
appear; CM-side feature codes that CSM still inherits via the synced
reference data must also appear).
This contract is auditable by `_planning/run_export_audit_2026-05-29.py`.
A CSM build that does not pass that audit for every CSM-owned client is
not shippable.
Attribute Manipulator and MetaConnex are useful 12d-native reference tools and
export adapters, not mandatory architecture. They explain many of the same
operations CSM needs: copy attributes, set defaults, convert types, validate
schemas, and apply rule-like transformations. CSM should reuse them when they
fit cleanly, but the production system is allowed to generate its own 12dPL
runtime code whenever CSM's Project Runtime Profile, Excel/Power Automate-style
rules, fallback chains, polygon/TIN/string references, or bulk value logic is
too complex for native ATTMF/MetaConnex.
The official 12d documentation for Attribute Manipulator and MetaConnex should
be used as a reference model for behaviour and UI patterns, not as a limit on
what CSM may implement. If the documented tool shows useful concepts such as
ordered rules, type conversion, evaluated defaults, panel variables, validation
tabs, or calculated defaults, CSM may reproduce those concepts in its own rule
model and compile them to generated 12dPL when that gives a clearer, more
auditable runtime than forcing the logic through the native files.
The execution contract is therefore capability-based:
- use `.12dattmf` for simple static/client-schema-safe attribute copies and
stable fixed defaults;
- use `.12dMetaConnex` where its schema/apply/validate model genuinely executes
the required behaviour;
- use CSM-generated 12dPL as the primary runtime engine for project/job values,
operator-entered defaults, TIN/model/string/polygon selections, geometry
calculations, fallback chains, value maps, conditionals, and evidence;
- mark anything unsupported as blocked/manual until an executable path is built
and verified.
Treat the native tools as adapters inside the CSM runtime, not as the runtime
itself. A rule is ready only when the chosen path is explicit and proven:
native ATTMF, native MetaConnex, CSM-native 12dPL, another verified exporter, or
blocked/manual with exact unblock criteria.
Do not force project-specific values into static ATTMF/MetaConnex files just
because those formats can hold a default. A builder/job/project value must come
from the active Project Runtime Profile or from a 12d runtime input that is
written back to evidence.
Native 12d GUI automation must not be treated as successful just because a
return code is zero. CSM needs evidence that attributes were actually written,
outputs were created, and validation passed.
Project runtime configuration is not optional for real jobs. Even when a client
schema is fully mapped, each Veris job still needs the selected project profile
that supplies job numbers, project defaults, office values, selected references,
and the export/evidence destination for that run.
For production, the runtime configuration must be written as the saved Project
Runtime Profile file before the macro runs. The macro should receive the file
path as an input parameter or via a small launcher command. It should not depend
on unsaved UI state. A run is not auditable unless the exact profile file used
by 12d is preserved.
The 12d macro panel is profile-driven. CSM defines the runtime parameters for
the active client/profile, then the macro exposes matching inputs for that run:
one TIN if the profile needs one, ten TIN boxes if it declares `TIN_A` through
`TIN_J`, plus any required source models, boundary/polygon models, strings,
alignments, fixed values, office values, and output model prefix. The fixed
macro is generic; the Project Runtime Profile decides which runtime bindings
are required.
For agent testing, the same macro may run in baked-input auto-run mode: command
arguments or a fixture profile pre-fill the same inputs a human would fill, and
the macro skips the Run-button wait. This is test automation only. Production
operators still use the panel, confirm/fill the runtime references, and press
Run.
At runtime the generated macro must:
- read the saved CSM Project Runtime Profile file
- apply the approved attribute/value/rule mappings
- read project default and office-ingestion values
- resolve selected 12d references such as models, strings, points, TINs,
alignments, boundaries, and polygons
- execute spatial/polygon lookup rules where configured
- write evidence identifying the active client, schema version, project profile,
profile file path/hash, source values, resolved values, and output paths
11. Validation And Review
CSM must expose a validation centre that shows:
- missing required attributes
- invalid lookup/choice values
- feature codes not linked to Veris codes
- attributes mapped to unavailable Veris attributes
- value map entries with no client choice match
- unresolved Tier 1/2/3 map gaps (Veris value has no client translation)
- illegal 12d model/layer names
- unsupported rule functions
- manual office-ingestion requirements
- missing required project runtime values
- missing saved Project Runtime Profile file for a real run
- Project Runtime Profile file not matching the selected client/schema/version
- missing or unresolved runtime references
- project defaults incorrectly stored as client-wide schema values
- stale project profile values reused on the wrong job/client/schema/version
- macro/export blockers
- ADAC/A-SPEC validation errors
- evidence from runtime runs
The user should see what is ready, warning, blocked, or manual. No fake green
status.
12. Delivery Pack
The final product should help Veris deliver, **all generated by CSM itself
without requiring a running CM**:
**Four core export artifacts (per Section 10):**
- `<Client>.mapfile` — canonical 12d mapfile (Veris ↔ client code linking)
- `<Client>.12dfieldcodes` — 12d field-code list (UTF-16 LE)
- `<Client>_Leica.xml` — LandXML 1.2 field-code library
- `<Client>_Trimble.fxl` — Trimble FXL SchemaVersion 8
**Three PDFs (per Section 10):**
- `<Client>_field_code_list.pdf` — surveyor reference (visible codes only)
- `<Client>_attributes_breakdown.pdf` — per-feature attribute roll-up with
source_kind, choice lists, mandatory flags, blocked-manual flags
- `<Client>_schema_summary.pdf` — office-handoff: counts, coverage %,
validation gaps, runtime-profile readiness
**Additional (where applicable):**
- ADAC XML/GML for ADAC-standard clients
- per-project runtime templates/configuration for the 12d macro
- saved Project Runtime Profile files for each real project/job
- evidence reports
- reduced rework and faster acceptance by asset owners
**Not in scope:** A-SPEC GIS deliverables, GIS-ready datasets, A-SPEC XML.
**Independence contract.** A CSM user with only their local CSM install +
the synced read-only copy of `master_library.db` must be able to produce
every artifact above for their assigned client(s). No call into CM at
export time. No dependency on a CM-side `<client>_veris_assigned.12dfieldcodes`
or any other CM-exported base file. The shipping audit
(`_planning/run_export_audit_2026-05-29.py`) verifies this contract for
every CSM-owned client × every artifact above.
**The complete supported path:**
new client onboarding
→ new-client wizard (name → schema → CSV import or manual)
→ sync Veris master DB from CM (Import Veris Master DB)
→ client checklist / scope selection
→ schema intake: CSV bulk import OR manual feature/attr entry
→ Step 1: client feature → Veris code linking
→ Step 2: client attribute → Veris attribute name linking
→ Step 3: client choice values ↔ Veris choice values (3-tier value maps)
→ Step 4: conditional rules (multi-attribute IF/THEN)
→ human review and approval (Workstation)
→ skill improvement agent updates training examples
→ project runtime profile: job defaults / office values / references
→ save project runtime profile file
→ 12d macro reads saved profile file and applies it to the project
→ 12d/runtime execution
→ validation and evidence
→ delivery
→ export client DB → send to Abdullah (hub-and-spoke collaboration)---
UI/UX Direction
The UI should be built around how a human actually uses the system:
- side-by-side Veris attributes and client attributes
- a clear assignment/mapping area with status and source kind columns
- row expanders for detailed rule/value-map/office/default settings
- project setup/profile screen for job-specific defaults, office values, and
12d references
- save/duplicate/import/export controls for Project Runtime Profile files
- profile validation showing missing values, missing 12d references, stale
schema version, and whether the file is ready for macro execution
- guided pickers, not free typing where choices are known
- value-map table editor inline in the row expander (three-tier, Section 5)
- function hints like Excel
- flow cards like Power Automate for conditional rules
- batch approve + global propagation controls
- validation/evidence centre that points back to the affected feature, mapping,
rule, or export
The UI must help a user understand what the system can do. Unsupported options
should be visible as blocked/manual with explanation, not hidden and not fake.
---
Hard Boundaries
- Do not rebuild the old CAM mental model.
- Do not treat test pass or parser success as production delivery.
- Do not map attributes before feature/code linking (Steps 1 before 2, 2 before 3).
- Do not invent Veris attribute mappings where the linked Veris code does not
contain that attribute.
- Do not let client choice values be free-text where the client has a predefined
choice list — the picker must enforce the list.
- Do not let a value map export with unresolved Veris values (red gaps block export).
- Do not let illegal 12d model/layer names reach export.
- Do not present a rule as executable unless the backend/exporter/runtime path
exists and is verified.
- Do not assume Attribute Manipulator or MetaConnex must be the execution engine.
They are optional adapters; CSM-generated 12dPL is the correct path when the
rule needs project profile inputs, runtime references, or logic the native
tools cannot honestly execute.
- Do not store project/job-specific values as permanent client-schema defaults
unless they are truly client-wide.
- Do not run macro/export work without an explicit active project runtime
profile for real jobs.
- Do not rely on unsaved UI state, transient database rows, or macro memory as
the only source of project runtime values. Every real job needs a saved Project
Runtime Profile file.
- Do not call ADAC deliverables complete without validation and evidence.
- Do not apply one client's value maps or overrides to another client's schema.
Per-client independence is non-negotiable.
- **Do not build A-SPEC GIS export, A-SPEC XML, or GIS-ready dataset paths.**
These are explicitly out of scope for CSM. If an audit flags their absence as a
gap, discard the finding.
- **Do not create CSM schemas for Veris-native clients.** Veris-native clients are
CM's responsibility. If a client uses Veris attribute names and choice values
as-is, they do not need CSM.
- **Do not write to CM's `master_library.db`.** CSM reads it; CM owns it. Any Veris
master change must go through CM, not CSM.
- **Do not start Phase 2 work (Vida pipeline, 12d macro) until Phase 1 (CSM app
perfection) is complete.** Phase 2 depends on a stable, distributable CSM app.
---
Durable Agent Instruction
When unsure, return to the north star:
CM/CSM exists so Veris can produce compliant client asset data correctly the first
time, with the mapping, attribution, rules, validation, and delivery evidence all
controlled by the system — for any delivery standard, any client.
**Architecture in one line:** CM = Veris master DB + Veris-native clients. CSM = custom
attribute clients, syncs from CM, distributes via file-based hub-and-spoke.
**Current phase:** Phase 1 — perfect the CSM app. Wizard, bulk import, Sync & Share,
all gaps resolved. Do not start Phase 2 (Vida pipeline, 12d macro) until this is done.
**Priority clients:** Vida first (Phase 2 target). UnityWater second. Everything else
is backlog.
**Only 3 exports matter:** `.12dfieldcodes`, Leica XML, Trimble FXL. All other exports
are secondary or out of scope.
**A-SPEC / GIS:** Out of scope. Discard any finding that flags their absence.
The four-step chain (Section 3) is the spine of everything. The three-tier value
map system (Section 5) is how bulk translation scales. The AI-human review model
(Section 6) is how the work gets done efficiently. The self-improving skill loop
(Section 7) is how it gets better with every client. The distributed collaboration
model (Section 0E) is how multiple staff work simultaneously without conflicts.
Do not shortcut any of these. Do not re-architect them without reading this
document first and getting explicit direction from Abdullah.