# TraderX > TraderX This file contains all documentation content in a single document following the llmstxt.org standard. ## FINOS | TraderX Sample Trading App ![DEV Only Warning](https://badgen.net/badge/warning/not-for-production/red) ![Local Dev Machine Supported](http://badgen.net/badge/local-dev/supported/green) TraderX is a simple, distributed reference application for exploring trading workflows in financial services. It is intentionally approachable and non-production, and is meant for education, experimentation, and demos. ## Start here - [Overview](overview.md) for architecture and the C4-inspired diagram. - [Flows](flows.md) for sequence diagrams of core user journeys. - [Code map](code.md) for links to each service. - The [Running TraderX](running.md) guide for run options (manual, Docker, Kubernetes/Tilt). ## What is in this docs folder - `overview.md` contains the architectural diagram and links to system context. - `flows.md` contains sequence diagrams for key workflows. - `code.md` links to each service root. - `home.mdx`, `roadmap.mdx`, `running.md`, and `project-history.md` are Docusaurus site pages. - `c4/` contains the original C4 DSL and rendered image. You can view more details here: * [Generated State Branches](spec-kit/generated-state-branches.md) * [Learning Guides](learning/index.md) * [Overview](overview.md) * [State 001 Architecture](/specs/baseline-uncontainerized-parity/system/architecture) * [State 001 Flows](/specs/baseline-uncontainerized-parity/system/end-to-end-flows) --- ## ADR-001 Adopt GitHub Spec Kit # Adopt GitHub Spec Kit as the Canonical Specification Workflow ## Context and Problem Statement TraderX is being repositioned as a spec-driven learning platform where implementation is generated from requirements instead of copied from legacy source. The repository previously mixed historical implementation guidance with multiple migration approaches, which made the project difficult to understand and evolve consistently. The core decision is how to standardize specification authoring, planning, task execution, traceability, and verification so future state transitions can be implemented from specs with predictable quality. ## Decision Drivers * Need one canonical specification workflow for requirements, stories, and implementation planning. * Need stronger traceability from system-level requirements to generated component behavior. * Need onboarding simplicity for new contributors. * Need compatibility with AI-assisted, iterative generation workflows. * Need documentation that can expose specs and governance artifacts directly in the docs portal. ## Considered Options * Continue with custom TraderSpec-only scripts and local conventions. * Build a fully custom internal specification framework. * Adopt GitHub Spec Kit as the canonical workflow and align repo structure/process around it. ## Decision Outcome Chosen option: "Adopt GitHub Spec Kit as the canonical workflow and align repo structure/process around it", because it provides an opinionated, documented, and repeatable path for moving from requirements to implementation while preserving traceability and governance artifacts. ### Consequences * Good, because contributors can follow a known workflow (`specify` -> `plan` -> `tasks` -> implement) with less ambiguity. * Good, because specs, constitution, and execution artifacts become first-class and easier to review. * Good, because generation quality can be improved by enriching requirements and non-functional constraints instead of changing ad hoc scripts. * Bad, because existing legacy docs and workflows must be clearly deprecated or removed to avoid confusion. * Bad, because initial migration overhead is non-trivial and requires disciplined document maintenance. ### Confirmation Decision compliance is confirmed when all of the following are true: * Root Spec Kit artifacts remain the source of truth (`.specify/**`, `specs/**`). * Generated baseline components can be produced and validated from spec-first pipelines. * Docusaurus nav exposes Spec Kit artifacts and hides deprecated legacy guide material. * CI gate scripts for Spec Kit readiness and coverage continue to pass. ## Pros and Cons of the Options ### Continue with custom TraderSpec-only scripts and local conventions * Good, because migration effort is lowest in the short term. * Neutral, because existing contributors already know the current scripts. * Bad, because it keeps process semantics implicit and harder for new contributors to follow. * Bad, because it reduces external alignment and discoverability. ### Build a fully custom internal specification framework * Good, because the process can be tailored exactly to TraderX. * Neutral, because it can model learning-path-specific needs deeply. * Bad, because long-term maintenance cost is high. * Bad, because governance and conventions become project-specific and harder to transfer. ### Adopt GitHub Spec Kit as the canonical workflow and align repo structure/process around it * Good, because it provides a recognized structure for specs, planning, and tasks. * Good, because it fits the goal of regenerating code from explicit requirements and constraints. * Neutral, because some TraderX-specific conventions still need local adaptation. * Bad, because migration requires cleanup of legacy content and re-training contributors. ## More Information Primary reference: [GitHub Spec Kit](https://github.github.com/spec-kit/index.html) Related repository decisions: * Branching strategy for generated state snapshots. * Deprecation of pre-SpecKit guide material under `docs/guide/**`. --- ## ADR-002 Generated State Branching Strategy # Use a Two-Surface Branching Strategy for Spec Sources and Generated Code States ## Context and Problem Statement TraderX must support two valid consumer modes: 1. contributors who evolve requirements and generation logic through GitHub Spec Kit artifacts, and 2. developers who want a directly runnable code snapshot for a specific learning state. Keeping only generated code in `main` would reduce clarity around source-of-truth requirements. Keeping only spec artifacts in `main` would make it harder for code-first users to consume state snapshots quickly. The decision is how to organize branches so the repository stays spec-first while still publishing easy-to-consume generated state outputs. ## Decision Drivers * Preserve specs as the canonical source of truth. * Provide discoverable generated snapshots for code-first users. * Avoid creating and maintaining multiple repositories. * Keep CI and docs workflows manageable. * Support future learning-path state transitions with minimal operational overhead. ## Considered Options * Single branch with only generated code. * Two separate repositories (one spec repo, one generated-code repo). * Single repository with spec-first `main` plus dedicated generated-state branches. ## Decision Outcome Chosen option: "Single repository with spec-first `main` plus dedicated generated-state branches", because it preserves one project identity while separating authoring concerns (specification) from distribution concerns (generated runnable snapshots). ### Consequences * Good, because `main` remains cleanly focused on specs, tooling, and governance. * Good, because generated branches can be consumed without needing full Spec Kit context. * Good, because branch diffs between generated states become a practical learning artifact. * Bad, because release discipline is required to ensure generated branches are reproducible from `main`. * Bad, because documentation must clearly explain the difference between spec-source and generated-state branches. ### Confirmation Decision compliance is confirmed when all of the following are true: * `main` stores canonical specs and generation pipelines, not checked-in generated outputs. * Generated outputs are published in dedicated branches using an explicit naming convention. * Each generated branch references the source spec commit and generation command set used. * Smoke tests pass for generated runtime flows before a generated branch is published. ## Pros and Cons of the Options ### Single branch with only generated code * Good, because code-first consumption is straightforward. * Neutral, because branch management overhead is low. * Bad, because requirements and planning intent are no longer first-class. * Bad, because regeneration provenance becomes weak. ### Two separate repositories (one spec repo, one generated-code repo) * Good, because separation of concerns is explicit. * Good, because access controls and release cadence can differ by repo. * Bad, because cross-repo synchronization and governance overhead increases. * Bad, because contributors must navigate two project homes. ### Single repository with spec-first `main` plus dedicated generated-state branches * Good, because it keeps one canonical project with clear authoring vs distribution surfaces. * Good, because it supports both spec-driven and code-first contributor workflows. * Neutral, because branch conventions and automation must be maintained. * Bad, because process quality depends on consistent release hygiene. ## More Information Operational notes: * Generated-state branch naming should be documented centrally (for example, `generated//`). * Branch publish flow should include generation commands, validation output, and traceability links back to source specs. --- ## ADR-003 Intentional Legacy-Shaped Baseline # Keep the First Learning State Intentionally Simplistic and Legacy-Shaped ## Context and Problem Statement The first TraderX learning state is the baseline from which users will evolve architecture and quality characteristics across multiple learning paths. If this baseline starts with mature modern practices already in place, users lose the ability to understand why those practices were introduced and what they changed. The project needs a starting state that resembles legacy-style distributed applications commonly found in larger enterprises: chatty service calls, minimal platform automation, and limited cross-cutting engineering practices. ## Decision Drivers * Preserve educational contrast between baseline and later improved states. * Represent realistic legacy starting conditions. * Minimize conceptual load for first-time users. * Enable clear, incremental demonstrations of DevEx, NFR, and functional evolution. * Avoid prematurely constraining future learning-path design. ## Considered Options * Start with a production-grade modern baseline. * Start with a heavily mocked single-process demo. * Start with a simple distributed baseline that is intentionally legacy-shaped. ## Decision Outcome Chosen option: "Start with a simple distributed baseline that is intentionally legacy-shaped", because it best supports progressive learning and makes the value of each subsequent architecture or quality improvement explicit. ### Consequences * Good, because learners can see concrete before/after impact for each improvement step. * Good, because the baseline matches conditions many enterprise teams actually inherit. * Good, because later state transitions can be mapped to specific pain points rather than abstract goals. * Bad, because the baseline intentionally violates some modern best practices. * Bad, because some users may initially mistake baseline limitations for target end-state guidance. ### Confirmation Decision compliance is confirmed when baseline documentation and generated runtime behavior show: * direct multi-service communication patterns without advanced platform abstractions, * no mandatory containerization, ingress, or auth assumptions in the first state, and * explicit learning-path references showing where these concerns are introduced later. ## Pros and Cons of the Options ### Start with a production-grade modern baseline * Good, because immediate technical quality is higher. * Good, because fewer remediation steps are needed before real-world deployment. * Bad, because it reduces educational visibility into why improvements matter. * Bad, because it increases upfront complexity for beginners. ### Start with a heavily mocked single-process demo * Good, because setup complexity is minimal. * Neutral, because local onboarding can be fast. * Bad, because it fails to represent distributed-system realities. * Bad, because later transitions to realistic architecture become abrupt. ### Start with a simple distributed baseline that is intentionally legacy-shaped * Good, because it balances realism and approachability. * Good, because it creates a clear narrative for incremental upgrades. * Neutral, because some baseline imperfections are intentional by design. * Bad, because explicit communication is needed to avoid confusion about quality targets. ## More Information This decision is paired with the minimal-runtime dependency decision to keep baseline setup accessible while preserving replaceability of infrastructure choices. --- ## ADR-004 Lightweight Baseline Infra and Replaceability # Prefer Lightweight Default Infrastructure and Explicit Replaceability in Baseline ## Context and Problem Statement The baseline state must be runnable by users with minimal local setup and without requiring heavy infrastructure installation. Using production-grade infrastructure by default (for example, Kafka clusters and heavyweight database setups) raises onboarding cost and conflicts with the goal of fast experimentation. At the same time, the architecture must not lock users into toy technologies. The project should make it clear that messaging, storage, and runtime components are replaceable through spec-driven evolution. ## Decision Drivers * Minimize friction for first-time local runs. * Avoid requiring complex external infrastructure for baseline usage. * Preserve ability to swap infrastructure for functional or non-functional reasons. * Keep learning-path transitions explicit and measurable. * Support spec-driven regeneration across infrastructure choices. ## Considered Options * Use production-grade infrastructure by default in baseline. * Use lightweight local defaults and define replacement paths in specs. * Provide no default infrastructure and require users to choose everything up front. ## Decision Outcome Chosen option: "Use lightweight local defaults and define replacement paths in specs", because it optimizes for accessibility while preserving architecture evolution and component replaceability. ### Consequences * Good, because baseline setup is simple and fast for most users. * Good, because users can run and inspect behavior without specialized platform operations. * Good, because later learning paths can intentionally introduce Kafka, stronger databases, or other infrastructure upgrades as explicit changes. * Bad, because baseline defaults are not intended for production characteristics. * Bad, because docs must clearly communicate that infrastructure choices are educational defaults, not universal recommendations. ### Confirmation Decision compliance is confirmed when: * baseline runtime can start with lightweight local dependencies, * specs describe upgrade paths for messaging and persistence engines, * generated implementations keep infrastructure boundaries explicit enough to swap components, and * learning-path documentation identifies where and why each infrastructure substitution is introduced. ## Pros and Cons of the Options ### Use production-grade infrastructure by default in baseline * Good, because production-like behavior is available from day one. * Good, because fewer later infrastructure migrations are required. * Bad, because onboarding complexity is significantly higher. * Bad, because it increases operational burden for users who only want to learn core flows. ### Use lightweight local defaults and define replacement paths in specs * Good, because it is approachable for beginners and fast to run locally. * Good, because it supports progressive enhancement through explicit spec changes. * Neutral, because baseline defaults intentionally trade robustness for accessibility. * Bad, because users need clear guidance to move to stronger infrastructure. ### Provide no default infrastructure and require users to choose everything up front * Good, because maximum flexibility is preserved. * Neutral, because advanced users can tailor the stack immediately. * Bad, because newcomer onboarding becomes ambiguous and fragmented. * Bad, because cross-user reproducibility suffers. ## More Information This decision complements the intentional legacy-shaped baseline decision and informs future learning-path deltas for containerization, messaging hardening, and database modernization. --- ## ADR-005 Use NATS for State 006 Messaging Replacement # Use NATS as the Messaging Backbone in State 006 ## Context and Problem Statement The baseline and early states used a Socket.IO-based messaging service (`trade-feed`) for both backend event propagation and frontend live updates. This was intentionally simple, but it became brittle for architecture evolution: - backend inter-process messaging was coupled to a browser-oriented protocol and conventions, - eventing semantics were harder to reason about as the system evolved, - multi-language service interoperability required extra glue and custom handling. State `006-messaging-nats-replacement` introduces a core architecture-track step after state `005-postgres-database-replacement`. We needed a lightweight but robust messaging foundation that remains easy for local users while improving correctness and extensibility. State scope: this ADR applies specifically to state `005` and descendants that inherit its messaging model. ## Decision Drivers * Improve backend pub/sub clarity and robustness without increasing local setup burden. * Keep browser realtime streaming support for the Angular UI. * Support Java, .NET, Node.js, and Python service clients. * Keep incremental migration from state `003` possible and testable. * Preserve simple containerized local runtime for contributors. ## Considered Options * Keep Socket.IO trade-feed as-is. * Replace trade-feed with NATS broker + subject model. * Replace trade-feed with MQTT broker. ## Decision Outcome Chosen option: "Replace trade-feed with NATS broker + subject model", because it provides the best balance of local simplicity, multi-language interoperability, and architecture clarity while preserving browser websocket support. ### Consequences * Good, because backend eventing moves to a purpose-built messaging protocol with clear subjects. * Good, because runtime remains lightweight and easy to run in Docker Compose. * Good, because browser realtime remains available through ingress websocket routing. * Good, because future states can add durability and richer patterns incrementally. * Bad, because migration requires touching multiple services and frontend stream handling. * Bad, because maintainers must preserve contract and flow compatibility during transport changes. ### Confirmation Decision compliance is confirmed when: * `trade-feed` is removed from state `005` runtime topology and replaced by `nats-broker`, * trade submit and processing flows continue to pass conformance/smoke checks, * frontend realtime trade/position updates remain functional via websocket route, * generated code and docs retain state lineage and compareability against state `003`. ## Pros and Cons of the Options ### Keep Socket.IO trade-feed as-is * Good, because no migration effort is required. * Good, because existing behavior is already known. * Bad, because backend messaging remains tied to browser-centric conventions. * Bad, because architecture evolution keeps accumulating protocol friction. ### Replace trade-feed with NATS broker + subject model * Good, because NATS is lightweight and straightforward for local/container use. * Good, because subject routing and wildcard patterns fit account-scoped event streams. * Good, because ecosystem support across TraderX service languages is strong. * Neutral, because browser streaming still needs ingress websocket routing discipline. * Bad, because migration requires cross-component coordination and test hardening. ### Replace trade-feed with MQTT broker * Good, because MQTT is also lightweight and broadly supported. * Good, because topic-based routing can fit event fan-out patterns. * Bad, because it is less aligned with our immediate request/reply and service-pattern direction. * Bad, because it adds selection ambiguity when the team already validated NATS pathing. ## More Information Related state and artifacts: * State pack: `/specs/messaging-nats-replacement` * Learning guide: `/docs/learning/state-006-messaging-nats-replacement` * Generated code branch: `code/generated-state-006-messaging-nats-replacement` --- ## ADR-006 Use PostgreSQL for State 005 Database Replacement # Use PostgreSQL as the Database Engine in State 005 ## Context and Problem Statement State `004-containerized-compose-runtime` uses H2 as the runtime database. That choice kept local startup simple, but it limited realism for SQL behavior and migration paths. We need an architecture-track branch that swaps only the DB engine while preserving baseline functional behavior and simple local execution. State scope: this ADR applies specifically to state `005-postgres-database-replacement` and descendants that inherit its database model. ## Decision Drivers * Keep baseline behavioral compatibility for account, trade, and position flows. * Improve realism of SQL runtime behavior in a local-friendly setup. * Keep startup simple in Docker Compose. * Keep migration incremental and traceable from state `003`. ## Considered Options * Keep H2 runtime as-is. * Replace H2 with PostgreSQL container. * Replace H2 with MongoDB container. ## Decision Outcome Chosen option: "Replace H2 with PostgreSQL container", because it improves database realism while preserving local simplicity and minimizing contract drift. ### Consequences * Good, because SQL behavior is closer to common production patterns. * Good, because official PostgreSQL container is straightforward for local Compose usage. * Good, because this replacement is isolated to one architecture axis. * Bad, because DB-dependent service configuration and sequence handling require migration changes. * Bad, because H2 web-console diagnostics are no longer part of the runtime. ### Confirmation Decision compliance is confirmed when: * runtime uses PostgreSQL container with deterministic init SQL, * DB-dependent services run against PostgreSQL and baseline flows pass, * REST/event contracts remain baseline-compatible, * generated state and docs are publishable as state `004`. ## Pros and Cons of the Options ### Keep H2 runtime as-is * Good, because no migration effort is required. * Good, because existing local behavior is already known. * Bad, because SQL runtime realism remains limited. ### Replace H2 with PostgreSQL container * Good, because PostgreSQL is widely used and production-like for SQL behavior. * Good, because local Compose setup remains simple. * Good, because migration can stay constrained to DB/runtime axis. * Bad, because config and schema-init migration work is required. ### Replace H2 with MongoDB container * Good, because operational setup can also be simple in containers. * Bad, because current schema/query model is relational and SQL-oriented. * Bad, because this would introduce a larger functional model change than desired for this step. ## More Information Related state and artifacts: * State pack: `/specs/postgres-database-replacement` * Learning guide: `/docs/learning/state-005-postgres-database-replacement` * Generated code branch: `code/generated-state-005-postgres-database-replacement` --- ## ADR-007 Adopt LGTM + OpenTelemetry Stack for State 007 Baseline # Adopt LGTM + OpenTelemetry Stack from State 007 Onward ## Context and Problem Statement Before state `006`, TraderX had no consistent, end-to-end observability baseline in generated runtime states. We needed a repeatable, local-first observability platform that works across all services and can be layered onto multiple state branches without changing functional behavior. State `007-observability-lgtm-compose` introduces this baseline on the canonical architecture path. Descendant states (including `007`, `008`, and convergence states) inherit the same observability decision unless explicitly changed. State scope: this ADR applies specifically to state `006` and descendants that inherit its observability model. ## Decision Drivers * Keep observability open-source and runnable locally in Docker Compose. * Provide logs, metrics, traces, and health probing in one baseline. * Keep functional flows unchanged while improving operational visibility. * Support pre-provisioned dashboards for immediate value to users. * Keep generated-state behavior deterministic and testable in CI. ## Considered Options * Minimal logging-only stack (for example Loki + Promtail + Grafana). * Metrics-only stack (Prometheus + Grafana). * Full LGTM + OpenTelemetry stack in Compose: * Grafana * Loki * Tempo * Prometheus * OpenTelemetry Collector * Promtail * Blackbox Exporter ## Decision Outcome Chosen option: "Full LGTM + OpenTelemetry stack in Compose", because it provides complete, local-first observability coverage while preserving state behavior and allowing consistent instrumentation across future branches. ### Consequences * Good, because users get operational visibility without extra infrastructure. * Good, because the same observability baseline can be reused across state branches. * Good, because dashboards, probes, and scrape targets can be tested and versioned in specs. * Bad, because runtime complexity and startup time increase versus app-only states. * Bad, because dashboard/provisioning drift must be managed carefully in generated artifacts. ### Confirmation Decision compliance is confirmed when: * state `006` runtime includes Grafana, Loki, Tempo, Prometheus, OTel Collector, Promtail, and Blackbox Exporter, * observability endpoints are healthy in smoke checks (`/api/health`, `/-/ready`, `/ready`, collector health), * provisioned dashboards are available by default, including TraderX service/runtime views, * descendant states that keep this stack continue to expose healthy observability endpoints. ## Pros and Cons of the Options ### Minimal logging-only stack * Good, because setup is smaller and faster. * Bad, because metrics and traces are missing. * Bad, because non-functional comparisons across states are weaker. ### Metrics-only stack * Good, because it covers service health and basic SLO signals. * Bad, because root-cause analysis is limited without centralized logs/traces. * Bad, because operational demonstrations are incomplete. ### Full LGTM + OpenTelemetry stack * Good, because logs/metrics/traces/probes are all available. * Good, because Grafana can present a unified operational view. * Good, because this becomes a reusable observability baseline for future states. * Bad, because it adds operational overhead for local runs. ## More Information Related states and artifacts: * State pack: `/specs/007-observability-lgtm-compose` * Learning guide: `/docs/learning/state-007-observability-lgtm-compose` * Generated code branch: `code/generated-state-007-observability-lgtm-compose` --- ## ADR-008 Convergence State Model (C0-C3) # Adopt Explicit Convergence States (C0-C3) with Single-Parent Publish Lineage ## Context and Problem Statement TraderX now contains multiple tracks (architecture, functional, and platform/devex). We need a predictable way to: - preserve clean single-parent branch ancestry for generated code publishing, - show where tracks converge into recommended jump-off points, - avoid ad hoc multi-parent lineage that complicates automation and compareability. ## Decision Drivers * Keep generated branch ancestry linear and reproducible. * Make “best starting points” explicit for future feature design. * Preserve track-level learning visibility without forcing merge ancestry. * Keep state model machine-validated through CI gates. ## Considered Options * Keep only plain state lineage (`previous`) and no explicit convergence semantics. * Allow true multi-parent lineage for publish ancestry. * Keep single-parent publish lineage and add explicit convergence metadata plus dotted-line parents. ## Decision Outcome Chosen option: "Keep single-parent publish lineage and add explicit convergence metadata plus dotted-line parents". ### Consequences * Good, because generated branch ancestry remains deterministic and CI/publish tooling stays simple. * Good, because documentation can still show where tracks converge (`C0`, `C1`, `C2`, `C3`). * Good, because maintainers can recommend stable jump-off points for new work. * Bad, because convergence context now requires additional metadata governance. * Bad, because dotted-line parents must be documented carefully to avoid confusion with publish ancestry. ### Confirmation Decision compliance is confirmed when: * every state in `catalog/state-catalog.json` defines: - `convergenceLevel`, - `isConvergence`, - `dottedParents`, - `primaryLineageRole`. * `previous` remains single-parent (`length <= 1`) for all states. * non-convergence states do not define dotted parents. * convergence states include `system/convergence-rationale.md`. * generated learning docs and visual graphs show convergence metadata and dotted-line lineage. ## Pros and Cons of the Options ### Plain lineage only * Good, because metadata is minimal. * Bad, because recommended convergence checkpoints are implicit and easy to miss. * Bad, because parallel-track reasoning is harder for contributors. ### True multi-parent publish lineage * Good, because ancestry can mirror conceptual convergence. * Bad, because publish and compare tooling becomes significantly more complex. * Bad, because branch history maintenance cost increases. ### Single-parent lineage + convergence metadata * Good, because it balances tooling simplicity with explicit convergence semantics. * Good, because docs can visualize both publish lineage and conceptual convergence. * Neutral, because additional metadata/governance is required. ## More Information Related artifacts: * Catalog source of truth: `/catalog/state-catalog.json` * Convergence reference: `/docs/spec-kit/convergence-states` * Visual learning graph: `/docs/learning-paths` --- ## Project ADRs This is the active ADR set for TraderX SpecKit governance. - [ADR-001 Adopt GitHub Spec Kit](/docs/adr/001-adopt-github-speckit) - [ADR-002 Generated State Branching Strategy](/docs/adr/002-generated-state-branching-strategy) - [ADR-003 Intentional Legacy-Shaped Baseline](/docs/adr/003-baseline-intentionally-simplistic-legacy-shaped) - [ADR-004 Lightweight Baseline Infrastructure and Replaceability](/docs/adr/004-prefer-lightweight-default-infra-and-swappable-components) - [ADR-005 Use NATS for State 006 Messaging Replacement](/docs/adr/005-state-006-use-nats-for-messaging-replacement) - [ADR-006 Use PostgreSQL for State 005 Database Replacement](/docs/adr/006-state-005-use-postgres-for-database-replacement) - [ADR-007 Adopt LGTM + OpenTelemetry Stack for State 007 Baseline](/docs/adr/007-state-007-adopt-lgtm-observability-stack) - [ADR-008 Convergence State Model (C0-C3)](/docs/adr/008-convergence-state-model) --- ## From London Hackathon to Open-Source Baseline Published: **May 2023** TraderX did not start as a perfectly planned greenfield open-source project. The early shape came from practical prototyping work, including hackathon-style collaboration in London at BMO, where the goal was to turn an internal concept into a learning-oriented open-source base. By May 2023, the repository began to show real structure: - `Initial checkin` landed core service wiring (`#13`). - Java build and corporate build support were improved (`#15`). - Both Angular and React front-end variants were added (`#17`, `#19`). - Trade ticket and feed workflow behavior were refined (`#20`, `#21`). - Messaging and "corp-friendly" structure were reworked (`#22`). This period set a lasting theme for TraderX: keep the stack understandable, runnable, and polyglot enough to teach integration concerns without requiring enterprise-scale setup on day one. Even at this early stage, the long-term question was already visible: how do we preserve simplicity as the project grows? That question eventually shaped almost every architectural decision that followed. --- ## Open-Source Hardening and Governance Foundations Published: **September 2023** After the initial build-out, the next milestone was less visible to casual users but critical for long-term sustainability: open-source hardening. Key steps landed across late summer and early fall 2023: - CVE scanning added across project modules (`#30`). - FINOS blueprint alignment work applied (`#32`). - License scanning for Node assets added (`#33`). - Follow-up updates to satisfy scan gates (`#34`). The effect was immediate: TraderX moved from "interesting demo code" toward "maintainable open-source project with enforceable hygiene." These changes also introduced a tradeoff that never went away: - More controls improve trust and project health. - More controls also increase contributor and maintainer overhead. That tradeoff shaped later choices around automation, dependency policy, and eventually the shift to spec-first generation for multi-state maintenance. --- ## OSFF NYC Keynote Demo and Go-Live Momentum Published: **November 2023** Fall 2023 marked an important visibility moment for TraderX: the project was presented in New York at the Open Source in Finance Forum (OSFF), including keynote-demo context that helped position TraderX as a practical learning project for modern trading-system architecture. Around the same period, repository changes reflected go-live readiness: - Documentation and branding updates were prepared for public visibility (`#72`). - Ongoing dependency and security updates continued across languages. - The project identity became more intentional, including demo discoverability. Relevant event reference: - [Open Source in Finance Forum New York 2023](https://events.linuxfoundation.org/archive/2023/open-source-finance-forum-new-york/) This milestone mattered beyond publicity. It reinforced the need for TraderX to serve two audiences at once: - newcomers looking for a runnable, understandable system, - maintainers and contributors pushing for modernization and extensibility. Balancing those audiences remains central to the project's evolution. --- ## Docusaurus Portal and Project Storytelling Published: **March 2024** By early 2024, TraderX had accumulated meaningful technical depth. The next problem was discoverability. We needed better answers for first-time visitors: - What is this project? - How do I run it? - Which UI/runtime path should I start with? - Where are the architectural boundaries? Major changes in this period addressed exactly that: - Docusaurus was introduced and integrated (`#138` / docs-website merge). - Homepage and UX refinements improved clarity (`#129`). - README and demo-link quality improved in follow-up patches. This was the first strong step toward treating TraderX as a learning portal, not only a code repository. That direction later became the foundation for visual state mapping and spec-linked documentation, which are now central to the SpecKit-era workflow. --- ## Platform Paths on Kubernetes and Radius Published: **September 2024** September 2024 was a major platform-expansion window for TraderX. Several streams converged: - Initial conversion of main services for build/deploy progression (`#206`). - CI and image-scanning hardening (`#222`, `#227`). - Improved Kubernetes local developer experience (`#232`). - Radius deployment/modeling contributions landed (`#204`). This period demonstrated both potential and tension: - Potential: TraderX could support richer runtime paths and platform abstraction experiments. - Tension: each new path increased maintenance complexity and branching pressure. In hindsight, this was one of the strongest signals that we needed a better way to model parallel and sometimes competing paths without forcing all outcomes into one evolving code line. That signal directly influenced the later shift to state-based spec-first generation. --- ## TraderX + GitHub SpecKit Migration (Technical Retrospective) Published: **Sunday, March 29, 2026** For the last 2.5 years, TraderX has had one persistent tension: keep the project simple and approachable for learners, while receiving a steady flow of good ideas and PRs that naturally push toward more production-style complexity. The hardest part was never a shortage of contributions. The hard part was fitting those contributions into a structure that preserved the project's educational simplicity. ## The Core Challenge We needed to support all of the following: - A baseline that feels lightweight and accessible. - Advanced paths for platform and non-functional evolution. - A visual learning portal that maps state transitions clearly. - A sustainable maintenance model for a small maintainer group. In practice, we were already feeling the strain of dependency maintenance on one line of code. Maintaining many parallel long-lived branches with different architectural maturity levels was not realistic. ## The First Attempt: High Momentum, Wrong Foundation When I initially worked with an LLM on this direction, the early results looked great: - It migrated component-by-component toward spec-first flow. - It could run generated components alongside traditional components. - It advanced step-by-step to full-system parity. - It produced progress documentation, including Mermaid visuals on Docusaurus. From the outside, it looked like a breakthrough. Under the hood, it was not. The model had effectively invented its own flavor of "spec-driven development" while labeling it "speckit." It was inspired by SpecKit, but not aligned with **GitHub SpecKit** conventions and workflow. Even worse, some of the so-called generation scripts were effectively large shell scripts that dumped file contents into targets. That created a convincing facade, but not a trustworthy or maintainable generation mechanism. ## Reset and Migration to Real GitHub SpecKit At that point, I reset the direction and enforced stricter constraints: - Learn and follow official GitHub SpecKit patterns end-to-end. - Document a migration plan from fake speckit artifacts to canonical SpecKit structure. - Track retirement of legacy docs/scripts that no longer matched the target model. - Keep commits small and continuous in a dedicated migration branch. This process was not smooth. It required close oversight, repeated corrections, and explicit TODO planning to keep the agent from drifting into plausible-but-wrong shortcuts. ## What Changed in Practice After that correction phase, we reached a clean baseline where state-oriented specs became the source of truth and code became generated output. We first validated that the generated harness could reproduce existing behavior for the initial baseline state. Then the process became more interesting: we layered additional state transitions through spec-defined deltas and regenerated. ## Branch and State Strategy We chose to keep canonical spec authoring in the main renovation branch, and publish generated code snapshots as separate state branches: - `code/generated-state-001-baseline-uncontainerized-parity` - `code/generated-state-002-edge-proxy-uncontainerized` - `code/generated-state-004-containerized-compose-runtime` - `code/generated-state-005-postgres-database-replacement` - `code/generated-state-006-messaging-nats-replacement` - `code/generated-state-007-observability-lgtm-compose` - `code/generated-state-008-pricing-awareness-market-data` - `code/generated-state-009-order-management-matcher` - `code/generated-state-010-kubernetes-runtime` - `code/generated-state-011-tilt-kubernetes-dev-loop` - `code/generated-state-012-platform-convergence-c3` - `code/generated-state-013-radius-kubernetes-platform` This gave us a useful split: - Developers who just want code can read state branches directly. - Maintainers solve the harder meta-problem in the spec branch, then regenerate. As cross-cutting changes land (README improvements, learning graph updates, docs cleanup), we can regenerate all states quickly and keep them aligned. ## Why This Is Better This model makes contribution intake different: - It is trickier in one sense because contributions should target specs and state deltas, not ad hoc code drops. - It is much easier in another sense because intent, scope, and compatibility are explicit. For maintainers, the tradeoff is worth it. We gain speed, clarity, and reproducibility. For learners, we can preserve a simple starting point while still exposing richer paths without collapsing everything into one intimidating codebase. ## What Is Still Open We still need to define some transition boundaries more clearly, for example: - Where to shift from simple baseline runtime into mature containerized paths. - When to introduce heavier NFR tracks such as observability and service mesh. - How to layer functional expansions (for example advanced pricing engines) while keeping state coherence. - How to present this whole model so newcomers are not overwhelmed on first contact. One likely next move is a "pre-phase-0" explanation state that is intentionally simple and onboarding-focused. ## Where to Inspect the Work - Renovation branch: [feature/agentic-renovation](https://github.com/finos/traderX/tree/feature/agentic-renovation) - Commit history: [feature/agentic-renovation commits](https://github.com/finos/traderX/commits/feature/agentic-renovation) - Generated state-code branches: [search `code/generated-state-*`](https://github.com/finos/traderX/branches/all?query=code%2Fgenerated-state-) - Canonical state metadata and lineage: [state docs](/docs/spec-kit/state-docs) The commit trail and generated-state branches show how this evolved in real time, including course-corrections and cleanup decisions that were necessary to move from "looks like it works" to "this is a real and sustainable foundation." --- ## State 006: Swapping Socket.IO for NATS Published: **Tuesday, March 31, 2026** State `006-messaging-nats-replacement` is in place as the messaging architecture transition on top of `005-postgres-database-replacement`. For a long time, TraderX used a deliberately simple messaging layer that was easy to run but increasingly brittle as we pushed toward more realistic multi-service evolution. The old Socket.IO-centric setup worked, but it forced us into awkward conventions for server-to-server eventing and created fragility when we wanted clearer protocol behavior. ## What Changed We replaced the `trade-feed` Socket.IO runtime role with a `nats-broker` component and migrated eventing to NATS subjects. This change keeps baseline user flows intact, but modernizes the transport layer: - backend pub/sub now uses a broker pattern designed for service-to-service communication, - browser streaming now uses NATS WebSocket routing through ingress (`/nats-ws`), - runtime remains approachable in Docker Compose for this branch. Code and specs: - State spec pack: [/specs/messaging-nats-replacement](/specs/messaging-nats-replacement) - State learning guide: [/docs/learning/state-006-messaging-nats-replacement](/docs/learning/state-006-messaging-nats-replacement) - Generated code branch: [code/generated-state-006-messaging-nats-replacement](https://github.com/finos/traderX/tree/code/generated-state-006-messaging-nats-replacement) - Compare against parent state `004`: [generated-state 005...005](https://github.com/finos/traderX/compare/code%2Fgenerated-state-005-postgres-database-replacement...code%2Fgenerated-state-006-messaging-nats-replacement) ## Why NATS We selected NATS because it is a better fit for this stage of TraderX evolution: - lightweight and simple to operate locally, - strong multi-language clients (Java, .NET, Node.js, Python), - clean subject model and wildcard subscriptions, - request/reply support with room for future durability extension, - browser support through websocket-compatible clients. The formal state-scoped rationale is captured in: - [ADR-005 Use NATS for State 006 Messaging Replacement](/docs/adr/005-state-006-use-nats-for-messaging-replacement) ## Why This Was Easier With SpecKit This change is a clear example of why we moved to SpecKit-first maintenance: - requirements and deltas were defined once in the feature pack, - code generation and runtime scripts were updated from that intent, - docs and learning guides were regenerated with lineage and compare links, - tests and conformance checks were updated as part of the same state delta. Instead of manually curating long-lived code branches, we changed the system of record and regenerated state outputs. ## Track Implications State `005` sets up the messaging baseline that later states inherit: - `006` adds observability, - `007` layers pricing streams on top of NATS, - `008` layers order matching on top of pricing and NATS. This keeps transitions explicit, diffs reviewable, and runnable outputs available at each step. --- ## March 31, 2026: State 005 - Replacing H2 with PostgreSQL # State 005: Replacing H2 with PostgreSQL State `005-postgres-database-replacement` is the first architecture-focused database transition after `004-containerized-compose-runtime`. This change keeps the same baseline functional behavior, APIs, and runtime entry model from state `004`, while replacing the underlying database runtime from H2 to PostgreSQL. ## Why This State The project needed a more production-like SQL engine without adding heavy setup overhead. PostgreSQL gives us that while keeping local Docker Compose usage straightforward. This validates the SpecKit-first approach: define a focused architecture delta, generate the state, run smoke checks, and publish a clean generated code branch. ## What Changed - Added a PostgreSQL container runtime with deterministic init SQL. - Updated `account-service`, `position-service`, and `trade-processor` to use PostgreSQL datasource/driver settings. - Preserved baseline flow behavior (`F1` to `F4`) and existing REST/event contracts. - Added state-specific runtime scripts and smoke tests for PostgreSQL readiness + baseline compatibility. ## Spec + Decision Links - State spec pack: [/specs/postgres-database-replacement](/specs/postgres-database-replacement) - ADR: [/docs/adr/006-state-005-use-postgres-for-database-replacement](/docs/adr/006-state-005-use-postgres-for-database-replacement) - Learning guide: [/docs/learning/state-005-postgres-database-replacement](/docs/learning/state-005-postgres-database-replacement) - Generated code branch: [code/generated-state-005-postgres-database-replacement](https://github.com/finos/traderX/tree/code/generated-state-005-postgres-database-replacement) - Compare vs parent (`004`): [code/generated-state-004-containerized-compose-runtime...code/generated-state-005-postgres-database-replacement](https://github.com/finos/traderX/compare/code%2Fgenerated-state-004-containerized-compose-runtime...code%2Fgenerated-state-005-postgres-database-replacement) ## Why This Matters For The Learning Graph State `005` demonstrates that architecture substitutions can be done as isolated, reversible state transitions: - branch from a stable parent state, - apply one focused replacement axis, - keep the rest of the system stable, - publish as an independent generated branch for developers to inspect and run. --- ## March 31, 2026: State 008 - Pricing Awareness as a Functional Track # State 008: Pricing Awareness as a Functional Track State `008-pricing-awareness-market-data` extends `007-observability-lgtm-compose` and marks the first major functional expansion after the NATS messaging upgrade. This state adds pricing awareness to trade execution, persistence, and portfolio presentation, while preserving the existing end-to-end trading flow. ## Why This State The system previously handled quantities and securities but had no execution pricing context in persisted trades or aggregated positions. State `008` introduces that missing functional layer and keeps it realtime via NATS topics. ## What Changed - Added a new `price-publisher` component that emits random-walk quotes per ticker on `pricing.`. - Stamped execution price on trade submission in `trade-service`. - Persisted price on trades and volume-weighted average cost basis on positions in `trade-processor`. - Extended schema to include `TRADES.Price` and `POSITIONS.AverageCostBasis`. - Updated Angular blotters to show trade price, relative execution time, position value, cost basis, and net P&L totals. ## Verification and Regression Coverage - State-specific smoke suite: `./scripts/test-state-008-pricing-awareness-market-data.sh` - Realtime checks validated: - account trade stream - account position stream - pricing stream (`pricing.IBM`) - Persistence checks validated: - `trade.price` is populated - `position.averageCostBasis` is populated - Baseline overlay smoke tests also pass under the state runtime to ensure no regressions in prior behavior. ## Spec + Code Links - State spec pack: [/specs/pricing-awareness-market-data](/specs/pricing-awareness-market-data) - Learning guide: [/docs/learning/state-008-pricing-awareness-market-data](/docs/learning/state-008-pricing-awareness-market-data) - Generated code branch: [code/generated-state-008-pricing-awareness-market-data](https://github.com/finos/traderX/tree/code/generated-state-008-pricing-awareness-market-data) - Compare vs parent (`007`): [code/generated-state-007-observability-lgtm-compose...code/generated-state-008-pricing-awareness-market-data](https://github.com/finos/traderX/compare/code%2Fgenerated-state-007-observability-lgtm-compose...code%2Fgenerated-state-008-pricing-awareness-market-data) --- ## April 6, 2026: Convergence Milestone Releases (C0-C3) # Convergence Milestone Releases (C0-C3) We introduced explicit convergence milestones so developers have clear, recommended jump-off points for new state work. Convergence states are not just another node in the graph. They are curated checkpoints where multiple concerns have been intentionally composed and validated. ## Why We Added Convergence States - Keep transitional states focused and easy to reason about. - Provide stable branch points for new enhancements. - Avoid repeatedly re-solving baseline architecture choices in each new state. ## Current Convergence Levels - `C0`: `004-containerized-compose-runtime` - `C1`: `007-observability-lgtm-compose` - `C2`: `009-order-management-matcher` - `C3`: `012-platform-convergence-c3` ## Dotted-Line Inheritance Convergence states can declare dotted-line lineage for documentation and learning context, while keeping a single publish parent for branch ancestry. That lets us carry forward improvements into the best “next base” without creating ambiguous multi-parent publish history. ## Convergence View ```mermaid flowchart LR C0["C0 / 003"] --> C1["C1 / 006"] C1 --> C2["C2 / 008"] C2 --> C3["C3 / 011"] ``` ## Contribution Guidance - For most new features, start from the nearest convergence state that already contains needed prerequisites. - If a convergence state is changed, document explicit rationale. - Use dotted-line lineage only when it clarifies learning inheritance, not branch ancestry. ## Decision + Navigation Links - ADR: [ADR-008 Convergence State Model](/docs/adr/008-convergence-state-model) - Visual learning graphs: [/docs/learning-paths](/docs/learning-paths) - State catalog: [/docs/spec-kit/state-docs](/docs/spec-kit/state-docs) --- ## April 6, 2026: State 007 - Observability Baseline with LGTM # State 007: Observability Baseline with LGTM State `007-observability-lgtm-compose` adds a reusable observability baseline on top of the architecture chain (`005` PostgreSQL + `006` NATS). The goal was to make diagnostics first-class without changing business behavior. We wanted consistent local visibility across logs, metrics, traces, and service health for every generated runtime that inherits this state. ## What Was Added - LGTM stack in compose runtime: Grafana, Loki, Tempo, Prometheus. - OpenTelemetry collector wiring for cross-service telemetry flow. - Baseline probes and health checks for observability services. - Default dashboards and smoke checks so observability is verifiable, not aspirational. ## Why This Matters - Functional states can now be developed with immediate telemetry feedback. - Regressions are easier to detect and isolate across services. - Later states (`008`, `009`, and convergence states) inherit the same baseline by default. ## Decision Record The state-scoped rationale is captured in: - [ADR-007 Adopt LGTM + OpenTelemetry Stack for State 007 Baseline](/docs/adr/007-state-007-adopt-lgtm-observability-stack) ## Spec + Code Links - State spec pack: [/specs/observability-lgtm-compose](/specs/observability-lgtm-compose) - Learning guide: [/docs/learning/state-007-observability-lgtm-compose](/docs/learning/state-007-observability-lgtm-compose) - Generated code branch: [code/generated-state-007-observability-lgtm-compose](https://github.com/finos/traderX/tree/code/generated-state-007-observability-lgtm-compose) - Compare vs parent (`005`): [code/generated-state-006-messaging-nats-replacement...code/generated-state-007-observability-lgtm-compose](https://github.com/finos/traderX/compare/code%2Fgenerated-state-006-messaging-nats-replacement...code%2Fgenerated-state-007-observability-lgtm-compose) --- ## April 6, 2026: State 009 - Rudimentary Order Manager and Matcher # State 009: Rudimentary Order Manager and Matcher State `009-order-management-matcher` introduces a deliberately simple order-management model to demonstrate architecture and flow shape, not market microstructure correctness. This state adds pending orders, cancel/force-fill workflows, and a basic matching loop on top of pricing + messaging foundations. ## What This State Adds - User order ticket and account-scoped order blotter. - Admin screen for global order inspection, cancel, and force-fill actions. - Basic matching engine behavior: - evaluate open orders against streaming prices, - auto-fill rules for in-the-money orders, - emit fills as trade events into the existing trade/position pipeline. - Order-focused observability metrics and dashboard coverage. ## Messaging Architecture (State 009) ```mermaid flowchart LR UI["Web UI"] -->|Market Trade| TS["trade-service"] UI -->|Limit/Stop Order| OM["order-matcher / order engine"] UI <-->|Account Streams| NATS["NATS Broker"] PP["price-publisher"] -->|pricing.| NATS NATS -->|pricing.| OM TS -->|TradeOrder| NATS OM -->|Filled TradeOrder| NATS NATS -->|TradeOrder| TP["trade-processor"] TP -->|Persist Trade + Position| DB["PostgreSQL"] TP -->|account..trades / positions| NATS NATS -->|account..trades / positions| UI DB --> PS["position-service"] PS --> UI ``` ## Why “Rudimentary” Is Intentional - The matching logic is intentionally simple for learning and extension. - The state demonstrates where richer matching policies, risk checks, and lifecycle controls would plug in. - Teams can branch from this state to build more realistic order workflows without redesigning the system spine. ## Spec + Code Links - State spec pack: [/specs/order-management-matcher](/specs/order-management-matcher) - Learning guide: [/docs/learning/state-009-order-management-matcher](/docs/learning/state-009-order-management-matcher) - Generated code branch: [code/generated-state-009-order-management-matcher](https://github.com/finos/traderX/tree/code/generated-state-009-order-management-matcher) - Compare vs parent (`008`): [code/generated-state-008-pricing-awareness-market-data...code/generated-state-009-order-management-matcher](https://github.com/finos/traderX/compare/code%2Fgenerated-state-008-pricing-awareness-market-data...code%2Fgenerated-state-009-order-management-matcher) --- ## April 10, 2026: Custom Overlay Model for Environment-Specific TraderX # Custom Overlay Model for Environment-Specific TraderX TraderX now documents a clear customization path for teams that need environment-specific runtime, package, messaging, or governance constraints while keeping upstream TraderX clean. ## Why Overlay Instead of Fork - Keep upstream state packs and generation logic as the public source of truth. - Keep environment-specific changes in a separate overlay repository. - Regenerate and republish snapshots in a controlled, reproducible way. This avoids long-term drift and keeps contributions focused on upstream specs when a change should benefit everyone. ## Recommended Structure The canonical starter is: - `examples/custom-overlay-template/` The documented target model is: - `upstream/traderX` as a pinned submodule - overlay-owned `profiles/`, `states/`, `transforms/`, and `runtime/` - overlay-owned `pipeline/generate-state.sh` and `pipeline/publish-state-branch.sh` ## Suggested Example Custom Rule A practical example for constrained environments: - runtime policy disallows default local database images - each generated state uses a managed database endpoint pattern - connection/auth material is injected through environment setup scripts This pattern is reflected in template examples so teams can adapt the mechanism without copying environment-specific values into upstream docs. ## Key Operational Rules - Treat generated outputs as reproducible artifacts, not hand-edited code. - Keep transform scripts idempotent. - Apply transforms after layout copy steps that overwrite target directories. - Enforce one-commit-per-generated-state-branch publishing in overlay repos as well. ## Where To Start - Getting started: [/docs/spec-kit/getting-started-with-traderx](/docs/spec-kit/getting-started-with-traderx) - Customization hub: [/docs/spec-kit/customizing-traderx](/docs/spec-kit/customizing-traderx) - Overlay architecture contract: [/docs/spec-kit/custom-overlay-architecture](/docs/spec-kit/custom-overlay-architecture) - Environment and integration guidance: [/docs/spec-kit/custom-environments-guide](/docs/spec-kit/custom-environments-guide) --- ## April 10, 2026: State 003 - Agentic Harness Foundation # State 003 - Agentic Harness Foundation State `003-agentic-harness-foundation` was added as an early, non-domain feature state so generated codebases are easier to work with using agentic tooling before larger architecture and runtime transitions. ## What This State Adds - baseline agent-oriented repository guidance in generated outputs - normalized non-code metadata scaffolding for codebase understanding - contribution guidance that keeps generated repos focused on local experimentation while directing upstream enhancements back to spec/source states ## Why It Sits Before C0 Placing this state before the first convergence baseline means all later generated states inherit a consistent foundation for: - onboarding with coding agents - predictable repository documentation - contribution path clarity This keeps downstream generated branches more usable without mixing these concerns into later feature-specific states. ## Generated Branch - [code/generated-state-003-agentic-harness-foundation](https://github.com/finos/traderX/tree/code/generated-state-003-agentic-harness-foundation) - Compare vs parent (`002`): [code/generated-state-002-edge-proxy-uncontainerized...code/generated-state-003-agentic-harness-foundation](https://github.com/finos/traderX/compare/code%2Fgenerated-state-002-edge-proxy-uncontainerized...code%2Fgenerated-state-003-agentic-harness-foundation) ## Related Docs - Learning guide: [/docs/learning/state-003-agentic-harness-foundation](/docs/learning/state-003-agentic-harness-foundation) - State catalog overview: [/docs/spec-kit/state-docs](/docs/spec-kit/state-docs) - Generation model: [/docs/spec-kit/generated-state-branches](/docs/spec-kit/generated-state-branches) --- ## April 18, 2026: Integrating FDC3 in TraderX (What Worked, What Was Flaky, What We Learned) # Integrating FDC3 in TraderX: What Worked, What Was Flaky, What We Learned State `014-fdc3-intent-interoperability` was our first end-to-end pass at wiring FDC3 interaction patterns into a live TraderX environment. The goal was straightforward: prove that intent-style interoperability is practical in a realistic demo stack, not just in isolated snippets. ## Why This State Matters This state shows that a single TraderX deployment can coordinate: - pricing and market-data context, - order-management workflows, - chart and news views on separate tabs, - and FDC3-based action routing between apps. In short, it demonstrates the point of FDC3 in an environment people can actually run and explore. For FDC3 background, see [fdc3.finos.org](https://fdc3.finos.org). ## What We Built To make the integration explicit and easy to understand, we introduced a separate Sail app that dispatches TraderX-related FDC3 events. That gave us a clear place to: - map UI actions to FDC3 payloads, - push context on channels, - and route action-specific behavior without entangling every component. It is not the final elegance target, but it is a practical and inspectable starting point. ## Where We Saw Flakiness During integration and demo validation, we ran into several reliability pain points: - High-volume heartbeat and context-sync logs obscured signal during debugging. - `broadcast`-driven button flows were less deterministic than `raiseIntent` flows in some paths. - Tab/channel context timing was easy to get wrong when multiple widgets were syncing state. - TradingView interactions exposed symbol-normalization mismatches across components. One concrete pattern: raising intent from the trade list path behaved consistently, while some button-triggered paths required tighter alignment of context/action payloads before behavior was reliable. ## Symbology Lessons (and Why CDM Helps) Symbol normalization turned out to be one of the biggest practical interoperability issues. Even with a common ticker value (`AAPL`), differences in source assumptions and widget behavior can create subtle drift in how symbols are interpreted or propagated. This is exactly where broader semantic normalization helps. A stronger shared model such as CDM should reduce these integration seams over time. For CDM background, see [cdm.finos.org](https://cdm.finos.org). ## Why This Is Still a Strong Result Despite the rough edges, this state proves real value: - We can run a self-contained demo with FDC3 behavior that is visible and testable. - We can show intent/context propagation across independent app surfaces. - We now have a concrete baseline for hardening, not a theoretical design. There is much more we can make it do, but this already demonstrates the core interoperability point with live, inspectable software. ## Spec + Code Links - State spec pack: [/specs/fdc3-intent-interoperability](/specs/fdc3-intent-interoperability) - Learning guide: [/docs/learning/state-014-fdc3-intent-interoperability](/docs/learning/state-014-fdc3-intent-interoperability) - Generated code branch: [code/generated-state-014-fdc3-intent-interoperability](https://github.com/finos/traderX/tree/code/generated-state-014-fdc3-intent-interoperability) - Compare vs parent (`012`): [code/generated-state-012-platform-convergence-c3...code/generated-state-014-fdc3-intent-interoperability](https://github.com/finos/traderX/compare/code%2Fgenerated-state-012-platform-convergence-c3...code%2Fgenerated-state-014-fdc3-intent-interoperability) --- ## April 20, 2026: Why Our TraderX + Sail Migration to @morgan-stanley/fdc3-web Failed (and What We Learned) # Why Our TraderX + Sail Migration to `@morgan-stanley/fdc3-web` Failed (and What We Learned) This post documents a real migration attempt in state `014-fdc3-intent-interoperability`: replacing our working FDC3 integration path with `@morgan-stanley/fdc3-web` across TraderX and Sail demo apps. It did not meet our success criteria for demo reliability, and we reverted to the prior path. ## Process Transparency An explicit admission up front: I drove this migration based on specs/docs and functional expectations, but I did not personally hand-write the FDC3 integration code. I relied on LLM-assisted implementation and focused my effort on functional testing, log analysis, and iterative feedback to converge on working behavior. That accelerated iteration, but it also exposed where generated code can be insufficient without deep host/runtime lifecycle understanding. ## Success Criteria We Used We considered the migration successful only if all of the following held consistently: 1. TraderX row selection broadcasts `fdc3.instrument`. 2. TradingView widgets react in the same channel/context. 3. Polygon news reacts in the same channel/context. 4. TraderX Intent Launcher resolves current channel + ticker and can trigger TraderX ticket flows. 5. Sail app discovery shows TraderX as an existing app where expected. ## What We Observed Some interoperability signals looked healthy: - `getCurrentChannelRequest/Response` worked. - `getCurrentContextRequest/Response` worked. - Channel IDs were returned correctly. But end-to-end behavior was unstable due to initialization failures and timing races: - `IdentityValidationHandler: Identity validation timed out` - `The Desktop Agent didn't respond to ID validation within 5 seconds` - `Discovery results: [{ status: "rejected", reason: "ErrorOnConnect" }]` - `TradingView failed to initialize` - `Polygon failed to initialize` - listener registration timeouts followed by "best effort" paths In other words: transport-level activity existed, but app-level readiness was not durable. ## Root Cause Categories ### 1) Lifecycle mismatch between host startup and strict validation windows `@morgan-stanley/fdc3-web` behavior in our flow was stricter around identity validation/connection timing than our Sail startup sequence could reliably satisfy on first attempt. When first connect failed (`ErrorOnConnect`), app widgets often remained disconnected. ### 2) One-shot initialization in widget code TradingView and Polygon widgets initially performed a single connection attempt and then logged failure. Without reconnect/backoff, transient startup lag became persistent app failure. ### 3) Identity URL instability during experiments At one point, URL query seeding/cache-busting was introduced to avoid stale session collisions. That caused handshake errors in Sail's proxy path (`Illegal handshake attempt` style behavior), indicating identity contract mismatch for this host. ### 4) Channel topology confusion in seeded demo state A seeded layout had multiple TraderX instances on different tabs/channels. Broadcasts from one TraderX were valid, but consumers on another channel did not react, creating a false appearance of protocol failure. ## Why This Is Not Simply a "Spec Compliance" Story FDC3 standardizes interop semantics (`broadcast`, context listeners, intents, channels), but not every host/discovery/bootstrap implementation detail: - discovery handshake timing, - identity validation policy, - reconnect strategy, - listener re-registration behavior, - iframe/proxy boot choreography. Two libraries can be FDC3-compliant and still behave very differently under real startup conditions. ## What We Changed to Restore Reliability After reverting to the established Sail-aligned path, we hardened the runtime: 1. Stable identity URL handling for TraderX. 2. Seeded state updates to avoid stale session/panel IDs. 3. Demo layout normalization so TraderX + TradingView + Polygon boot on the same tab/channel. 4. Reconnect/backoff logic in TradingView/Polygon widget initialization instead of fail-fast one-shot init. With these changes, the functional criteria were met again in repeated local validation runs. ## Practical Guidance for Teams Attempting Similar Migrations Treat an FDC3 library swap as a platform integration change, not a package replacement. Validate explicitly: 1. cold-start handshake behavior, 2. identity validation timeout behavior, 3. reconnect/listener re-registration after transient failures, 4. seeded layout/channel defaults, 5. stale local-storage state interactions. ## Final Takeaway This migration attempt was not successful in its target form, but it produced useful engineering knowledge: - spec-compliant APIs are necessary but not sufficient for runtime reliability, - host lifecycle assumptions matter as much as API surface, - LLM-assisted implementation can accelerate delivery but still requires rigorous integration testing and operator-driven diagnostics. For now, we prioritize deterministic demo behavior on the proven integration path and may revisit `@morgan-stanley/fdc3-web` later behind a host-aware adapter. ## References - State spec pack: [/specs/fdc3-intent-interoperability](/specs/fdc3-intent-interoperability) - Learning guide: [/docs/learning/state-014-fdc3-intent-interoperability](/docs/learning/state-014-fdc3-intent-interoperability) - FDC3 standard background: [fdc3.finos.org](https://fdc3.finos.org) --- ## May 20, 2026: We’re Finally Live with TraderX Demos # We’re Finally Live with TraderX Demos After a lot of design, implementation, debugging, and deployment work, TraderX now has live FINOS-hosted demo environments: - Original Demo (state `004`): [https://demo.traderx.finos.org](https://demo.traderx.finos.org) - Advanced Demo (state `009`): [https://demo-advanced.traderx.finos.org](https://demo-advanced.traderx.finos.org) Huge thanks to the FINOS team and everyone who helped us get this over the line. Tracking issue: [#334](https://github.com/finos/traderX/issues/334) We are now in position for final merge, with live environments proving the current generated-state path and deployment model. ## What We Did Along The Way - Moved to a SpecKit-first, state-driven architecture. - Standardized generated-state branch publishing and lineage. - Thanks to this approach, we rapidly built out key states in the TraderX [visual learning graph](/docs/learning-paths). - Added state-aware reverse-proxy snippet generation for websocket transport routes. - Tightened CORS/domain propagation so deployed environments use correct FQDN-based origins. - Removed dev-runtime leakage (for example Vite hot-reload endpoints) from deployed container images. - Added canonical live environment mapping in project source-of-truth docs/catalog. ## What’s Next - Harden deployment automation further across environments. - Move generated-state branch publication deeper into CI-driven automation. - Continue improving convergence-state deployment contracts, including Kubernetes-profile deployment bundles. - Expand smoke/regression coverage so future state changes stay safe by default. TraderX is in a strong position for the next phase: reliable generated-state delivery, better operational repeatability, and faster evolution on top of a clearer architecture model. --- ## Blog # TraderX Engineering Blog This blog captures the broader TraderX story over time: how the project started, how it was prepared for open-source growth, how platform and documentation milestones landed, and how we eventually moved to a SpecKit-first state model. ## Articles - [May 20, 2026: We’re Finally Live with TraderX Demos](/docs/blog/2026-05-20-were-finally-live-demos) - [April 20, 2026: Why Our TraderX + Sail Migration to @morgan-stanley/fdc3-web Failed (and What We Learned)](/docs/blog/2026-04-20-fdc3-web-migration-retrospective) - [April 18, 2026: Integrating FDC3 in TraderX (What Worked, What Was Flaky, What We Learned)](/docs/blog/2026-04-18-fdc3-integration-lessons) - [April 10, 2026: Custom Overlay Model for Environment-Specific TraderX](/docs/blog/2026-04-10-custom-overlay-model-and-example) - [April 10, 2026: State 003 - Agentic Harness Foundation](/docs/blog/2026-04-10-state-003-agentic-harness-foundation) - [April 6, 2026: Convergence Milestone Releases (C0-C3)](/docs/blog/2026-04-06-convergence-milestone-releases) - [April 6, 2026: State 009 - Rudimentary Order Manager and Matcher](/docs/blog/2026-04-06-state-009-order-management-matcher) - [April 6, 2026: State 007 - Observability Baseline with LGTM](/docs/blog/2026-04-06-state-007-observability-baseline) - [March 31, 2026: State 008 - Pricing Awareness as a Functional Track](/docs/blog/2026-03-31-state-011-pricing-awareness-functional-track) - [March 31, 2026: State 005 - Replacing H2 with PostgreSQL](/docs/blog/2026-03-31-state-010-postgres-architecture-track) - [March 31, 2026: State 006 - Swapping Socket.IO for NATS](/docs/blog/2026-03-31-state-007-nats-architecture-track) - [March 29, 2026: TraderX + GitHub SpecKit Migration (Technical Retrospective)](/docs/blog/2026-03-29-traderx-speckit-migration) - [September 2024: Platform Paths on Kubernetes and Radius](/docs/blog/2024-09-platform-paths-kubernetes-radius) - [March 2024: Docusaurus Portal and Project Storytelling](/docs/blog/2024-03-docs-portal-and-project-storytelling) - [November 2023: OSFF NYC Keynote Demo and Go-Live Momentum](/docs/blog/2023-11-osff-nyc-keynote-and-go-live) - [September 2023: Open-Source Hardening and Governance Foundations](/docs/blog/2023-09-open-source-hardening-and-governance) - [May 2023: From London Hackathon to Open-Source Baseline](/docs/blog/2023-05-from-hackathon-to-open-source-baseline) --- ## TraderX Portal # TraderX Documentation Portal TraderX is a SpecKit-first reference trading system. ## Start Here 1. Read the project orientation: - [Getting Started](/docs/spec-kit/getting-started-with-traderx) 2. Generate and run the baseline from requirements: - [Spec Kit Generation Guide](/docs/spec-kit/spec-kit-generation-guide) 3. Understand the end-to-end workflow and quality gates: - [GitHub Spec Kit Workflow](/docs/spec-kit/spec-kit-workflow) ## Browse Canonical Sources - [Feature Specs Catalog](/specs) - [Spec Kit Constitution and Templates](/specify/memory/constitution) - [API Explorer (Runtime)](/docs/spec-kit/api-explorer) ## Where Is All The Code? All TraderX state code lives in this same repository. The canonical source is specs in `main`, and runnable code snapshots are published as generated-state branches and tags. - [Generated State Branches Guide](/docs/spec-kit/generated-state-branches) - [Generated-State Branches on GitHub](https://github.com/finos/traderX/branches/all?query=code%2Fgenerated-state-) - [Generated-State Tags on GitHub](https://github.com/finos/traderX/tags?query=generated%2F) - [State Catalog (`catalog/state-catalog.json`)](https://github.com/finos/traderX/blob/main/catalog/state-catalog.json) ## Live Demos - [TraderX Demo (State 004)](https://demo.traderx.finos.org) - [TraderX Advanced Demo (State 009)](https://demo-advanced.traderx.finos.org) - [Live Environments Registry](/docs/spec-kit/live-environments) ## Architecture Decisions - [Project ADRs (Architecture Decision Records)](/docs/adr) - ADRs follow the [MADR format/specification](https://github.com/adr/madr) ## Learning Paths and Future States - [Learning Guides](/docs/learning) - [Learning Paths](/docs/learning-paths) - [SpecKit Strategy for State Transitions](/docs/spec-kit/spec-kit-learning-path-strategy) --- ## Learning Guides This section is the developer-focused learning layer for generated TraderX code states. Canonical requirements, contracts, and architecture remain in `specs/**`. These guides explain how to read each generated code snapshot, compare it to previous states, and understand the code delta in plain English. For the visual progression map, see [Visual Learning Paths](/docs/learning-paths). ## State Guide Catalog | State | Learning Guide | Spec Pack | Generated Code Branch | Convergence | Dotted Parents | Previous State(s) | Compare To Previous | ADR | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `001-baseline-uncontainerized-parity` | [link](/docs/learning/state-001-baseline-uncontainerized-parity) | [link](/specs/baseline-uncontainerized-parity) | [code/generated-state-001-baseline-uncontainerized-parity](https://github.com/finos/traderX/tree/code/generated-state-001-baseline-uncontainerized-parity) | `none` | `none` | `none` | `n/a` | `n/a` | | `002-edge-proxy-uncontainerized` | [link](/docs/learning/state-002-edge-proxy-uncontainerized) | [link](/specs/edge-proxy-uncontainerized) | [code/generated-state-002-edge-proxy-uncontainerized](https://github.com/finos/traderX/tree/code/generated-state-002-edge-proxy-uncontainerized) | `none` | `none` | [001-baseline-uncontainerized-parity](/docs/learning/state-001-baseline-uncontainerized-parity) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-001-baseline-uncontainerized-parity...code%2Fgenerated-state-002-edge-proxy-uncontainerized) | `n/a` | | `003-agentic-harness-foundation` | [link](/docs/learning/state-003-agentic-harness-foundation) | [link](/specs/agentic-harness-foundation) | [code/generated-state-003-agentic-harness-foundation](https://github.com/finos/traderX/tree/code/generated-state-003-agentic-harness-foundation) | `none` | `none` | [002-edge-proxy-uncontainerized](/docs/learning/state-002-edge-proxy-uncontainerized) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-002-edge-proxy-uncontainerized...code%2Fgenerated-state-003-agentic-harness-foundation) | `n/a` | | `004-containerized-compose-runtime` | [link](/docs/learning/state-004-containerized-compose-runtime) | [link](/specs/containerized-compose-runtime) | [code/generated-state-004-containerized-compose-runtime](https://github.com/finos/traderX/tree/code/generated-state-004-containerized-compose-runtime) | `C0` | `none` | [003-agentic-harness-foundation](/docs/learning/state-003-agentic-harness-foundation) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-003-agentic-harness-foundation...code%2Fgenerated-state-004-containerized-compose-runtime) | `n/a` | | `005-postgres-database-replacement` | [link](/docs/learning/state-005-postgres-database-replacement) | [link](/specs/postgres-database-replacement) | [code/generated-state-005-postgres-database-replacement](https://github.com/finos/traderX/tree/code/generated-state-005-postgres-database-replacement) | `none` | `none` | [004-containerized-compose-runtime](/docs/learning/state-004-containerized-compose-runtime) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-004-containerized-compose-runtime...code%2Fgenerated-state-005-postgres-database-replacement) | [link](/docs/adr/006-state-005-use-postgres-for-database-replacement) | | `006-messaging-nats-replacement` | [link](/docs/learning/state-006-messaging-nats-replacement) | [link](/specs/messaging-nats-replacement) | [code/generated-state-006-messaging-nats-replacement](https://github.com/finos/traderX/tree/code/generated-state-006-messaging-nats-replacement) | `none` | `none` | [005-postgres-database-replacement](/docs/learning/state-005-postgres-database-replacement) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-005-postgres-database-replacement...code%2Fgenerated-state-006-messaging-nats-replacement) | [link](/docs/adr/005-state-006-use-nats-for-messaging-replacement) | | `007-observability-lgtm-compose` | [link](/docs/learning/state-007-observability-lgtm-compose) | [link](/specs/observability-lgtm-compose) | [code/generated-state-007-observability-lgtm-compose](https://github.com/finos/traderX/tree/code/generated-state-007-observability-lgtm-compose) | `C1` | `none` | [006-messaging-nats-replacement](/docs/learning/state-006-messaging-nats-replacement) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-006-messaging-nats-replacement...code%2Fgenerated-state-007-observability-lgtm-compose) | [link](/docs/adr/007-state-007-adopt-lgtm-observability-stack) | | `008-pricing-awareness-market-data` | [link](/docs/learning/state-008-pricing-awareness-market-data) | [link](/specs/pricing-awareness-market-data) | [code/generated-state-008-pricing-awareness-market-data](https://github.com/finos/traderX/tree/code/generated-state-008-pricing-awareness-market-data) | `none` | `none` | [007-observability-lgtm-compose](/docs/learning/state-007-observability-lgtm-compose) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-007-observability-lgtm-compose...code%2Fgenerated-state-008-pricing-awareness-market-data) | `n/a` | | `009-order-management-matcher` | [link](/docs/learning/state-009-order-management-matcher) | [link](/specs/order-management-matcher) | [code/generated-state-009-order-management-matcher](https://github.com/finos/traderX/tree/code/generated-state-009-order-management-matcher) | `C2` | `none` | [008-pricing-awareness-market-data](/docs/learning/state-008-pricing-awareness-market-data) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-008-pricing-awareness-market-data...code%2Fgenerated-state-009-order-management-matcher) | `n/a` | | `010-kubernetes-runtime` | [link](/docs/learning/state-010-kubernetes-runtime) | [link](/specs/kubernetes-runtime) | [code/generated-state-010-kubernetes-runtime](https://github.com/finos/traderX/tree/code/generated-state-010-kubernetes-runtime) | `none` | `none` | [009-order-management-matcher](/docs/learning/state-009-order-management-matcher) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-009-order-management-matcher...code%2Fgenerated-state-010-kubernetes-runtime) | `n/a` | | `011-tilt-kubernetes-dev-loop` | [link](/docs/learning/state-011-tilt-kubernetes-dev-loop) | [link](/specs/tilt-kubernetes-dev-loop) | [code/generated-state-011-tilt-kubernetes-dev-loop](https://github.com/finos/traderX/tree/code/generated-state-011-tilt-kubernetes-dev-loop) | `none` | `none` | [010-kubernetes-runtime](/docs/learning/state-010-kubernetes-runtime) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-010-kubernetes-runtime...code%2Fgenerated-state-011-tilt-kubernetes-dev-loop) | `n/a` | | `012-platform-convergence-c3` | [link](/docs/learning/state-012-platform-convergence-c3) | [link](/specs/platform-convergence-c3) | [code/generated-state-012-platform-convergence-c3](https://github.com/finos/traderX/tree/code/generated-state-012-platform-convergence-c3) | `C3` | [009-order-management-matcher](/docs/learning/state-009-order-management-matcher) | [011-tilt-kubernetes-dev-loop](/docs/learning/state-011-tilt-kubernetes-dev-loop) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-011-tilt-kubernetes-dev-loop...code%2Fgenerated-state-012-platform-convergence-c3) | [link](/docs/adr/008-convergence-state-model) | | `013-radius-kubernetes-platform` | [link](/docs/learning/state-013-radius-kubernetes-platform) | [link](/specs/radius-kubernetes-platform) | [code/generated-state-013-radius-kubernetes-platform](https://github.com/finos/traderX/tree/code/generated-state-013-radius-kubernetes-platform) | `none` | `none` | [012-platform-convergence-c3](/docs/learning/state-012-platform-convergence-c3) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-012-platform-convergence-c3...code%2Fgenerated-state-013-radius-kubernetes-platform) | `n/a` | | `014-fdc3-intent-interoperability` | [link](/docs/learning/state-014-fdc3-intent-interoperability) | [link](/specs/fdc3-intent-interoperability) | [code/generated-state-014-fdc3-intent-interoperability](https://github.com/finos/traderX/tree/code/generated-state-014-fdc3-intent-interoperability) | `none` | `none` | [012-platform-convergence-c3](/docs/learning/state-012-platform-convergence-c3) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-012-platform-convergence-c3...code%2Fgenerated-state-014-fdc3-intent-interoperability) | `n/a` | ## How To Use This Section 1. Open a state guide from this catalog. 2. Use the GitHub compare link to inspect exact code changes against previous state(s). 3. Read the plain-English delta for rationale and intent. 4. Follow links back to spec architecture/flows/contracts when you need system context. ## Recommended Starting Points For New State Design - `004-containerized-compose-runtime` (C0): Containerized Compose Runtime (NGINX Ingress) - `007-observability-lgtm-compose` (C1): Observability with LGTM on Compose - `009-order-management-matcher` (C2): Order Management and Matcher - `012-platform-convergence-c3` (C3): Platform Convergence C3 --- ## State 001: Simple App - Base Uncontainerized App # State 001 Learning Guide ## Position In Learning Graph - Previous state(s): none - Dotted-line parent(s): none - Next state(s): [002-edge-proxy-uncontainerized](/docs/learning/state-002-edge-proxy-uncontainerized) ## Convergence Metadata - Convergence state: `no` - Convergence level: `none` - Lineage role: `prelude` - Nearest previous convergence: `none` - Nearest next convergence: `none` ## Rendered Code - Generated branch: [code/generated-state-001-baseline-uncontainerized-parity](https://github.com/finos/traderX/tree/code/generated-state-001-baseline-uncontainerized-parity) - Authoring branch (spec source): [main](https://github.com/finos/traderX/tree/main) ## Code Comparison With Previous State - No previous-state compare link for this state. ## Plain-English Code Delta - **Code focus:** Establishes the full baseline service set and Angular UI in a local multi-process runtime. - **Runtime behavior:** Uses explicit host ports and direct cross-origin service calls. - **Learning takeaway:** This is the reference implementation all later state diffs are measured against. ## Run This State ```bash CORS_ALLOWED_ORIGINS=http://localhost:18093 ./scripts/start-base-uncontainerized-generated.sh ``` ## Canonical Spec Links - State spec pack: [/specs/baseline-uncontainerized-parity](/specs/baseline-uncontainerized-parity) - Architecture: [/specs/baseline-uncontainerized-parity/system/architecture](/specs/baseline-uncontainerized-parity/system/architecture) - Flows / topology: [/specs/baseline-uncontainerized-parity/system/end-to-end-flows](/specs/baseline-uncontainerized-parity/system/end-to-end-flows) - Research: [link](/specs/baseline-uncontainerized-parity/research) - Data model: [link](/specs/baseline-uncontainerized-parity/data-model) - Quickstart: [link](/specs/baseline-uncontainerized-parity/quickstart) --- ## State 002: Edge Proxy Uncontainerized # State 002 Learning Guide ## Position In Learning Graph - Previous state(s): [001-baseline-uncontainerized-parity](/docs/learning/state-001-baseline-uncontainerized-parity) - Dotted-line parent(s): none - Next state(s): [003-agentic-harness-foundation](/docs/learning/state-003-agentic-harness-foundation) ## Convergence Metadata - Convergence state: `no` - Convergence level: `none` - Lineage role: `prelude` - Nearest previous convergence: `none` - Nearest next convergence: `none` ## Rendered Code - Generated branch: [code/generated-state-002-edge-proxy-uncontainerized](https://github.com/finos/traderX/tree/code/generated-state-002-edge-proxy-uncontainerized) - Authoring branch (spec source): [main](https://github.com/finos/traderX/tree/main) ## Code Comparison With Previous State - Compare against `001-baseline-uncontainerized-parity`: [code/generated-state-001-baseline-uncontainerized-parity...code/generated-state-002-edge-proxy-uncontainerized](https://github.com/finos/traderX/compare/code%2Fgenerated-state-001-baseline-uncontainerized-parity...code%2Fgenerated-state-002-edge-proxy-uncontainerized) ## Plain-English Code Delta - **Code focus:** Adds an edge proxy component and routes browser traffic through one origin. - **Runtime behavior:** Keeps services uncontainerized while reducing client-side cross-origin complexity. - **Learning takeaway:** Introduces ingress-style edge concerns before containerization. ## Run This State ```bash ./scripts/start-state-002-edge-proxy-generated.sh ``` ## Canonical Spec Links - State spec pack: [/specs/edge-proxy-uncontainerized](/specs/edge-proxy-uncontainerized) - Architecture: [/specs/edge-proxy-uncontainerized/system/architecture](/specs/edge-proxy-uncontainerized/system/architecture) - Flows / topology: [/specs/edge-proxy-uncontainerized/system/runtime-topology](/specs/edge-proxy-uncontainerized/system/runtime-topology) - Research: [link](/specs/edge-proxy-uncontainerized/research) - Data model: [link](/specs/edge-proxy-uncontainerized/data-model) - Quickstart: [link](/specs/edge-proxy-uncontainerized/quickstart) --- ## State 003: Agentic Harness Foundation # State 003 Learning Guide ## Position In Learning Graph - Previous state(s): [002-edge-proxy-uncontainerized](/docs/learning/state-002-edge-proxy-uncontainerized) - Dotted-line parent(s): none - Next state(s): [004-containerized-compose-runtime](/docs/learning/state-004-containerized-compose-runtime) ## Convergence Metadata - Convergence state: `no` - Convergence level: `none` - Lineage role: `prelude` - Nearest previous convergence: `none` - Nearest next convergence: `none` ## Rendered Code - Generated branch: [code/generated-state-003-agentic-harness-foundation](https://github.com/finos/traderX/tree/code/generated-state-003-agentic-harness-foundation) - Authoring branch (spec source): [main](https://github.com/finos/traderX/tree/main) ## Code Comparison With Previous State - Compare against `002-edge-proxy-uncontainerized`: [code/generated-state-002-edge-proxy-uncontainerized...code/generated-state-003-agentic-harness-foundation](https://github.com/finos/traderX/compare/code%2Fgenerated-state-002-edge-proxy-uncontainerized...code%2Fgenerated-state-003-agentic-harness-foundation) ## Plain-English Code Delta - **Added:** Generated codebase root includes `AGENTS.md` with agent operating contract. - **Added:** Generated codebase root includes `ARCHITECTURE.md` summarizing module boundaries and extension points. - **Added:** Generated codebase root includes `CONTRIBUTING.md` clarifying generated snapshot contribution policy. - **Changed:** State lineage introduces a new prelude state before containerized compose baseline. - **Flow Impact:** No runtime endpoint or service-flow changes compared with state `002`. - **Flow Impact:** Development workflow gains explicit agent guidance and contribution boundary in generated outputs. ## Run This State ```bash ./scripts/start-state-003-agentic-harness-foundation-generated.sh ``` ## Canonical Spec Links - State spec pack: [/specs/agentic-harness-foundation](/specs/agentic-harness-foundation) - Architecture: [/specs/agentic-harness-foundation/system/architecture](/specs/agentic-harness-foundation/system/architecture) - Flows / topology: [/specs/agentic-harness-foundation/system/runtime-topology](/specs/agentic-harness-foundation/system/runtime-topology) - Research: [link](/specs/agentic-harness-foundation/research) - Data model: [link](/specs/agentic-harness-foundation/data-model) - Quickstart: [link](/specs/agentic-harness-foundation/quickstart) --- ## State 004: Containerized Compose Runtime (NGINX Ingress) # State 004 Learning Guide ## Position In Learning Graph - Previous state(s): [003-agentic-harness-foundation](/docs/learning/state-003-agentic-harness-foundation) - Dotted-line parent(s): none - Next state(s): [005-postgres-database-replacement](/docs/learning/state-005-postgres-database-replacement) ## Convergence Metadata - Convergence state: `yes` - Convergence level: `C0` - Lineage role: `canonical` - Nearest previous convergence: `none` - Nearest next convergence: [007-observability-lgtm-compose](/docs/learning/state-007-observability-lgtm-compose) ## Rendered Code - Generated branch: [code/generated-state-004-containerized-compose-runtime](https://github.com/finos/traderX/tree/code/generated-state-004-containerized-compose-runtime) - Authoring branch (spec source): [main](https://github.com/finos/traderX/tree/main) ## Code Comparison With Previous State - Compare against `003-agentic-harness-foundation`: [code/generated-state-003-agentic-harness-foundation...code/generated-state-004-containerized-compose-runtime](https://github.com/finos/traderX/compare/code%2Fgenerated-state-003-agentic-harness-foundation...code%2Fgenerated-state-004-containerized-compose-runtime) ## Plain-English Code Delta - **Code focus:** Adds Dockerfiles and Compose assembly for all baseline services. - **Runtime behavior:** Keeps behavior from state 002 but moves orchestration to containers. - **Learning takeaway:** Establishes the containerized baseline that other architecture branches build from. ## Run This State ```bash ./scripts/start-state-004-containerized-generated.sh ``` ## Canonical Spec Links - State spec pack: [/specs/containerized-compose-runtime](/specs/containerized-compose-runtime) - Architecture: [/specs/containerized-compose-runtime/system/architecture](/specs/containerized-compose-runtime/system/architecture) - Flows / topology: [/specs/containerized-compose-runtime/system/runtime-topology](/specs/containerized-compose-runtime/system/runtime-topology) - Research: [link](/specs/containerized-compose-runtime/research) - Data model: [link](/specs/containerized-compose-runtime/data-model) - Quickstart: [link](/specs/containerized-compose-runtime/quickstart) --- ## State 005: PostgreSQL Database Replacement # State 005 Learning Guide ## Position In Learning Graph - Previous state(s): [004-containerized-compose-runtime](/docs/learning/state-004-containerized-compose-runtime) - Dotted-line parent(s): none - Next state(s): [006-messaging-nats-replacement](/docs/learning/state-006-messaging-nats-replacement) ## Convergence Metadata - Convergence state: `no` - Convergence level: `none` - Lineage role: `canonical` - Nearest previous convergence: `none` - Nearest next convergence: `none` ## Rendered Code - Generated branch: [code/generated-state-005-postgres-database-replacement](https://github.com/finos/traderX/tree/code/generated-state-005-postgres-database-replacement) - Authoring branch (spec source): [main](https://github.com/finos/traderX/tree/main) ## Code Comparison With Previous State - Compare against `004-containerized-compose-runtime`: [code/generated-state-004-containerized-compose-runtime...code/generated-state-005-postgres-database-replacement](https://github.com/finos/traderX/compare/code%2Fgenerated-state-004-containerized-compose-runtime...code%2Fgenerated-state-005-postgres-database-replacement) ## Plain-English Code Delta - **Added:** PostgreSQL runtime initialization pack (`postgres-init/initialSchema.sql`) for deterministic baseline schema and seed data. - **Added:** Postgres-specific smoke checks (readiness + baseline data presence). - **Changed:** Runtime database engine changes from H2 server process to PostgreSQL container. - **Changed:** Service datasource wiring for: - **Changed:** `account-service` - **Changed:** `position-service` - **Changed:** `trade-processor` - **Changed:** Account sequence ID generation query changes to PostgreSQL sequence syntax. ## Run This State ```bash ./scripts/start-state-005-postgres-database-replacement-generated.sh ``` ## Canonical Spec Links - State spec pack: [/specs/postgres-database-replacement](/specs/postgres-database-replacement) - Architecture: [/specs/postgres-database-replacement/system/architecture](/specs/postgres-database-replacement/system/architecture) - Flows / topology: [/specs/postgres-database-replacement/system/runtime-topology](/specs/postgres-database-replacement/system/runtime-topology) - Research: [link](/specs/postgres-database-replacement/research) - Data model: [link](/specs/postgres-database-replacement/data-model) - Quickstart: [link](/specs/postgres-database-replacement/quickstart) - State ADR: [link](/docs/adr/006-state-005-use-postgres-for-database-replacement) --- ## State 006: Messaging Layer Replacement with NATS # State 006 Learning Guide ## Position In Learning Graph - Previous state(s): [005-postgres-database-replacement](/docs/learning/state-005-postgres-database-replacement) - Dotted-line parent(s): none - Next state(s): [007-observability-lgtm-compose](/docs/learning/state-007-observability-lgtm-compose) ## Convergence Metadata - Convergence state: `no` - Convergence level: `none` - Lineage role: `canonical` - Nearest previous convergence: `none` - Nearest next convergence: `none` ## Rendered Code - Generated branch: [code/generated-state-006-messaging-nats-replacement](https://github.com/finos/traderX/tree/code/generated-state-006-messaging-nats-replacement) - Authoring branch (spec source): [main](https://github.com/finos/traderX/tree/main) ## Code Comparison With Previous State - Compare against `005-postgres-database-replacement`: [code/generated-state-005-postgres-database-replacement...code/generated-state-006-messaging-nats-replacement](https://github.com/finos/traderX/compare/code%2Fgenerated-state-005-postgres-database-replacement...code%2Fgenerated-state-006-messaging-nats-replacement) ## Plain-English Code Delta - **Added:** Broker-backed subject contract for backend event publication/consumption. - **Added:** Broker-backed websocket stream path for frontend subscriptions. - **Changed:** Trade messaging transport changes: - **Changed:** from Socket.IO channels in `trade-feed`, - **Changed:** to NATS subjects in `nats-broker`. - **Changed:** Event producer/consumer client logic in trade-service, trade-processor, and frontend stream subscriber. - **Changed:** Frontend realtime position handling keeps baseline aggregate blotter semantics by upserting rows for existing securities. - **Removed:** Dedicated Socket.IO messaging service role (`trade-feed`) in target runtime topology. ## Run This State ```bash ./scripts/start-state-006-messaging-nats-replacement-generated.sh ``` ## Canonical Spec Links - State spec pack: [/specs/messaging-nats-replacement](/specs/messaging-nats-replacement) - Architecture: [/specs/messaging-nats-replacement/system/architecture](/specs/messaging-nats-replacement/system/architecture) - Flows / topology: [/specs/messaging-nats-replacement/system/runtime-topology](/specs/messaging-nats-replacement/system/runtime-topology) - Research: [link](/specs/messaging-nats-replacement/research) - Data model: [link](/specs/messaging-nats-replacement/data-model) - Quickstart: [link](/specs/messaging-nats-replacement/quickstart) - State ADR: [link](/docs/adr/005-state-006-use-nats-for-messaging-replacement) --- ## State 007: Observability with LGTM on Compose # State 007 Learning Guide ## Position In Learning Graph - Previous state(s): [006-messaging-nats-replacement](/docs/learning/state-006-messaging-nats-replacement) - Dotted-line parent(s): none - Next state(s): [008-pricing-awareness-market-data](/docs/learning/state-008-pricing-awareness-market-data) ## Convergence Metadata - Convergence state: `yes` - Convergence level: `C1` - Lineage role: `canonical` - Nearest previous convergence: [004-containerized-compose-runtime](/docs/learning/state-004-containerized-compose-runtime) - Nearest next convergence: [009-order-management-matcher](/docs/learning/state-009-order-management-matcher) ## Rendered Code - Generated branch: [code/generated-state-007-observability-lgtm-compose](https://github.com/finos/traderX/tree/code/generated-state-007-observability-lgtm-compose) - Authoring branch (spec source): [main](https://github.com/finos/traderX/tree/main) ## Code Comparison With Previous State - Compare against `006-messaging-nats-replacement`: [code/generated-state-006-messaging-nats-replacement...code/generated-state-007-observability-lgtm-compose](https://github.com/finos/traderX/compare/code%2Fgenerated-state-006-messaging-nats-replacement...code%2Fgenerated-state-007-observability-lgtm-compose) ## Plain-English Code Delta - **Flow Impact:** No functional flow changes from state `006`. - **Flow Impact:** Existing flow IDs remain valid; this state only adds runtime observability capabilities. ## Run This State ```bash ./scripts/start-state-007-observability-lgtm-compose-generated.sh ``` ## Canonical Spec Links - State spec pack: [/specs/observability-lgtm-compose](/specs/observability-lgtm-compose) - Architecture: [/specs/observability-lgtm-compose/system/architecture](/specs/observability-lgtm-compose/system/architecture) - Flows / topology: [/specs/observability-lgtm-compose/system/runtime-topology](/specs/observability-lgtm-compose/system/runtime-topology) - Research: [link](/specs/observability-lgtm-compose/research) - Data model: [link](/specs/observability-lgtm-compose/data-model) - Quickstart: [link](/specs/observability-lgtm-compose/quickstart) - State ADR: [link](/docs/adr/007-state-007-adopt-lgtm-observability-stack) --- ## State 008: Pricing Awareness and Market Data Streaming # State 008 Learning Guide ## Position In Learning Graph - Previous state(s): [007-observability-lgtm-compose](/docs/learning/state-007-observability-lgtm-compose) - Dotted-line parent(s): none - Next state(s): [009-order-management-matcher](/docs/learning/state-009-order-management-matcher) ## Convergence Metadata - Convergence state: `no` - Convergence level: `none` - Lineage role: `canonical` - Nearest previous convergence: `none` - Nearest next convergence: `none` ## Rendered Code - Generated branch: [code/generated-state-008-pricing-awareness-market-data](https://github.com/finos/traderX/tree/code/generated-state-008-pricing-awareness-market-data) - Authoring branch (spec source): [main](https://github.com/finos/traderX/tree/main) ## Code Comparison With Previous State - Compare against `007-observability-lgtm-compose`: [code/generated-state-007-observability-lgtm-compose...code/generated-state-008-pricing-awareness-market-data](https://github.com/finos/traderX/compare/code%2Fgenerated-state-007-observability-lgtm-compose...code%2Fgenerated-state-008-pricing-awareness-market-data) ## Plain-English Code Delta - **Added:** Trade execution pricing (`trade.price`) with 3-decimal precision. - **Added:** Position pre-aggregated volume-weighted average cost basis (`position.averageCostBasis`). - **Added:** Market price stream topics (`pricing.`) from a new `price-publisher` component. - **Added:** Startup-assigned per-ticker volatility band profile for synthetic pricing bounds (20% @ ±4%, 60% @ ±2%, 20% strict open/close). - **Added:** UI valuation fields: market price, position value, unrealized P&L, portfolio totals. - **Added:** Position blotter `OPEN` column and directional market marker (`▲/▼/■`) against open price. - **Added:** Conditional valuation highlighting in position blotter for market-price/open and value-vs-cost comparisons. - **Added:** Trade ticket selected-security live price stream subscription from `pricing.`. ## Run This State ```bash ./scripts/start-state-008-pricing-awareness-market-data-generated.sh ``` ## Canonical Spec Links - State spec pack: [/specs/pricing-awareness-market-data](/specs/pricing-awareness-market-data) - Architecture: [/specs/pricing-awareness-market-data/system/architecture](/specs/pricing-awareness-market-data/system/architecture) - Flows / topology: [/specs/pricing-awareness-market-data/system/runtime-topology](/specs/pricing-awareness-market-data/system/runtime-topology) - Research: [link](/specs/pricing-awareness-market-data/research) - Data model: [link](/specs/pricing-awareness-market-data/data-model) - Quickstart: [link](/specs/pricing-awareness-market-data/quickstart) --- ## State 009: Order Management and Matcher # State 009 Learning Guide ## Position In Learning Graph - Previous state(s): [008-pricing-awareness-market-data](/docs/learning/state-008-pricing-awareness-market-data) - Dotted-line parent(s): none - Next state(s): [010-kubernetes-runtime](/docs/learning/state-010-kubernetes-runtime) ## Convergence Metadata - Convergence state: `yes` - Convergence level: `C2` - Lineage role: `canonical` - Nearest previous convergence: [007-observability-lgtm-compose](/docs/learning/state-007-observability-lgtm-compose) - Nearest next convergence: [012-platform-convergence-c3](/docs/learning/state-012-platform-convergence-c3) ## Rendered Code - Generated branch: [code/generated-state-009-order-management-matcher](https://github.com/finos/traderX/tree/code/generated-state-009-order-management-matcher) - Authoring branch (spec source): [main](https://github.com/finos/traderX/tree/main) ## Code Comparison With Previous State - Compare against `008-pricing-awareness-market-data`: [code/generated-state-008-pricing-awareness-market-data...code/generated-state-009-order-management-matcher](https://github.com/finos/traderX/compare/code%2Fgenerated-state-008-pricing-awareness-market-data...code%2Fgenerated-state-009-order-management-matcher) ## Plain-English Code Delta - **Added:** Order domain with lifecycle states (`NEW`, `PARTIALLY_FILLED`, `FILLED`, `CANCELED`, `REJECTED`). - **Added:** Order matcher component (Java Spring Boot) for evaluating executable orders and publishing fill events. - **Added:** Order management API surface for create/list/cancel/force-fill workflows. - **Added:** Database-backed order persistence so active orders survive order-matcher restarts. - **Added:** Tick-driven auto-fill policy for in-the-money orders: - **Added:** remaining quantity `< 1000`: fill full remaining quantity - **Added:** remaining quantity `>= 1000`: fill half (rounded up) - **Added:** Trader UI order ticket for limit-order creation, separated from market-trade ticket workflow. ## Run This State ```bash ./scripts/start-state-009-order-management-matcher-generated.sh ``` ## Canonical Spec Links - State spec pack: [/specs/order-management-matcher](/specs/order-management-matcher) - Architecture: [/specs/order-management-matcher/system/architecture](/specs/order-management-matcher/system/architecture) - Flows / topology: [/specs/order-management-matcher/system/runtime-topology](/specs/order-management-matcher/system/runtime-topology) - Research: [link](/specs/order-management-matcher/research) - Data model: [link](/specs/order-management-matcher/data-model) - Quickstart: [link](/specs/order-management-matcher/quickstart) --- ## State 010: Kubernetes Runtime on C2 # State 010 Learning Guide ## Position In Learning Graph - Previous state(s): [009-order-management-matcher](/docs/learning/state-009-order-management-matcher) - Dotted-line parent(s): none - Next state(s): [011-tilt-kubernetes-dev-loop](/docs/learning/state-011-tilt-kubernetes-dev-loop) ## Convergence Metadata - Convergence state: `no` - Convergence level: `none` - Lineage role: `canonical` - Nearest previous convergence: `none` - Nearest next convergence: `none` ## Rendered Code - Generated branch: [code/generated-state-010-kubernetes-runtime](https://github.com/finos/traderX/tree/code/generated-state-010-kubernetes-runtime) - Authoring branch (spec source): [main](https://github.com/finos/traderX/tree/main) ## Code Comparison With Previous State - Compare against `009-order-management-matcher`: [code/generated-state-009-order-management-matcher...code/generated-state-010-kubernetes-runtime](https://github.com/finos/traderX/compare/code%2Fgenerated-state-009-order-management-matcher...code%2Fgenerated-state-010-kubernetes-runtime) ## Plain-English Code Delta - **Added:** No new user-facing domain functionality. - **Changed:** Runtime substrate changes from Docker Compose to Kubernetes. - **Changed:** Browser/API entry remains single-origin (`http://localhost:8080`) through an NGINX edge proxy service. - **Removed:** No functional endpoints removed. - **Flow Impact:** F1 Place Trade: unchanged behavior; routed through Kubernetes edge proxy. - **Flow Impact:** F2 View Positions: unchanged behavior; data path unchanged. - **Flow Impact:** F3 Account/User lookups: unchanged behavior; cross-service validation unchanged. - **Flow Impact:** F4 Reference Data lookup: unchanged behavior; ticker validation unchanged. ## Run This State ```bash ./scripts/start-state-010-kubernetes-runtime-generated.sh ``` ## Canonical Spec Links - State spec pack: [/specs/kubernetes-runtime](/specs/kubernetes-runtime) - Architecture: [/specs/kubernetes-runtime/system/architecture](/specs/kubernetes-runtime/system/architecture) - Flows / topology: [/specs/kubernetes-runtime/system/runtime-topology](/specs/kubernetes-runtime/system/runtime-topology) - Research: [link](/specs/kubernetes-runtime/research) - Data model: [link](/specs/kubernetes-runtime/data-model) - Quickstart: [link](/specs/kubernetes-runtime/quickstart) --- ## State 011: Tilt Local Dev on Kubernetes # State 011 Learning Guide ## Position In Learning Graph - Previous state(s): [010-kubernetes-runtime](/docs/learning/state-010-kubernetes-runtime) - Dotted-line parent(s): none - Next state(s): [012-platform-convergence-c3](/docs/learning/state-012-platform-convergence-c3) ## Convergence Metadata - Convergence state: `no` - Convergence level: `none` - Lineage role: `canonical` - Nearest previous convergence: `none` - Nearest next convergence: `none` ## Rendered Code - Generated branch: [code/generated-state-011-tilt-kubernetes-dev-loop](https://github.com/finos/traderX/tree/code/generated-state-011-tilt-kubernetes-dev-loop) - Authoring branch (spec source): [main](https://github.com/finos/traderX/tree/main) ## Code Comparison With Previous State - Compare against `010-kubernetes-runtime`: [code/generated-state-010-kubernetes-runtime...code/generated-state-011-tilt-kubernetes-dev-loop](https://github.com/finos/traderX/compare/code%2Fgenerated-state-010-kubernetes-runtime...code%2Fgenerated-state-011-tilt-kubernetes-dev-loop) ## Plain-English Code Delta - **Flow Impact:** No functional flow deltas. Baseline flows `F1`-`F6` and startup behavior remain compatible with state `010`, including inherited price snapshot+stream freshness behavior and push-based realtime blotter subscriptions. ## Run This State ```bash ./scripts/start-state-011-tilt-kubernetes-dev-loop-generated.sh ``` ## Canonical Spec Links - State spec pack: [/specs/tilt-kubernetes-dev-loop](/specs/tilt-kubernetes-dev-loop) - Architecture: [/specs/tilt-kubernetes-dev-loop/system/architecture](/specs/tilt-kubernetes-dev-loop/system/architecture) - Flows / topology: [/specs/tilt-kubernetes-dev-loop/system/runtime-topology](/specs/tilt-kubernetes-dev-loop/system/runtime-topology) - Research: [link](/specs/tilt-kubernetes-dev-loop/research) - Data model: [link](/specs/tilt-kubernetes-dev-loop/data-model) - Quickstart: [link](/specs/tilt-kubernetes-dev-loop/quickstart) --- ## State 012: Platform Convergence C3 # State 012 Learning Guide ## Position In Learning Graph - Previous state(s): [011-tilt-kubernetes-dev-loop](/docs/learning/state-011-tilt-kubernetes-dev-loop) - Dotted-line parent(s): [009-order-management-matcher](/docs/learning/state-009-order-management-matcher) - Next state(s): [013-radius-kubernetes-platform](/docs/learning/state-013-radius-kubernetes-platform), [014-fdc3-intent-interoperability](/docs/learning/state-014-fdc3-intent-interoperability) ## Convergence Metadata - Convergence state: `yes` - Convergence level: `C3` - Lineage role: `canonical` - Nearest previous convergence: [009-order-management-matcher](/docs/learning/state-009-order-management-matcher) - Nearest next convergence: `none` ## Rendered Code - Generated branch: [code/generated-state-012-platform-convergence-c3](https://github.com/finos/traderX/tree/code/generated-state-012-platform-convergence-c3) - Authoring branch (spec source): [main](https://github.com/finos/traderX/tree/main) ## Code Comparison With Previous State - Compare against `011-tilt-kubernetes-dev-loop`: [code/generated-state-011-tilt-kubernetes-dev-loop...code/generated-state-012-platform-convergence-c3](https://github.com/finos/traderX/compare/code%2Fgenerated-state-011-tilt-kubernetes-dev-loop...code%2Fgenerated-state-012-platform-convergence-c3) ## Plain-English Code Delta - **Flow Impact:** No functional flow deltas. Baseline flows `F1`-`F6` and startup behavior remain compatible with state `011`, including inherited price snapshot+stream freshness behavior and push-based realtime blotter subscriptions. ## Run This State ```bash ./scripts/start-state-012-platform-convergence-c3-generated.sh ``` ## Canonical Spec Links - State spec pack: [/specs/platform-convergence-c3](/specs/platform-convergence-c3) - Architecture: [/specs/platform-convergence-c3/system/architecture](/specs/platform-convergence-c3/system/architecture) - Flows / topology: [/specs/platform-convergence-c3/system/runtime-topology](/specs/platform-convergence-c3/system/runtime-topology) - Research: [link](/specs/platform-convergence-c3/research) - Data model: [link](/specs/platform-convergence-c3/data-model) - Quickstart: [link](/specs/platform-convergence-c3/quickstart) - State ADR: [link](/docs/adr/008-convergence-state-model) --- ## State 013: Radius Platform on Kubernetes (Optional) # State 013 Learning Guide ## Position In Learning Graph - Previous state(s): [012-platform-convergence-c3](/docs/learning/state-012-platform-convergence-c3) - Dotted-line parent(s): none - Next state(s): none ## Convergence Metadata - Convergence state: `no` - Convergence level: `none` - Lineage role: `optional` - Nearest previous convergence: `none` - Nearest next convergence: `none` ## Rendered Code - Generated branch: [code/generated-state-013-radius-kubernetes-platform](https://github.com/finos/traderX/tree/code/generated-state-013-radius-kubernetes-platform) - Authoring branch (spec source): [main](https://github.com/finos/traderX/tree/main) ## Code Comparison With Previous State - Compare against `012-platform-convergence-c3`: [code/generated-state-012-platform-convergence-c3...code/generated-state-013-radius-kubernetes-platform](https://github.com/finos/traderX/compare/code%2Fgenerated-state-012-platform-convergence-c3...code%2Fgenerated-state-013-radius-kubernetes-platform) ## Plain-English Code Delta - **Flow Impact:** No functional flow deltas. Baseline flows `F1`-`F6` and startup behavior remain compatible with state `012`. ## Run This State ```bash ./scripts/start-state-013-radius-kubernetes-platform-generated.sh ``` ## Canonical Spec Links - State spec pack: [/specs/radius-kubernetes-platform](/specs/radius-kubernetes-platform) - Architecture: [/specs/radius-kubernetes-platform/system/architecture](/specs/radius-kubernetes-platform/system/architecture) - Flows / topology: [/specs/radius-kubernetes-platform/system/runtime-topology](/specs/radius-kubernetes-platform/system/runtime-topology) - Research: [link](/specs/radius-kubernetes-platform/research) - Data model: [link](/specs/radius-kubernetes-platform/data-model) - Quickstart: [link](/specs/radius-kubernetes-platform/quickstart) --- ## State 014: FDC3 Intent Interoperability on C3 # State 014 Learning Guide ## Position In Learning Graph - Previous state(s): [012-platform-convergence-c3](/docs/learning/state-012-platform-convergence-c3) - Dotted-line parent(s): none - Next state(s): none ## Convergence Metadata - Convergence state: `no` - Convergence level: `none` - Lineage role: `canonical` - Nearest previous convergence: `none` - Nearest next convergence: `none` ## Rendered Code - Generated branch: [code/generated-state-014-fdc3-intent-interoperability](https://github.com/finos/traderX/tree/code/generated-state-014-fdc3-intent-interoperability) - Authoring branch (spec source): [main](https://github.com/finos/traderX/tree/main) ## Code Comparison With Previous State - Compare against `012-platform-convergence-c3`: [code/generated-state-012-platform-convergence-c3...code/generated-state-014-fdc3-intent-interoperability](https://github.com/finos/traderX/compare/code%2Fgenerated-state-012-platform-convergence-c3...code%2Fgenerated-state-014-fdc3-intent-interoperability) ## Plain-English Code Delta - **Added:** Orders, trades and positions start on **All tickers**. Each blotter has a labelled, keyboard-accessible **Filter on selected ticker** control and a status showing its effective ticker (or no ticker selected). - **Added:** Ticker mode stays local: selecting securities still shares FDC3 context with charts/tickets, turning filtering off restores all account rows, and opting back in uses the latest selection. Account changes and tab switches preserve modes; new windows default to All tickers even with retained context. - **Added:** Account selection propagates independently through `fdc3.account`. Live order updates use the configured event transport, not FDC3; snapshot reconciliation protects creates/fills/cancellations and reconnect refreshes missed data. - **Added:** Empty or invalid instrument messages retain the last valid selection; this state defines no clear-instrument message. Before any selection, an opted-in blotter shows all tickers and explains why. - **Added:** FDC3 interop adapter in TraderX frontend for DesktopAgent integration. - **Added:** Outbound `fdc3.instrument` context publishing when a user selects a ticker-bearing row in trade/order/position views. - **Added:** Inbound `fdc3.instrument` context handling that retains shared selection for optional blotter filters and ticket defaults. - **Added:** Standard intent handling for `ViewOrders` (orders view routing without implicit filter opt-in). ## Run This State ```bash ./scripts/start-state-014-fdc3-intent-interoperability-generated.sh ``` ## Canonical Spec Links - State spec pack: [/specs/fdc3-intent-interoperability](/specs/fdc3-intent-interoperability) - Architecture: [/specs/fdc3-intent-interoperability/system/architecture](/specs/fdc3-intent-interoperability/system/architecture) - Flows / topology: [/specs/fdc3-intent-interoperability/system/runtime-topology](/specs/fdc3-intent-interoperability/system/runtime-topology) - Research: [link](/specs/fdc3-intent-interoperability/research) - Data model: [link](/specs/fdc3-intent-interoperability/data-model) - Quickstart: [link](/specs/fdc3-intent-interoperability/quickstart) --- ## Learning Paths This page is generated from `catalog/state-catalog.json`. Green nodes represent convergence checkpoints (C-level milestones such as `[C0]`, `[C1]`, `[C2]`, `[C3]`). ## Convergence-Level Graph This high-level view shows only the canonical convergence progression from `C0` to `C3`. ```mermaid flowchart LR S004_containerized_compose_runtime["004: Containerized Compose Runtime (NGINX Ingress) [C0]"] S007_observability_lgtm_compose["007: Observability with LGTM on Compose [C1]"] S009_order_management_matcher["009: Order Management and Matcher [C2]"] S012_platform_convergence_c3["012: Platform Convergence C3 [C3]"] S004_containerized_compose_runtime --> S007_observability_lgtm_compose S007_observability_lgtm_compose --> S009_order_management_matcher S009_order_management_matcher --> S012_platform_convergence_c3 click S004_containerized_compose_runtime href "/specs/containerized-compose-runtime" "Open 004-containerized-compose-runtime" click S007_observability_lgtm_compose href "/specs/observability-lgtm-compose" "Open 007-observability-lgtm-compose" click S009_order_management_matcher href "/specs/order-management-matcher" "Open 009-order-management-matcher" click S012_platform_convergence_c3 href "/specs/platform-convergence-c3" "Open 012-platform-convergence-c3" classDef convergence fill:#d7f5dd,stroke:#2e7d32,stroke-width:2px class S004_containerized_compose_runtime convergence class S007_observability_lgtm_compose convergence class S009_order_management_matcher convergence class S012_platform_convergence_c3 convergence ``` ## Official Current Graph ```mermaid flowchart TB S001_baseline_uncontainerized_parity["001: Simple App - Base Uncontainerized App"] S002_edge_proxy_uncontainerized["002: Edge Proxy Uncontainerized"] S003_agentic_harness_foundation["003: Agentic Harness Foundation"] S004_containerized_compose_runtime["004: Containerized Compose Runtime (NGINX Ingress) [C0]"] S005_postgres_database_replacement["005: PostgreSQL Database Replacement"] S006_messaging_nats_replacement["006: Messaging Layer Replacement with NATS"] S007_observability_lgtm_compose["007: Observability with LGTM on Compose [C1]"] S008_pricing_awareness_market_data["008: Pricing Awareness and Market Data Streaming"] S009_order_management_matcher["009: Order Management and Matcher [C2]"] S010_kubernetes_runtime["010: Kubernetes Runtime on C2"] S011_tilt_kubernetes_dev_loop["011: Tilt Local Dev on Kubernetes"] S012_platform_convergence_c3["012: Platform Convergence C3 [C3]"] S013_radius_kubernetes_platform["013: Radius Platform on Kubernetes (Optional)"] S014_fdc3_intent_interoperability["014: FDC3 Intent Interoperability on C3"] S001_baseline_uncontainerized_parity --> S002_edge_proxy_uncontainerized S002_edge_proxy_uncontainerized --> S003_agentic_harness_foundation S003_agentic_harness_foundation --> S004_containerized_compose_runtime S004_containerized_compose_runtime --> S005_postgres_database_replacement S005_postgres_database_replacement --> S006_messaging_nats_replacement S006_messaging_nats_replacement --> S007_observability_lgtm_compose S007_observability_lgtm_compose --> S008_pricing_awareness_market_data S008_pricing_awareness_market_data --> S009_order_management_matcher S009_order_management_matcher --> S010_kubernetes_runtime S010_kubernetes_runtime --> S011_tilt_kubernetes_dev_loop S011_tilt_kubernetes_dev_loop --> S012_platform_convergence_c3 S012_platform_convergence_c3 --> S013_radius_kubernetes_platform S012_platform_convergence_c3 --> S014_fdc3_intent_interoperability S009_order_management_matcher -.-> S012_platform_convergence_c3 click S001_baseline_uncontainerized_parity href "/specs/baseline-uncontainerized-parity" "Open State 001 Spec Pack" click S002_edge_proxy_uncontainerized href "/specs/edge-proxy-uncontainerized" "Open State 002 Spec Pack" click S003_agentic_harness_foundation href "/specs/agentic-harness-foundation" "Open State 003 Spec Pack" click S004_containerized_compose_runtime href "/specs/containerized-compose-runtime" "Open State 004 Spec Pack" click S005_postgres_database_replacement href "/specs/postgres-database-replacement" "Open State 005 Spec Pack" click S006_messaging_nats_replacement href "/specs/messaging-nats-replacement" "Open State 006 Spec Pack" click S007_observability_lgtm_compose href "/specs/observability-lgtm-compose" "Open State 007 Spec Pack" click S008_pricing_awareness_market_data href "/specs/pricing-awareness-market-data" "Open State 008 Spec Pack" click S009_order_management_matcher href "/specs/order-management-matcher" "Open State 009 Spec Pack" click S010_kubernetes_runtime href "/specs/kubernetes-runtime" "Open State 010 Spec Pack" click S011_tilt_kubernetes_dev_loop href "/specs/tilt-kubernetes-dev-loop" "Open State 011 Spec Pack" click S012_platform_convergence_c3 href "/specs/platform-convergence-c3" "Open State 012 Spec Pack" click S013_radius_kubernetes_platform href "/specs/radius-kubernetes-platform" "Open State 013 Spec Pack" click S014_fdc3_intent_interoperability href "/specs/fdc3-intent-interoperability" "Open State 014 Spec Pack" classDef convergence fill:#d7f5dd,stroke:#2e7d32,stroke-width:2px class S004_containerized_compose_runtime convergence class S007_observability_lgtm_compose convergence class S009_order_management_matcher convergence class S012_platform_convergence_c3 convergence ``` ## State To Artifact Mapping | State | Spec Pack | Architecture | Flows / Runtime Topology | Learning Guide | Generated Code Branch | | --- | --- | --- | --- | --- | --- | | [`001-baseline-uncontainerized-parity`](pathname:///specs/baseline-uncontainerized-parity) | [link](pathname:///specs/baseline-uncontainerized-parity) | [link](pathname:///specs/baseline-uncontainerized-parity/system/architecture) | [link](pathname:///specs/baseline-uncontainerized-parity/system/end-to-end-flows) | [link](pathname:///docs/learning/state-001-baseline-uncontainerized-parity) | [code/generated-state-001-baseline-uncontainerized-parity](https://github.com/finos/traderX/tree/code/generated-state-001-baseline-uncontainerized-parity) | | [`002-edge-proxy-uncontainerized`](pathname:///specs/edge-proxy-uncontainerized) | [link](pathname:///specs/edge-proxy-uncontainerized) | [link](pathname:///specs/edge-proxy-uncontainerized/system/architecture) | [link](pathname:///specs/edge-proxy-uncontainerized/system/runtime-topology) | [link](pathname:///docs/learning/state-002-edge-proxy-uncontainerized) | [code/generated-state-002-edge-proxy-uncontainerized](https://github.com/finos/traderX/tree/code/generated-state-002-edge-proxy-uncontainerized) | | [`003-agentic-harness-foundation`](pathname:///specs/agentic-harness-foundation) | [link](pathname:///specs/agentic-harness-foundation) | [link](pathname:///specs/agentic-harness-foundation/system/architecture) | [link](pathname:///specs/agentic-harness-foundation/system/runtime-topology) | [link](pathname:///docs/learning/state-003-agentic-harness-foundation) | [code/generated-state-003-agentic-harness-foundation](https://github.com/finos/traderX/tree/code/generated-state-003-agentic-harness-foundation) | | **[`004-containerized-compose-runtime`](pathname:///specs/containerized-compose-runtime)** [(C0)](pathname:///docs/spec-kit/convergence-states#c0) | [link](pathname:///specs/containerized-compose-runtime) | [link](pathname:///specs/containerized-compose-runtime/system/architecture) | [link](pathname:///specs/containerized-compose-runtime/system/runtime-topology) | [link](pathname:///docs/learning/state-004-containerized-compose-runtime) | [code/generated-state-004-containerized-compose-runtime](https://github.com/finos/traderX/tree/code/generated-state-004-containerized-compose-runtime) | | [`005-postgres-database-replacement`](pathname:///specs/postgres-database-replacement) | [link](pathname:///specs/postgres-database-replacement) | [link](pathname:///specs/postgres-database-replacement/system/architecture) | [link](pathname:///specs/postgres-database-replacement/system/runtime-topology) | [link](pathname:///docs/learning/state-005-postgres-database-replacement) | [code/generated-state-005-postgres-database-replacement](https://github.com/finos/traderX/tree/code/generated-state-005-postgres-database-replacement) | | [`006-messaging-nats-replacement`](pathname:///specs/messaging-nats-replacement) | [link](pathname:///specs/messaging-nats-replacement) | [link](pathname:///specs/messaging-nats-replacement/system/architecture) | [link](pathname:///specs/messaging-nats-replacement/system/runtime-topology) | [link](pathname:///docs/learning/state-006-messaging-nats-replacement) | [code/generated-state-006-messaging-nats-replacement](https://github.com/finos/traderX/tree/code/generated-state-006-messaging-nats-replacement) | | **[`007-observability-lgtm-compose`](pathname:///specs/observability-lgtm-compose)** [(C1)](pathname:///docs/spec-kit/convergence-states#c1) | [link](pathname:///specs/observability-lgtm-compose) | [link](pathname:///specs/observability-lgtm-compose/system/architecture) | [link](pathname:///specs/observability-lgtm-compose/system/runtime-topology) | [link](pathname:///docs/learning/state-007-observability-lgtm-compose) | [code/generated-state-007-observability-lgtm-compose](https://github.com/finos/traderX/tree/code/generated-state-007-observability-lgtm-compose) | | [`008-pricing-awareness-market-data`](pathname:///specs/pricing-awareness-market-data) | [link](pathname:///specs/pricing-awareness-market-data) | [link](pathname:///specs/pricing-awareness-market-data/system/architecture) | [link](pathname:///specs/pricing-awareness-market-data/system/runtime-topology) | [link](pathname:///docs/learning/state-008-pricing-awareness-market-data) | [code/generated-state-008-pricing-awareness-market-data](https://github.com/finos/traderX/tree/code/generated-state-008-pricing-awareness-market-data) | | **[`009-order-management-matcher`](pathname:///specs/order-management-matcher)** [(C2)](pathname:///docs/spec-kit/convergence-states#c2) | [link](pathname:///specs/order-management-matcher) | [link](pathname:///specs/order-management-matcher/system/architecture) | [link](pathname:///specs/order-management-matcher/system/runtime-topology) | [link](pathname:///docs/learning/state-009-order-management-matcher) | [code/generated-state-009-order-management-matcher](https://github.com/finos/traderX/tree/code/generated-state-009-order-management-matcher) | | [`010-kubernetes-runtime`](pathname:///specs/kubernetes-runtime) | [link](pathname:///specs/kubernetes-runtime) | [link](pathname:///specs/kubernetes-runtime/system/architecture) | [link](pathname:///specs/kubernetes-runtime/system/runtime-topology) | [link](pathname:///docs/learning/state-010-kubernetes-runtime) | [code/generated-state-010-kubernetes-runtime](https://github.com/finos/traderX/tree/code/generated-state-010-kubernetes-runtime) | | [`011-tilt-kubernetes-dev-loop`](pathname:///specs/tilt-kubernetes-dev-loop) | [link](pathname:///specs/tilt-kubernetes-dev-loop) | [link](pathname:///specs/tilt-kubernetes-dev-loop/system/architecture) | [link](pathname:///specs/tilt-kubernetes-dev-loop/system/runtime-topology) | [link](pathname:///docs/learning/state-011-tilt-kubernetes-dev-loop) | [code/generated-state-011-tilt-kubernetes-dev-loop](https://github.com/finos/traderX/tree/code/generated-state-011-tilt-kubernetes-dev-loop) | | **[`012-platform-convergence-c3`](pathname:///specs/platform-convergence-c3)** [(C3)](pathname:///docs/spec-kit/convergence-states#c3) | [link](pathname:///specs/platform-convergence-c3) | [link](pathname:///specs/platform-convergence-c3/system/architecture) | [link](pathname:///specs/platform-convergence-c3/system/runtime-topology) | [link](pathname:///docs/learning/state-012-platform-convergence-c3) | [code/generated-state-012-platform-convergence-c3](https://github.com/finos/traderX/tree/code/generated-state-012-platform-convergence-c3) | | [`013-radius-kubernetes-platform`](pathname:///specs/radius-kubernetes-platform) | [link](pathname:///specs/radius-kubernetes-platform) | [link](pathname:///specs/radius-kubernetes-platform/system/architecture) | [link](pathname:///specs/radius-kubernetes-platform/system/runtime-topology) | [link](pathname:///docs/learning/state-013-radius-kubernetes-platform) | [code/generated-state-013-radius-kubernetes-platform](https://github.com/finos/traderX/tree/code/generated-state-013-radius-kubernetes-platform) | | [`014-fdc3-intent-interoperability`](pathname:///specs/fdc3-intent-interoperability) | [link](pathname:///specs/fdc3-intent-interoperability) | [link](pathname:///specs/fdc3-intent-interoperability/system/architecture) | [link](pathname:///specs/fdc3-intent-interoperability/system/runtime-topology) | [link](pathname:///docs/learning/state-014-fdc3-intent-interoperability) | [code/generated-state-014-fdc3-intent-interoperability](https://github.com/finos/traderX/tree/code/generated-state-014-fdc3-intent-interoperability) | ## Swimlane View ```mermaid flowchart TB subgraph PRELUDE["Prelude Track"] S001_baseline_uncontainerized_parity["001: Simple App - Base Uncontainerized App"] S002_edge_proxy_uncontainerized["002: Edge Proxy Uncontainerized"] S003_agentic_harness_foundation["003: Agentic Harness Foundation"] end subgraph BASELINE["Baseline Track"] S004_containerized_compose_runtime["004: Containerized Compose Runtime (NGINX Ingress) [C0]"] end subgraph ARCHITECTURE["Architecture Track"] S005_postgres_database_replacement["005: PostgreSQL Database Replacement"] S006_messaging_nats_replacement["006: Messaging Layer Replacement with NATS"] end subgraph NONFUNCTIONAL["Nonfunctional Track"] S007_observability_lgtm_compose["007: Observability with LGTM on Compose [C1]"] end subgraph FUNCTIONAL["Functional Track"] S008_pricing_awareness_market_data["008: Pricing Awareness and Market Data Streaming"] S009_order_management_matcher["009: Order Management and Matcher [C2]"] S014_fdc3_intent_interoperability["014: FDC3 Intent Interoperability on C3"] end subgraph DEVEX["Devex Track"] S010_kubernetes_runtime["010: Kubernetes Runtime on C2"] S011_tilt_kubernetes_dev_loop["011: Tilt Local Dev on Kubernetes"] S012_platform_convergence_c3["012: Platform Convergence C3 [C3]"] S013_radius_kubernetes_platform["013: Radius Platform on Kubernetes (Optional)"] end S001_baseline_uncontainerized_parity --> S002_edge_proxy_uncontainerized S002_edge_proxy_uncontainerized --> S003_agentic_harness_foundation S003_agentic_harness_foundation --> S004_containerized_compose_runtime S004_containerized_compose_runtime --> S005_postgres_database_replacement S005_postgres_database_replacement --> S006_messaging_nats_replacement S006_messaging_nats_replacement --> S007_observability_lgtm_compose S007_observability_lgtm_compose --> S008_pricing_awareness_market_data S008_pricing_awareness_market_data --> S009_order_management_matcher S009_order_management_matcher --> S010_kubernetes_runtime S010_kubernetes_runtime --> S011_tilt_kubernetes_dev_loop S011_tilt_kubernetes_dev_loop --> S012_platform_convergence_c3 S012_platform_convergence_c3 --> S013_radius_kubernetes_platform S012_platform_convergence_c3 --> S014_fdc3_intent_interoperability S009_order_management_matcher -.-> S012_platform_convergence_c3 classDef convergence fill:#d7f5dd,stroke:#2e7d32,stroke-width:2px ``` Use `catalog/state-catalog.json` as the canonical state lineage record. --- ## Dependency security remediation (#465) The source baseline includes the H2 connection-retirement and after-commit publication fixes from #461 / PR #463. Do not publish a replacement snapshot from a source revision that omits those fixes. This change does not authorize a demo deployment or a new security exception. ## Version decisions The complete HTML reports and failed-job logs for [state 002](https://github.com/finos/traderX/actions/runs/35203378098) and [state 004](https://github.com/finos/traderX/actions/runs/35204061241) report the dependency families below. Multer is included here even though it was omitted from the issue's initial examples. All original state 004 image scan jobs passed; replacement images still require fresh scans. | Dependency | Old version | Replacement | Evidence | | --- | --- | --- | --- | | qs | 6.15.2 | 6.16.0 | [GHSA-4mjr-xmp4-gh2g](https://github.com/advisories/GHSA-4mjr-xmp4-gh2g) | | Multer | 2.2.0 | 2.3.0 | [GHSA-535w-7cp7-47q4](https://github.com/advisories/GHSA-535w-7cp7-47q4), [GHSA-qfvm-cv95-jqjf](https://github.com/advisories/GHSA-qfvm-cv95-jqjf), [GHSA-wc9g-mqfw-jrwm](https://github.com/advisories/GHSA-wc9g-mqfw-jrwm) | | Spring Framework | 6.2.19 | 7.0.9 via Boot 4.0.8 | Vendor advisories below | | Spring Data JPA | 3.5.13 | 4.0.7 via Boot 4.0.8 | [CVE-2026-47834](https://spring.io/security/cve-2026-47834/) | | Embedded Tomcat | 10.1.57 | 11.0.26 | [Tomcat 11 security advisories](https://tomcat.apache.org/security-11.html), fixes in 11.0.25 | The Spring 6.2.20 and Data JPA 3.5.14 fixes are enterprise-only and are not available from Maven Central. Boot 4.0.8 provides the patched public Spring Framework and Data JPA releases. Java 21 and Gradle 8.14.5 remain supported. Springdoc moves to 3.0.3 for Boot 4 compatibility, and Jackson moves to Boot's 3.1.5 baseline. The Socket.IO and NATS adapters use Jackson 3's immutable mapper builder while retaining null omission and the existing message envelope. Jackson annotations retain their `com.fasterxml.jackson.annotation` package. See the [Boot migration guide](https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-4.0-Migration-Guide) and [Springdoc compatibility information](https://springdoc.org/). A fresh npm audit additionally reports [GHSA-2m8v-j782-fhvr](https://github.com/advisories/GHSA-2m8v-j782-fhvr) in Socket.IO parser 4.2.6. The canonical override is therefore 4.2.7, including browser and pricing-publisher dependencies. ## Complete reported Java finding inventory All findings below are addressed by upgrading; no applicability suppression or scan-threshold reduction is introduced. Scanner severity is not an independent exploitability assessment. Spring Framework fixes are documented individually by the vendor: - [CVE-2026-47883](https://spring.io/security/cve-2026-47883/) - [CVE-2026-47884](https://spring.io/security/cve-2026-47884/) - [CVE-2026-47885](https://spring.io/security/cve-2026-47885/) - [CVE-2026-47886](https://spring.io/security/cve-2026-47886/) - [CVE-2026-47887](https://spring.io/security/cve-2026-47887/) - [CVE-2026-47888](https://spring.io/security/cve-2026-47888/) - [CVE-2026-47889](https://spring.io/security/cve-2026-47889/) - [CVE-2026-47890](https://spring.io/security/cve-2026-47890/) - [CVE-2026-47891](https://spring.io/security/cve-2026-47891/) - [CVE-2026-47892](https://spring.io/security/cve-2026-47892/) - [CVE-2026-47893](https://spring.io/security/cve-2026-47893/) - [CVE-2026-59281](https://spring.io/security/cve-2026-59281/) - [CVE-2026-59282](https://spring.io/security/cve-2026-59282/) - [CVE-2026-59283](https://spring.io/security/cve-2026-59283/) - [CVE-2026-59313](https://spring.io/security/cve-2026-59313/) The Tomcat findings are CVE-2026-65182, CVE-2026-65183, CVE-2026-65637, CVE-2026-65905, CVE-2026-65927, CVE-2026-66299, CVE-2026-66422, CVE-2026-68525, CVE-2026-68569, CVE-2026-68763, and CVE-2026-73180. The vendor lists all eleven as fixed in 11.0.25; 11.0.26 includes those fixes. ## Generation and release Canonical versions live in `catalog/dependency-version-targets.json` and the templates. State patchsets must carry the same Java baseline and Jackson API changes. Generation synchronizes Node overrides and refreshes lockfiles; verify the resolved versions as well as manifest pins before publication. Regenerate states sequentially, including 002 and 004, with an isolated `TRADERX_GENERATED_ROOT` when another task is using the normal output directory. Run repository gates, booking regressions, application builds, and dependency and image scans before using `pipeline/publish-generated-state-branch.sh`. Each published generated branch must remain one snapshot commit above its parent. Record source and snapshot revisions and scan runs in the remediation PR. Descendant states inherit this baseline and require regeneration before their next publication. ## Runtime image remediation Fresh Trivy scans also found OS and bundled npm-tooling vulnerabilities in the old Node 20 Alpine images and Debian-based .NET image. Node 20 is EOL according to the [Node release schedule](https://nodejs.org/en/about/previous-releases). The Node services now build on Node 24 LTS and run compiled code with production dependencies, updated Alpine packages, and no bundled npm/Yarn tooling. Runtime files belong to the unprivileged node user. The people service uses `aspnet:9.0-noble-chiseled-extra`, retaining ICU/timezone support while omitting unneeded Debian utilities; see [Microsoft's image guidance](https://learn.microsoft.com/en-us/dotnet/core/docker/container-images). The catalog's `docker.runtimeImages` entries and generated-target validator guard these final runtime stages. No image finding is suppressed. ## Validation record (2026-09-17) - All four repository quality gates and the documentation website build pass. - H2 schema lifetime and after-commit publication tests pass for baseline and pricing templates. - The added wire-contract test checks the legacy `accountID` input alias, numeric envelope date, enum and quantity values, and omission of null fields. It caught Jackson 3's changed date default before explicit timestamp configuration was added. - State 002 and 004 application builds pass. All four Java services start and serve OpenAPI documents; account and position endpoint smoke checks pass. - Nine state 004 runtime dependency scans pass at CVSS 5 with the existing suppressions. Inputs were resolved npm lockfiles, built Java archives, and .NET output. Dependency-Check 13.0.0 used the CI action image's September 16 database copied into the native scanner image. Live NVD updates failed, so stale local cached data was not used to establish a pass. OSS Index was unavailable without credentials, as in the existing CI setup. - All ten state 004 images build and pass Trivy 0.74.0 HIGH/CRITICAL scans. Local images are Linux ARM64; remote CI must validate the published snapshot and its target architecture independently. Images were exported before scanning to avoid concurrent local image cleanup. - No demo deployment was performed. Final replacement snapshots and remote checks are recorded below. The first replacement state 002 CI scan additionally found Angular advisories [GHSA-jhpw-976m-542j](https://github.com/angular/angular/security/advisories/GHSA-jhpw-976m-542j), [GHSA-jj27-h5hq-8x99](https://github.com/angular/angular/security/advisories/GHSA-jj27-h5hq-8x99), and [GHSA-hh8m-fm6v-7cvg](https://github.com/angular/angular/security/advisories/GHSA-hh8m-fm6v-7cvg). Angular framework and tooling minimums are raised to 20.3.28; regenerated lockfiles and remote checks are required because the earlier local scan did not detect these findings. Full HTML artifact review also includes low-severity Spring CVE-2026-59280 and CVE-2026-59314 (covered by Framework 7.0.9), qs GHSA-x5fp-wj9c-mxmx (covered by 6.16.0), Multer GHSA-qvfw-j98x-7q72 (covered by 2.3.0), and body-parser GHSA-v422-hmwv-36x6 (explicit reference-data override to 2.3.0). The reports retain the pre-existing CVE-2026-53914 suppression; this change adds no exception. ## Final replacement snapshots Both snapshots use source `2ed48551763d0a47b93070d226fb1b1e10b5e6d7` and preserve one snapshot commit above their respective parents. | State | Snapshot | Parent | Remote evidence | | --- | --- | --- | --- | | 002 | `3187136c1072f91cf4d9bbe9cbaceeaca8c521c5` | `d26ca87b16c985c66bc9e6bf7abd40ef0f90184f` | [Security passed](https://github.com/finos/traderX/actions/runs/35210750150) | | 004 | `43ffeb3640d36458e432b8a4b33a391e2ce0733d` | `1ae7eb50ff297819ada715c01a933cb73714a54b` | [Dependency and image security passed](https://github.com/finos/traderX/actions/runs/35210960971); [image build/publication passed](https://github.com/finos/traderX/actions/runs/35210960714) | Final state 009 regeneration and complete application preflight also pass, including the matcher and H2 booking regression in the PostgreSQL descendant. The fixture is generated from the canonical H2 schema; it is not a second maintained schema. Publication ran sequentially through the supported publisher with isolated output roots. Its repeated prepublish gate was skipped after standalone scans because live NVD updates failed; application preflight and lineage validation still ran. The successful remote checks above establish replacement-snapshot security status. No security exception, scan suppression, or demo deployment was added. An earlier image publication failed on a Maven download HTTP 403; the final publication completed successfully. --- ## Repair existing H2 booking constraints Issue #461 affects literal IN-list CHECK constraints in H2 2.4.240 after the schema-creating connection closes. Fresh generated schemas use equivalent CASE checks. Existing databases need a migration; do not run initialSchema.sql, which resets tables. Do not replay historical failed submissions automatically. 1. Stop all writers, including the matcher and trade processor. Back up the H2 database with `BACKUP TO '/absolute/path/traderx-before-461.zip'` using an admin connection (or stop H2 and copy its database files). Verify the backup exists. 2. Query `INFORMATION_SCHEMA.TABLE_CONSTRAINTS` joined to `INFORMATION_SCHEMA.CHECK_CONSTRAINTS` on constraint catalog/schema/name. Record the four literal IN checks on Trades.Side, Trades.State, OrderBook.Side, and OrderBook.Status and their actual constraint names. Do not assume generated names. 3. Add the following replacements. H2 validates existing rows when adding each constraint. If any addition fails, stop and investigate; retain all old checks. ```sql ALTER TABLE Trades ADD CONSTRAINT trades_side_case_461 CHECK (CASE Side WHEN 'Buy' THEN TRUE WHEN 'Sell' THEN TRUE ELSE Side IS NULL END); ALTER TABLE Trades ADD CONSTRAINT trades_state_case_461 CHECK (CASE State WHEN 'New' THEN TRUE WHEN 'Processing' THEN TRUE WHEN 'Settled' THEN TRUE WHEN 'Cancelled' THEN TRUE ELSE State IS NULL END); ALTER TABLE OrderBook ADD CONSTRAINT orderbook_side_case_461 CHECK (CASE Side WHEN 'Buy' THEN TRUE WHEN 'Sell' THEN TRUE ELSE Side IS NULL END); ALTER TABLE OrderBook ADD CONSTRAINT orderbook_status_case_461 CHECK (CASE Status WHEN 'NEW' THEN TRUE WHEN 'PARTIALLY_FILLED' THEN TRUE WHEN 'FILLED' THEN TRUE WHEN 'CANCELED' THEN TRUE WHEN 'REJECTED' THEN TRUE ELSE Status IS NULL END); ``` 4. Only after all four replacements validate, use `ALTER TABLE DROP CONSTRAINT ` for the four original IN checks. Preserve all quantity checks and foreign keys. H2 DDL commits individually: on an interrupted migration, inspect installed constraints before continuing; do not blindly rerun. 5. Close the migration connection. Verify valid inserts and invalid side/state rejection from a new connection using transactions that are rolled back. Deploy the updated processor, resume writers, and verify two sequential buys accumulate in REST, live grids, and after reload. Verify a filled limit order in environments with a matcher. Preserve existing rows and retain the backup until verified. Publication is best effort after commit. A logged notification failure means the booking is already durable; reload REST state and investigate the feed. Never resubmit the booking just to retry a notification. Crash-safe retries require a future transactional outbox. --- ## Overview State-specific architecture and flow documentation now lives under root SpecKit feature packs. Generated runnable code snapshots are published as state branches documented in [Generated State Branches](./spec-kit/generated-state-branches.md). Project ADRs (Architecture Decision Records) are documented at [/docs/adr](/docs/adr) and follow the [MADR format/specification](https://github.com/adr/madr). - Baseline architecture: [State 001 Architecture](/specs/baseline-uncontainerized-parity/system/architecture) - Baseline flows: [State 001 End-to-End Flows](/specs/baseline-uncontainerized-parity/system/end-to-end-flows) - Edge-proxy architecture: [State 002 Architecture](/specs/edge-proxy-uncontainerized/system/architecture) - Containerized architecture: [State 004 Architecture](/specs/containerized-compose-runtime/system/architecture) --- ## Project History # Project History & Release Notes This page tracks major milestones, architectural changes, and notable releases for TraderX. ## 2026 ### January 2026 - **Docs restructure**: Reorganized and clarified the docs to improve onboarding for both humans and agents - **AI-friendly guidance**: Added `AGENTS.md` plus code review guidance to help agentic tools and contributors work consistently in this repo ## 2023 ### OSFF 2023 (November) - **Project debut**: TraderX was presented at the [Open Source in Finance Forum 2023](https://events.linuxfoundation.org/open-source-finance-forum-new-york/) keynote demo session - **Initial architecture**: Established the core service mesh with Account, Position, Trade, and Reference Data services - **Multi-runtime support**: Java/Spring Boot, Node.js/NestJS, .NET Core, and Angular/React frontends ### Initial Contribution - **Database**: H2-based SQL database for accounts, trades, and positions - **Account Service** (Java/Spring Boot): Account management and validation - **Position Service** (Java/Spring Boot): Trade and position queries - **Trade Service** (Java/Spring Boot): Trade submission and validation - **Trade Processor** (Java/Spring Boot): Async trade processing via message feed - **Reference Data** (Node/NestJS): Security/ticker lookup service - **People Service** (.NET Core): User directory integration - **Trade Feed** (Node/Socket.IO): Pub-sub message bus for trade events - **Web Front-End**: Angular (full features) and React (trading/blotter) implementations --- ## Contributing to this page When making **major changes** to TraderX (new services, architectural shifts, significant feature additions), please add an entry here. Minor bug fixes and routine maintenance do not need to be recorded. --- ## Build & Run Guide There are several ways to run TraderX locally. Choose the method you're most comfortable with. ## Quick Start (Docker Compose) The easiest way to run the entire system: ```bash git clone https://github.com/finos/traderX.git cd traderX docker compose up ``` Once everything starts, the WebUI is accessible at http://localhost:8080. --- ## Run Options - [Docker Compose](#docker-compose) - Easiest, recommended for most users - [Kubernetes with Tilt](#kubernetes-with-tilt) - For K8s-native development - [Manual Run](#manual-run) - Run each service individually - [Corporate Environments](#corporate-environments) - Custom artifact repositories --- ## Docker Compose Docker Compose works on your local machine (tested on Mac Silicon) and in GitHub Codespaces. ### GitHub Codespaces For Codespaces, select an **8-core machine with 32GB RAM**: 1. Click the green **Code** button at the top of the repo 2. Select the **Codespaces** tab, click `...` → **New with options...** 3. Change machine type to **8-core** and click **Create codespace** > Personal GitHub accounts receive 120 free core hours per month. [Details](https://docs.github.com/en/billing/managing-billing-for-github-codespaces/about-billing-for-github-codespaces#monthly-included-storage-and-core-hours-for-personal-accounts) ### Running ```bash docker compose up ``` On first run, this builds all containers from project-specific Dockerfiles and starts them in sequence. Containers connect via a shared virtual network. **WebUI**: http://localhost:8080 (works with Codespaces too—localhost is mapped through). --- ## Kubernetes with Tilt Use [tilt.dev](https://tilt.dev) to deploy to a local Kubernetes cluster with hot-reload for local development. ### Prerequisites - [Docker Desktop](https://www.docker.com/products/docker-desktop/) or similar - A Kubernetes distribution: - [K8s with Docker Desktop](https://docs.docker.com/desktop/kubernetes/) - [Kind](https://kind.sigs.k8s.io/) - [Minikube](https://minikube.sigs.k8s.io/docs/start/) - [k3s](https://k3s.io/) - [Ingress Controller](https://kubernetes.github.io/ingress-nginx/deploy/) matching your K8s distribution - [tilt.dev](https://tilt.dev) ### Preflight Check ```bash kubectl get pods ``` You should see kube system pods and your ingress controller: ``` NAMESPACE NAME READY STATUS RESTARTS AGE ingress-nginx ingress-nginx-controller-7d4db76476-7wfl2 1/1 Running 0 38s kube-system coredns-7db6d8ff4d-ntdcr 1/1 Running 0 4h8m ... ``` ### Start Tilt ```bash cd ./gitops/local/ tilt up ``` Press **space** to open the Tilt web UI at http://localhost:10350/ > **Troubleshooting**: If you get `template engine not found for: up`, your OS may be running a Ruby library instead. Check the [tilt.dev installation instructions](https://docs.tilt.dev/install.html). ### Local Development with Tilt To build and deploy a service locally instead of using pre-built images, edit [gitops/local/Tiltfile](../gitops/local/Tiltfile) and uncomment the relevant line: ```python # Uncomment lines to use locally built version # docker_build('ghcr.io/finos/traderx/database', './../../database/.') # docker_build('ghcr.io/finos/traderx/account-service', './../../account-service/.') docker_build('ghcr.io/finos/traderx/position-service', './../../position-service/.') # ← uncommented # ... ``` ### Clean Up ```bash tilt down ``` --- ## Manual Run Run each service individually for maximum control. ### Environment Variables Export these port variables (or add to your shell profile): ```bash export DATABASE_TCP_PORT=18082 export DATABASE_PG_PORT=18083 export DATABASE_WEB_PORT=18084 export REFERENCE_DATA_SERVICE_PORT=18085 export TRADE_FEED_PORT=18086 export ACCOUNT_SERVICE_PORT=18088 export PEOPLE_SERVICE_PORT=18089 export POSITION_SERVICE_PORT=18090 export TRADE_PROCESSOR_SERVICE_PORT=18091 export TRADING_SERVICE_PORT=18092 export WEB_SERVICE_ANGULAR_PORT=18093 export WEB_SERVICE_REACT_PORT=18094 ``` ### Startup Sequence Start services in this order so dependencies are available: 1. `database` 2. `reference-data` 3. `trade-feed` 4. `people-service` 5. `account-service` 6. `position-service` 7. `trade-processor` 8. `trade-service` 9. `web-front-end` Each service has its own README with specific run instructions. --- ## Corporate Environments If you're behind a corporate artifact repository and need to override settings like `mavenCentral()` in Gradle: ### Local Gradle Override Create a `.corp` directory (git-ignored) with custom build scripts: ```bash mkdir .corp touch .corp/settings.gradle ``` Add to `.corp/settings.gradle`: ```groovy // Add your corporate repository overrides here rootProject.name = 'finos-traderX' includeFlat 'database' includeFlat 'account-service' includeFlat 'position-service' includeFlat 'trade-service' includeFlat 'trade-processor' ``` ### Build and Run ```bash # From traderX root gradle --settings-file .corp/settings.gradle build gradle --settings-file .corp/settings.gradle account-service:bootRun # Or from inside .corp directory cd .corp ./gradlew build ./gradlew account-service:bootRun ``` You can also store a custom Gradle wrapper here if your `distributionUrl` differs from the public one. --- ## API Explorer TraderX provides a standalone runtime API explorer mounted at `/api/docs`. The explorer is decoupled from the docs portal codebase and is generated into each runtime output by: - `pipeline/install-generated-api-explorer.sh` The explorer UI is static Swagger UI and the service list/spec URLs are metadata-driven from: - `catalog/state-catalog.json` → `apiCatalog` For states with message-bus subject maps (`006+`, with required pub/sub inspector baseline from `008+`), the same generated runtime catalog also carries inspector topic metadata derived from: - `specs//system/messaging-subject-map.md` → `/api/docs/catalog.json` → `messagingSubjects` ## Browse - Runtime route: `/api/docs` - Runtime catalog URL: `/api/docs/catalog.json` - Runtime static contracts: `/api/docs/contracts/*.yaml` - Runtime pub/sub inspector (canonical): `/api/docs/pubsub-inspector.html` (`/api/docs/pubsub-inspector` remains supported for compatibility) ## Usage Generate and start a state runtime: ```bash bash pipeline/generate-state.sh ``` For process-based states (`001`, `002`, `003`): ```bash ./scripts/start-base-uncontainerized-generated.sh --build-only ./scripts/start-base-uncontainerized-generated.sh # or for state 002/003: # ./scripts/start-state-002-edge-proxy-generated.sh --build-only # ./scripts/start-state-002-edge-proxy-generated.sh ``` For containerized/Kubernetes states (`004+`): ```bash ./scripts/start-state--generated.sh # optional restart without rebuild ./scripts/start-state--generated.sh --skip-build ``` Then open: - `http://localhost:18080/api/docs` for edge-proxy states (`002`, `003`) - `http://localhost:8080/api/docs` for ingress/containerized/Kubernetes states (`004+`) --- ## AWS EC2 Compose Host Prerequisites This guide documents host-level prerequisites for running generated TraderX deploy bundles on an EC2 host with an external NGINX reverse proxy. ## Scope - Applies to generated deployment bundles under `runtime/deploy/aws-ec2-compose/`. - Intended for containerized states that opt in to deploy bundles. - For Kubernetes runtime profile planning, see `/docs/spec-kit/aws-ec2-kubernetes-prerequisites`. ## Required Host Tools - `git` - `docker` engine - Docker Compose v2 plugin (`docker compose`) - `jq` - `curl` - `nginx` Recommended for TLS/public domains: - `certbot` (+ nginx plugin package) ## Generated Host Setup Scripts Every generated `aws-ec2-compose` deploy bundle should include: - `host-setup-check.sh`: verifies required host prerequisites and reports missing tools. - `host-setup-install.sh`: installs missing prerequisites using the host package manager (`apt-get`, `dnf`, or `yum`). Run check-only: ```bash ./runtime/deploy/aws-ec2-compose/host-setup-check.sh ``` Dry-run install plan: ```bash ./runtime/deploy/aws-ec2-compose/host-setup-install.sh --dry-run ``` Install prerequisites: ```bash ./runtime/deploy/aws-ec2-compose/host-setup-install.sh ``` ## Reverse Proxy For environments with an external NGINX in front of TraderX, use the generated snippet: - `runtime/deploy/aws-ec2-compose/nginx.reverse-proxy.snippet.conf` The snippet is state-aware and includes websocket mappings required by the emitted runtime transport for that state. ## Public Branch Clone Guidance Generated-state branches are public in the FINOS repository. Default deploy flow should clone without a GitHub token: ```bash git clone https://github.com/finos/traderX.git ``` Use authenticated clone URLs only for private forks/overlays that actually require credentials. --- ## AWS EC2 Kubernetes Host Prerequisites This document defines the host and cluster prerequisites for a future `aws-ec2-k8s` deployment profile. Status: - Profile contract defined in specs/docs. - Generation support not yet enabled. ## Scope - Intended for Kubernetes runtime lineage (`010+`) when deployment bundles are enabled for those states. - Complements `aws-ec2-compose` guidance and does not replace it. ## Required Host Tooling - `git` - `docker` (for local build/cache workflows where needed) - `kubectl` - `helm` - `jq` - `curl` - `nginx` (if using host-level edge TLS/termination) ## Required Cluster Capabilities - Kubernetes cluster reachable from host (single-node or multi-node). - Ingress controller installed and active. - Storage class available for stateful workloads. - Image pull path configured for emitted runtime images (local load, registry pull, or mirror). ## Planned Deploy Bundle Assets (`aws-ec2-k8s`) When this profile is enabled, generated bundles should include: - `runtime/deploy/aws-ec2-k8s/README.md` - `runtime/deploy/aws-ec2-k8s/deploy.sh` - `runtime/deploy/aws-ec2-k8s/upgrade.sh` - `runtime/deploy/aws-ec2-k8s/cleanup.sh` - `runtime/deploy/aws-ec2-k8s/host-setup-check.sh` - `runtime/deploy/aws-ec2-k8s/host-setup-install.sh` - `runtime/deploy/aws-ec2-k8s/ingress.values.yaml` (or equivalent) - `runtime/deploy/aws-ec2-k8s/smoke-test.sh` ## Enablement Rule Do not set Kubernetes state `deploy.enabled=true` with profile `aws-ec2-k8s` until: 1. Generator emits the profile assets above. 2. Generated deploy scripts support `--dry-run`. 3. Smoke tests validate ingress/API/UI/WebSocket behavior for the emitted state. --- ## Baseline vs Parity ## Baseline Baseline is the specification source of truth: - functional requirements - non-functional requirements - technical component matrix - API/event contracts - UI workflow requirements Generation should consume these specs. ## Parity Parity is behavioral equivalence verification against approved baseline contracts and flows. Use it for: - behavior comparison - drift detection - confidence checks while migrating to generated implementation Do not use source-copy hydration as generation input. ## Current Pipeline Spec-first generation: ```bash ./pipeline/validate-regeneration-readiness.sh ./pipeline/generate-from-spec.sh ``` Generation comparator (optional): ```bash bash pipeline/speckit/compare-all-component-generation.sh HEAD --allow-differences ``` --- ## Convergence States TraderX uses explicit convergence checkpoints to keep multi-track evolution understandable and maintainable. ## Why Convergence States Exist - They mark recommended jump-off points for new work. - They keep track progression explicit without forcing multi-parent branch ancestry. - They allow transition states to remain focused and incremental. Publish lineage remains single-parent via `previous`. Convergence context is represented separately via `isConvergence`, `convergenceLevel`, and `dottedParents`. ## Current Convergence Levels ### C0 - `004-containerized-compose-runtime` ### C1 - `007-observability-lgtm-compose` ### C2 - `009-order-management-matcher` ### C3 - `012-platform-convergence-c3` ## Contribution Policy - Prefer starting new state design from the nearest suitable convergence state. - If you modify a convergence state, update that state’s `system/convergence-rationale.md`. - Do not use dotted-line parents for non-convergence states. - Keep `previous` single-parent for publish lineage. ## CI + Artifact Publishing Policy Convergence states from `C0+` must include: - container build/publish workflow (`.github/workflows/build-and-publish.yml`) - generated run bundle artifacts for consuming GHCR images directly GHCR image namespaces are convergence-level scoped: - `C0`: `ghcr.io/finos/traderx-c0/` - `C1`: `ghcr.io/finos/traderx-c1/` - `C2`: `ghcr.io/finos/traderx-c2/` - `C3`: `ghcr.io/finos/traderx-c3/` Use immutable commit-SHA tags plus a moving `latest` tag per namespace/component. Reference: `/docs/spec-kit/generated-state-ci` Convergence level alone does not force a deployment profile. Deployment-bundle profile selection follows runtime substrate: - Compose runtime states -> `aws-ec2-compose` - Kubernetes runtime states -> `aws-ec2-k8s` (planned; enable only after generator support lands) ## Canonical Sources - State metadata: `catalog/state-catalog.json` - Convergence governance ADR: `/docs/adr/008-convergence-state-model` - Visual graph: `/docs/learning-paths` --- ## Corporate Environments Guide This guide defines how enterprises can extend TraderX privately without pushing corporate-specific changes upstream. Status: - This page is the high-level strategy overview. - Use [Customizing TraderX](/docs/spec-kit/customizing-traderx) as the primary entrypoint. - Use the two `custom-*` guides for implementation contracts and environment details. Detailed implementation contracts: - [Custom Overlay Architecture](/docs/spec-kit/custom-overlay-architecture) - [Custom Environments Guide](/docs/spec-kit/custom-environments-guide) ## Current Repository Baseline Current TraderX baseline is already set up with: - canonical authoring in `specs/**` and `.specify/**` - generated code publish model in `code/generated-state-*` branches - catalog-driven publish metadata in `catalog/state-catalog.json` - generation/publish automation in `pipeline/generate-state.sh` and `pipeline/publish-generated-state-branch.sh` Important current constraints: - a state must be present in `catalog/state-catalog.json` - `generation.mode` must be `implemented` - publish branch must match `code/generated-state-*` - working tree must be clean before publish ## Decision Use a separate corporate overlay repository as the runtime implementation model, and keep this TraderX repository upstream-canonical. Do not place corporate-only state definitions, transforms, secrets, proxy config, or private runtime assumptions into upstream TraderX. ## Recommended Corporate Repository Shape ```text traderx-corporate-overlay/ upstream/traderX/ # git submodule pinned to upstream commit corporate/catalog/state-catalog.json corporate/profiles/ corporate-internal.yaml public-dev.yaml corporate/states/ corp-001-internal-build/ corp-002-podman-runtime/ corporate/transforms/ corporate/runtime/ scripts/ docs/ ``` Bootstrap starter included in this repository: - canonical starter: `examples/custom-overlay-template/` ## State and Branch Policy Define three classes: 1. `mirrored-upstream`: generate and publish selected upstream states in your private remote. 2. `suppressed-upstream`: skip states that are not runnable or not approved for corporate runtime. 3. `internal-only`: generate additional corporate states (`corp-*`) only in the corporate repository. Example policy file in corporate overlay repo (`corporate/profiles/corporate-internal.yaml`): ```yaml profile: corporate-internal upstreamPin: main mirroredUpstreamStates: - 004-containerized-compose-runtime - 007-observability-lgtm-compose - 009-order-management-matcher suppressedUpstreamStates: - id: 010-kubernetes-runtime reason: "Cluster runtime not approved in this environment" internalStates: - id: corp-001-managed-postgres-runtime basedOn: 004-containerized-compose-runtime publishBranch: code/generated-state-corp-001-managed-postgres-runtime - id: corp-002-internal-docs-branding basedOn: corp-001-managed-postgres-runtime publishBranch: code/generated-state-corp-002-internal-docs-branding ``` ## Internal Docs Portal Strategy Corporate overlays should run an internal docs portal that: - clearly marks internal distribution with a persistent warning banner - shows only sanctioned internal learning-path branches - distinguishes mirrored upstream states from internal-only `corp-*` states - publishes a corporate learning graph derived from internal policy, not all public branches The bootstrap template includes: - `examples/custom-overlay-template/overlay/catalog/sanctioned-learning-graph.example.yaml` - `examples/custom-overlay-template/overlay/docs/internal-learning-graph.example.md` - `examples/custom-overlay-template/overlay/docs/internal-docs-portal.example.md` For docs-platform upgrade safety (Docusaurus/plugins/Webpack), follow: - `/docs/spec-kit/spec-kit-workflow#docusaurus-ci-compatibility` ## Overlay Learning Graph (required) Every corporate overlay implementation must maintain a learning-path index page at `docs/learning/index.md` (or an equivalent path documented in the overlay repository) that includes: - a Mermaid `flowchart TB` showing both upstream FINOS states used by the overlay and the overlay's own state chain - a state-to-artifact table directly beneath the diagram with these required columns: `State | Learning Guide | Spec Pack | Generated Branch | Diff vs Previous | Runnable` Required node classes and semantics: - `upstream`: FINOS canonical state shown for context; not owned by the overlay - `anchor`: overlay baseline that mirrors an upstream state but adds no runnable changes - `runnable`: first overlay state that is fully runnable as a standalone stack - `feature`: subsequent overlay states that add a capability on top of the prior state Required upstream styling and link behavior: - upstream FINOS nodes must use class `upstream` - class `upstream` must be defined as: `fill:#fff8e1,stroke:#f57f17,stroke-width:1px,stroke-dasharray:5 5` - every upstream node must include a Mermaid `click` target to the public FINOS learning guide for that state Convergence rule: - when an overlay state intentionally converges with (matches functional surface of) an upstream FINOS state, the diagram must include a dotted convergence edge with this syntax: `UpstreamNode -.->|converges| OverlayNode` The overlay template includes a starter example: - `examples/custom-overlay-template/overlay/catalog/learning-graph-diagram.example.md` ## Upstream Sync Workflow 1. Update submodule pin to a tested upstream commit. 2. Regenerate only profile-approved states. 3. Publish private generated-state branches to the corporate remote. 4. Run corporate validation gates. 5. Promote the new pin only after generation + runtime validation succeeds. Bootstrap commands: ```bash mkdir traderx-corporate-overlay cd traderx-corporate-overlay git init git submodule add -b main https://github.com/finos/traderX.git upstream/traderX git submodule update --init --recursive ``` Refresh upstream pin: ```bash git -C upstream/traderX fetch origin git -C upstream/traderX checkout main git -C upstream/traderX pull --ff-only git add upstream/traderX git commit -m "chore: bump TraderX upstream pin" ``` Generate from upstream submodule directly into corporate overlay output root: ```bash TRADERX_GENERATED_ROOT=/path/to/traderx-corporate-overlay/generated \ bash upstream/traderX/pipeline/generate-state.sh 004-containerized-compose-runtime ``` Generated state output includes a local runtime harness at: - `/path/to/traderx-corporate-overlay/generated/code/target-generated/scripts` - `/path/to/traderx-corporate-overlay/generated/code/target-generated/RUN_FROM_GENERATED.md` This lets corporate overlays run start/stop/status from generated artifacts directly, while upstream root scripts can forward to those local harness scripts. ## Generation Rules For Agents and Humans - Never modify `upstream/traderX` for corporate-only requirements. - Never hand-edit generated runtime outputs. - Encode persistent corporate deltas as profile/state/transform artifacts in the corporate repo. - Preserve explicit lineage (`previous`) for every corporate state. - Keep generation reproducible from pinned upstream commit + corporate artifacts. ## Why This Fits TraderX Today - Upstream already cleanly separates canonical specs from published generated branches. - Current publish tooling already enforces branch naming and implemented-state controls. - Corporate suppression/addition is better represented as a profile-level policy in a separate repo, not as upstream catalog edits. ## Transitional Guidance Inside Upstream TraderX This document is the upstream reference contract. Actual corporate runtime implementation should live outside this repository. --- ## Custom Environments Guide This guide defines implementation norms for custom TraderX overlays that run in controlled or non-default operating environments. ## Package Management Norms Define package management norms before implementing overlay-specific state changes. The first validation target is the initial known-good upstream baseline generated state. If baseline build does not pass under your norms, do not proceed to custom states yet. ### Upfront decisions Choose and document one consistent approach per package manager: - internal artifact proxy/repository manager - local pre-populated package cache - setup script that writes registry and credential files - a combination of the above There is no single required approach, but one approach must be selected, documented, and enforced consistently. ### Java (Gradle) - Decide whether to use each component's `gradlew` wrapper or an externally provided Gradle runtime. - If wrappers are used, ensure `gradle/wrapper/gradle-wrapper.properties` distribution URL is reachable in your environment. - If your environment injects repository config through `settings.gradle`, explicitly set `dependencyResolutionManagement` to `PREFER_SETTINGS` when needed so project-level repository declarations do not silently override settings-level policy. - Decide whether a setup script must write credentials or init files (for example `.gradle/gradle.properties`, `.gradle/init.gradle`, `.m2/settings.xml`) and document this requirement prominently. ### Node.js (npm/yarn) - Define the registry endpoint. - Define authentication mechanism and when tokens are injected. - Ensure `.npmrc` (or equivalent) is written before install commands run. - Decide whether this is done by setup script, CI injection, or component transform. ### .NET (NuGet) - Define the feed endpoint (v3 feed URL). - If suppressing public feeds, include `` in `NuGet.Config` under `` before declaring internal sources. - Decide whether config is setup-script generated or transform-injected. ### Credential isolation Credential files should be written under an isolated home directory for generated runtime sessions, not the user's primary home directory. Recommended pattern: - set `HOME` to a run-local tool cache path before package commands execute - create credential files in that isolated path - avoid cross-run credential bleed ### Future scope: container and binary package management When custom states add containerized components, extend package norms to container image sourcing and binary package installers used inside image builds. ### Required pre-overlay checklist - [ ] Chosen package management approach documented (proxy, cache, setup script, or other) - [ ] Gradle wrapper URL verified reachable or re-pointed to reachable mirror - [ ] Gradle repository resolution mode confirmed (`PREFER_SETTINGS` where required) - [ ] npm registry and auth mechanism defined and tested - [ ] NuGet feed and `` requirement confirmed - [ ] Credential isolation approach defined (isolated `HOME` or equivalent) - [ ] Baseline upstream state confirmed to build cleanly under these norms ## Runtime Version Management Do not assume runtime tools are pre-installed in the expected versions. Define runtime provisioning before writing transforms, start scripts, or smoke tests. ### Questions to answer first - How do developers get the required Java version on `PATH`? - How do developers get the required Node.js version on `PATH`? - How do developers get the required .NET SDK version on `PATH`? - How are auxiliary tools (`python`, `jq`, etc.) provided? - Is provisioning done through login profile, sourced script, module loader, container runtime, or manual install? ### Env-loader contract Create one shared loader file (for example `overlay/runtime/env-loader.sh`) and source it from every start, stop, smoke-test, and pipeline script. The loader must: - be idempotent - export runtime tool paths to child processes - pin specific versions rather than floating defaults - print active runtime versions for troubleshooting If a required non-default tool is missing, fail early with a clear error. Avoid hidden failures from missing utilities. ### Future scope: container runtime management If custom states add containerization, define container runtime and orchestration requirements with the same upfront discipline (runtime, version, and activation mechanism). ## Pub/Sub Replacement When replacing Socket.IO + trade-feed with another messaging approach, define behavior at architecture/spec level before wiring application code. ### Connection Lifecycle Determine whether the client connection model is synchronous or asynchronous. If asynchronous: - register connection/disconnection listeners before connect - subscribe or publish only in connected callback - reset subscription state on disconnect so reconnect can re-subscribe If client behavior is unclear, build a minimal publisher/subscriber test first. Reference pattern: ```text client.onConnected(() -> { client.subscribe(topic, messageHandler); }); client.onDisconnected(() -> { subscriptionActive = false; }); client.connect(); ``` ### Message Delivery Semantics Determine subscription behavior before implementing loops: - does subscribe block for messages, or return after subscription acknowledgement? - are messages push callback, poll/pull, or blocking read? - what acknowledgement mode is required? - who owns reconnect and re-subscription behavior, client or application? For callback-driven clients, subscribe once and process via handler callback. Do not subscribe in a loop. ### Payload Format Strategy Define payload formats per path: - backend-to-backend format - backend-to-browser format - any translation boundary if formats differ JSON remains the lowest-friction browser format, but backend messaging format may differ if documented and consistently implemented. ### Internal Library Dependency Completeness Libraries from internal repositories may omit transitive runtime dependencies. If runtime class/module errors appear from library internals, inspect client-library documentation/examples and add explicit dependencies to your build. ### Browser Connectivity Strategy Choose one strategy before implementation: - native WebSocket endpoint exposed by chosen messaging system - dedicated WebSocket gateway translating browser traffic to broker protocol If a gateway is required, model it as a first-class component with explicit port, start command, and health checks. Frontend service APIs should remain stable to callers where possible (for example, preserve `subscribe`/`unsubscribe` service contracts while changing internals). ### Port Allocation Replacement messaging systems may require multiple ports. For each custom state, explicitly catalog: - broker listener ports - browser/WebSocket listener ports - management/admin ports Update smoke tests for each required endpoint. ## Smoke Test Health Hint Guidance Health endpoints vary by component and implementation. Do not assume `/health` or `/actuator/health` is universally available. Use `catalog/component-spec.csv` `health_hint` as canonical guidance: - HTTP services can be checked via documented data or API docs endpoints. - TCP-only services should use TCP reachability checks (`nc -z` or equivalent), not HTTP probes. ## LLM-Friendly Docs Quality (AFDocs) For custom overlays that publish internal documentation portals, run AFDocs checks as part of documentation maintenance. Local preview check (while Docusaurus is running locally): ```bash npx afdocs check http://localhost: --format scorecard ``` Machine-readable scoring output: ```bash npx afdocs check http://localhost: --format json --score ``` Published-site check: ```bash npx afdocs check https://docs.example.com --format scorecard ``` Recommended workflow: 1. Run scorecard to identify highest-impact failing categories. 2. Use JSON output for automation and trend tracking. 3. Prioritize fixes that can be solved at docs pipeline/plugin level before editing large amounts of page content manually. Optional enhancement: - Generate `/llms.txt` and markdown mirrors using a Docusaurus llms plugin in the docs build pipeline, then re-run AFDocs to confirm score improvements. --- ## Custom Overlay Architecture This document defines the upstream reference contract for implementing a custom TraderX overlay repository without modifying upstream TraderX state definitions or generated-state publishing behavior. ## Goals - Keep upstream TraderX canonical and unchanged. - Allow environment-specific extensions in a separate repository. - Preserve reproducible generation and clean state lineage. - Keep publish semantics aligned with upstream generated-state contracts. ## Canonical Overlay Repository Structure ```text traderx-overlay/ upstream/traderX/ # upstream as git submodule overlay/ profiles/ # policy YAMLs (state selection) states/custom-/ # per-state: scripts/, catalog/, src/ transforms/ # named transform scripts runtime/ # shared env loaders, setup helpers catalog/ state-catalog.json # overlay state catalog pipeline/ generate-state.sh # mirrors upstream pipeline/generate-state.sh publish-state-branch.sh # mirrors upstream publish script specs/ # overlay spec packs (one per custom state) docs/ # overlay documentation ``` ## Required Overlay Repository Artifacts An overlay repository must include these required deliverables: - `catalog/state-catalog.json` - overlay state catalog with explicit lineage and publish metadata - `specs/` - overlay spec packs for overlay-owned states - `docs/learning/index.md` - overlay learning-path index; must contain the Mermaid convergence graph and state-to-artifact table per `/docs/spec-kit/corporate-environments-guide#overlay-learning-graph-required` ## Overlay State Catalog Contract Overlay state catalogs should keep state ids explicit, preserve lineage, and declare generation/publish metadata as first-class fields. Required fields per state entry: - `id` - `title` - `previous` - `generation.mode` - `publish.branch` - `featurePack` - `overlayState` Example: ```json { "id": "custom-001-example", "title": "Custom State 001 - Example", "previous": ["004-containerized-compose-runtime"], "generation": { "mode": "implemented" }, "publish": { "branch": "code/generated-state-custom-001-example" }, "featurePack": "specs/custom-001-example", "overlayState": { "kind": "custom", "extends": "004-containerized-compose-runtime" } } ``` Expected semantics: - `previous` expresses branch ancestry and overlay lineage. - `overlayState.extends` identifies the upstream or overlay state being extended. - `publish.branch` follows the same generated-state branch naming convention used upstream. ## Pipeline Responsibilities `pipeline/generate-state.sh` in an overlay should: 1. Load runtime/tooling environment. 2. Resolve the requested overlay state from overlay catalog metadata. 3. Generate upstream base output into an overlay-controlled output root. 4. Apply overlay transforms after layout copy. 5. Run optional validation/smoke checks. `pipeline/publish-state-branch.sh` in an overlay should: 1. Resolve base branch from `previous`. 2. Reset target generated-state branch to base with `git checkout -B`. 3. Commit full generated snapshot once. 4. Force-push (`git push --force`) to preserve one-commit invariant. ## State Inheritance Policy (Overlay + Upstream) Overlay states follow the same inheritance rule as upstream states: - Inherit all behavior from the base (`previous` / `overlayState.extends`) by default. - Diverge only when the overlay state spec explicitly declares a conflict, replacement, or deprecation. - Keep inherited capability checks in downstream smoke tests so runtime transitions do not silently drop prior behavior. ## Transform Script Contract Transform scripts must follow this contract: - Naming: `apply-.sh ` - Input: first positional argument is target generated directory. - Idempotence: safe to run repeatedly against the same target directory. - Failure behavior: exit non-zero on any failure. - Ordering: transforms must run after layout copy (never before). ### Critical Ordering Rule The layout function (`prepare_generated_base_layout` or equivalent) performs a destructive copy of base components into the target tree. Any transform output applied before that copy will be overwritten. Correct order: ```bash # Step 1: copy base components into target layout (destructive) prepare_generated_base_layout # Step 2: re-apply overlay transforms after layout bash "${OVERLAY_ROOT}/overlay/transforms/apply-.sh" "${TARGET}" ``` If transforms run before layout copy, the system can start successfully but run unmodified upstream code. ## `--dry-run` Contract For Start Scripts Custom start scripts used by generation must support `--dry-run`. Behavior requirements: - Build target layout fully (copy/sync all required components and runtime files). - Apply all transforms and generated overlays. - Exit without starting service processes. - Return non-zero if layout preparation fails. Generation pipelines rely on this mode to produce clean publishable snapshots. ## Runtime Version Management Before implementing transforms or start scripts, define and document how required tools become available in your environment. Required questions: - How is Java made available on `PATH`, at a pinned version? - How is Node.js made available on `PATH`, at a pinned version? - How is .NET SDK made available on `PATH`, at a pinned version? - How are additional tools (for example Python, `jq`) made available? - Is tool activation provided by login profile, sourced script, module loader, containers, or another mechanism? Implement a single shared loader, typically `overlay/runtime/env-loader.sh`, and source it from every start/stop script, smoke test, and pipeline entrypoint. The loader must: - Be idempotent (safe to source multiple times). - Export runtime tool paths for child processes. - Pin explicit versions. - Emit active version diagnostics for debugging. If a required tool is not globally installed, the loader must provide it through the chosen environment mechanism and fail clearly if not available. Future containerized states should apply the same pattern to container runtime and orchestration tool initialization. ## Working Directory Anchor Rule Any start/generate script that deletes and recreates directories must anchor current working directory to a safe location before destructive operations. ```bash # Anchor CWD to overlay root before destructive operations cd "${OVERLAY_ROOT}" prepare_generated_base_layout ``` If the shell is currently inside a directory being deleted, subsequent runtime commands can fail with misleading errors. ## `TRADERX_GENERATED_ROOT` Convention Overlay pipelines should use `TRADERX_GENERATED_ROOT` to direct upstream generation output into the overlay repository. Example: ```bash TRADERX_GENERATED_ROOT="${OVERLAY_ROOT}/generated" \ bash "${OVERLAY_ROOT}/upstream/traderX/pipeline/generate-state.sh" 004-containerized-compose-runtime ``` Default behavior remains unchanged when `TRADERX_GENERATED_ROOT` is unset (`upstream/traderX/generated`). ## LLM-Friendly Docs Output (Optional) If your overlay publishes a docs portal, optionally add an llms output generator plugin to your Docusaurus build so `/llms.txt` and related LLM-friendly artifacts are generated automatically. Implementation guidance: 1. Add a maintained Docusaurus llms plugin in website dependencies. 2. Configure plugin output paths and URL handling to match your site base path. 3. Regenerate docs and run AFDocs checks (`scorecard` and JSON score output). 4. Keep this as a build-time concern in docs pipeline/configuration rather than manual per-page edits. --- ## Customizing TraderX Use this page as the single entrypoint for building and operating a private overlay on top of upstream TraderX. ## Which Document Does What 1. [Corporate Environments Guide](/docs/spec-kit/corporate-environments-guide) - Strategy overview and decision model. - Explains when to use an external overlay repository and why upstream stays canonical. 2. [Custom Overlay Architecture](/docs/spec-kit/custom-overlay-architecture) - Repository layout, state catalog contract, transform ordering, `--dry-run`, working-directory anchor rules, and `TRADERX_GENERATED_ROOT`. 3. [Custom Environments Guide](/docs/spec-kit/custom-environments-guide) - Environment integration norms: package management, runtime/toolchain loading, pub/sub replacement, and health-check guidance. ## Recommended Implementation Order 1. Confirm the overlay model and governance decisions in the strategy guide. 2. Create a separate overlay repository and pin upstream TraderX as a submodule. 3. Define runtime/toolchain loading (`env-loader`) and package management norms up front. 4. Generate the first upstream baseline state into overlay output (`TRADERX_GENERATED_ROOT`). 5. Confirm baseline builds successfully under your environment constraints. 6. Implement idempotent overlay transforms and apply them after layout copy. 7. Add overlay-specific states and publish generated-state branches using the one-commit invariant. 8. Maintain alignment by regularly syncing upstream pin, regenerating, and validating. 9. Run AFDocs checks against local and published docs URLs to keep the portal agent-friendly. ## Templates - Canonical starter template: `examples/custom-overlay-template/` Recommended usage: 1. Start from `examples/custom-overlay-template/`. 2. Tailor the included profile, transform, and internal-docs examples for your environment policy. ## Compatibility Note `/docs/spec-kit/corporate-environments-guide` remains a valid URL and should be treated as the high-level overview, while the two `custom-*` guides are the detailed implementation references. --- ## Generated State Branches TraderX keeps the canonical source of truth in `specs/**` + `.specify/**`, and publishes code-only snapshots in dedicated `code/generated-state-*` branches. ## Canonical State Definitions Each state is defined in two places: 1. SpecKit feature pack: `specs/NNN-/` 2. State catalog entry: `catalog/state-catalog.json` The state catalog is the publish contract for generated snapshots: - state id/title/status - predecessor states (`previous`) - convergence metadata (`isConvergence`, `convergenceLevel`, `dottedParents`, `primaryLineageRole`) - generation readiness (`generation.mode`) - default generated-state branch name - release tag hint - optional deployment-bundle metadata (`deploy.enabled`, `deploy.profile`, environment labels/domain hints) Live demo environment mappings are tracked separately in: - `catalog/live-environments.json` - `/docs/spec-kit/live-environments` ## State Independence + Lineage Each state must be buildable from its own feature pack without requiring branch-local edits. Publish lineage is explicit through `previous` in `catalog/state-catalog.json`. Dotted-line parents are documentation lineage only and are not used for branch ancestry. Generated branches include metadata files so consumers can always see provenance: - `STATE.md` - `.traderx-state/state.json` - `LEARNING.md` - `FUNCTIONAL_TESTING.md` - `docs/learning/*` generated learning artifacts (component list, system design, architecture, diagram) - optional `runtime/deploy/*` deployment bundle artifacts for containerized states that explicitly opt in These files record: - current state id/title - prior states and known next states - source branch/commit used to generate snapshot - generation timestamp - state-oriented learning links back to canonical docs/specs - plain-English functional behavior expectations plus smoke-test entrypoints for that snapshot Canonical portal learning guides live at: - `/docs/learning` ## Publish The Baseline Generated Branch From a clean working tree: ```bash bash pipeline/publish-generated-state-branch.sh 001-baseline-uncontainerized-parity --push ``` By default, publish now runs a prepublish CI-parity gate before commit/push (`pipeline/prepublish-generated-state-gate.sh`) against generated outputs. Use `--skip-prepublish-gate` only for explicit operator override cases. When adding or updating a state, publish the lineage neighborhood so adjacent branches get refreshed lineage/compare links: ```bash bash pipeline/publish-generated-state-neighborhood.sh --push ``` Default branch target for baseline: - `code/generated-state-001-baseline-uncontainerized-parity` State `002-edge-proxy-uncontainerized` now uses: - generation: `bash pipeline/generate-state.sh 002-edge-proxy-uncontainerized` - runtime (first run/build): `./scripts/start-state-002-edge-proxy-generated.sh --build-only` - runtime (start after build): `./scripts/start-state-002-edge-proxy-generated.sh` - publish branch: `code/generated-state-002-edge-proxy-uncontainerized` Publish branch snapshot: ```bash bash pipeline/publish-generated-state-branch.sh 002-edge-proxy-uncontainerized --push ``` State `004-containerized-compose-runtime` now uses: - generation: `bash pipeline/generate-state.sh 004-containerized-compose-runtime` - runtime (default build + start): `./scripts/start-state-004-containerized-generated.sh` (NGINX ingress on `http://localhost:8080`) - runtime (restart without image rebuild): `./scripts/start-state-004-containerized-generated.sh --skip-build` - publish branch: `code/generated-state-004-containerized-compose-runtime` Publish branch snapshot: ```bash bash pipeline/publish-generated-state-branch.sh 004-containerized-compose-runtime --push ``` Current published generated branches: - `code/generated-state-001-baseline-uncontainerized-parity` - `code/generated-state-002-edge-proxy-uncontainerized` - `code/generated-state-004-containerized-compose-runtime` ## Branch Invariant -- One Commit Per State Every `code/generated-state-` branch, in both upstream and any custom overlay, must contain exactly one content commit on top of its base branch. How this works: 1. The publish script always starts from the base branch (the previous state's branch, or the root anchor), not from the existing generated branch tip. 2. The publish script creates or resets the generated branch to the base tip using `git checkout -B`. 3. The full generated snapshot is applied as a single commit. 4. The publish script force-pushes that branch (`git push --force`) to replace the prior snapshot commit. Why this matters: - `git diff code/generated-state-A code/generated-state-B` stays a clean state-to-state diff. - Lineage is expressed through branch ancestry (the base branch), not through cumulative commit depth. - History remains readable and reproducible for both humans and automation. Custom overlay pipelines must enforce the same invariant. Using `--force-with-lease` on a branch that accumulates commits does not satisfy this model. ## Behavior Invariant -- Inherit Unless Explicitly Replaced Across upstream and custom overlays, each state must preserve parent behavior by default. Required rule: 1. Downstream states inherit all prior behavior from `previous` lineage unless the state spec explicitly declares a conflict, replacement, or deprecation. 2. Runtime migrations (for example Compose to Kubernetes) must carry inherited capabilities forward unless an explicit requirement says otherwise. 3. Smoke tests for downstream states should include inherited capability checks so regressions are detected during publish. ## CI Invariant -- Required Generated Workflows Generated-state branches must carry CI workflows that match the state's technology surface. - States `002+` must include security and license scanning workflows. - Coverage must include all applicable generated components (Java/.NET/Node/containerized). - CVE suppression files used by the scans must be present and versioned with the generated state. Convergence states `C0+` must also include container image build/publish workflows and a generated run bundle that references published images. Canonical policy and local preflight guidance: - `/docs/spec-kit/generated-state-ci` ## Template Invariant -- Source Of Truth Lives In Templates/Patchsets Dependency/runtime version changes must be maintained in canonical generator sources, not as post-generation edits. Required rule: 1. Baseline component defaults are maintained in `templates/**` (for example Java `build.gradle`, Gradle wrapper, and shared runtime/library versions). 2. Derived state deviations are maintained in state patchsets under `specs//generation/patches/*.patch`. 3. Ad hoc post-generation mutation scripts are not allowed in steady-state generation; canonical templates, patchsets, and catalog-backed policy sync steps must define output content. 4. Version targets are declared in `catalog/dependency-version-targets.json` and must pass before publish. New shared dependency pins belong in this catalog first, including npm override pins; template npm overrides that are not cataloged fail validation. npm override targets are materialized during generation for modules whose lockfiles resolve the target package. ```bash bash pipeline/smoke-dependency-version-targets.sh bash pipeline/validate-template-version-consistency.sh bash pipeline/validate-generated-dependency-targets.sh generated/code/components generated/code/target-generated bash pipeline/validate-generated-branch-dependency-consistency.sh ``` ## How To Add A New State 1. Scaffold the feature pack: ```bash bash pipeline/scaffold-state-pack.sh --title "" --previous <prior-state-id> --track <prelude|baseline|architecture|nonfunctional|functional|devex> ``` 2. Add/update requirements, plan, tasks, contracts, and traceability for the new state. 3. Confirm state entry in `catalog/state-catalog.json` with: - `previous` lineage - publish branch/tag conventions - `generation.mode=planned` until implemented 4. Implement state generation pipeline and change `generation.mode` to `implemented`. 5. Publish generated branch + tag with validation evidence. Recommended publish sequence after implementing a new state: 1. `bash pipeline/publish-generated-state-neighborhood.sh <state-id> --push` 2. `bash pipeline/publish-generated-state-tree.sh --push` (optional full refresh) This keeps specification and generated code distribution clearly separated while preserving state-to-state continuity. --- ## Generated State CI Policy This document defines the CI contract for generated TraderX code branches. ## Scope - Security and license CI is required for all generated states from `002+`. - States `001-003` are pre-container states and should not carry Docker/Compose artifacts in published generated snapshots. - Container image build validation CI starts at state `004+` for states that include buildable container images. - Container image build/publish CI is required only for convergence states from `C0+`. ## Required CI For States `002+` Each generated `code/generated-state-*` branch must include CI workflows for dependency security and license scanning: - `.github/workflows/security.yml` - `.github/workflows/license-scanning-node.yml` (required when the state contains Node.js components) These workflows must cover every applicable component in the generated state: - Java/Gradle components in Gradle scan jobs - .NET components in .NET scan jobs - Node.js components in Node scan and license jobs - containerized components in container vulnerability scan jobs Coverage must be derived from generated component manifests/catalog data, not a manually maintained static list. ## Node License Requirement All generated Node.js projects must declare: ```json "license": "Apache-2.0" ``` Policy: - Any new Node.js project introduced by a state transition must include this license field in its generated `package.json`. - License scanning workflows should fail when a generated Node.js project omits this declaration. ## Node Lockfile Generation Policy Node lockfiles are source-controlled generated artifacts and part of state lineage. Policy: - Generated Node projects must include a `package-lock.json`. - Lockfiles must be refreshed only when the corresponding `package.json` changes, or when lockfiles are missing/invalid. - State patchsets should include lockfile deltas when state changes affect Node manifests. - Generated-branch publish is blocked if Node manifests and lockfiles are inconsistent. ## Hermetic Default Test Policy Default build/test CI must not depend on external runtime services. Policy: - Generated branch default build/test jobs must run hermetically on CI runners without requiring external databases, brokers, or network endpoints. - Database-backed tests in default suites must use in-memory/embedded engines (or equivalent isolated local test engines). - External runtime validation (for example real PostgreSQL or broker integration) must run in explicitly named integration profiles/jobs separate from default build/test gates. - State deltas introducing new database-backed modules must include test profile/config updates that preserve this policy. ## CVE Suppression Contract Generated branches must carry the CVE suppression files used by security scanning: - `.github/gradle-cve-ignore-list.xml` - `.github/node-cve-ignore-list.xml` - `.github/dotnet-cve-ignore-list.xml` Policy: - Keep a baseline suppression set shared across generated states. - Allow state-specific suppression deltas when dependency sets diverge. - When a state changes dependencies, update scanner coverage and suppression files in the same change. ## Dependency Source-Of-Truth Contract Version policy changes must be landed in generator sources, not as ad-hoc generated-output edits. - Baseline runtime/dependency versions belong in `templates/**`. - State-specific version deltas belong in `specs/<state>/generation/patches/*.patch`. - Post-generation mutation scripts are not allowed in steady-state; generation must be reproducible directly from templates and state patchsets. ## Messaging Subject Map Contract Messaging states must keep a canonical subject registry in-state. Policy: - States `006+` must include `system/messaging-subject-map.md`. - The file must be cumulative for that state (all active subject families). - Subject-map format requirements are defined in `docs/spec-kit/messaging-subject-map-standard.md`. - Generated-state contract validation must fail when order-matcher states omit required order subjects. ## External Dependency Pin Contract (Sail / State 014) State `014-fdc3-intent-interoperability` depends on a local Sail sidecar and must use explicit pin governance. Policy: - Sail pin metadata is state-owned in `specs/014-fdc3-intent-interoperability/generation/sail-pin.env`. - Pin metadata must include repo URL, tracking ref, pinned commit SHA, and updated-on date. - Generated state `014` artifacts must consume that pin (`sail/bootstrap/sail-pin.env` + bootstrap defaults). - Root quality gates must validate pin-manifest contract (`bash pipeline/validate-sail-pin-contract.sh`). - Maintenance workflows must run drift detection (`bash pipeline/check-sail-pin-drift.sh --fail-on-drift`) before repinning. ## Jump-Point Overlay Parity Contract When a state first materializes full runtime files that no longer point directly at `templates/**` (for example, root-level module `build.gradle` files in a convergence jump-point), that state must preserve inherited base-template policy unless an explicit state requirement overrides it. Policy: - Generic dependency/security/runtime controls remain owned by base templates. - Jump-point state patchsets must not silently drop inherited template controls (dependency pins, exclusion blocks, plugin/runtime version guards). - If a base template policy changes, maintainers must re-evaluate the nearest jump-point state patchset and propagate required parity updates. - Any intentional divergence must be explicitly documented in the state FR/NFR deltas. ## Gradle Wrapper Ownership Policy Gradle wrapper assets are baseline template artifacts, not state patch artifacts. Policy: - Canonical wrapper files live in `templates/gradle-wrapper/`: - `gradlew` - `gradlew.bat` - `gradle/wrapper/gradle-wrapper.jar` - `gradle/wrapper/gradle-wrapper.properties` - Generated Gradle modules must receive wrapper assets from this canonical baseline during generation. - State patchsets must not carry wrapper binary/script/properties diffs; wrapper version updates are made once in templates and inherited by all states. ## Patch Hygiene Ownership Policy State patchsets must contain authored state deltas only, not local build byproducts. Policy: - Do not include compiled/restored output directories in state patch payloads: - `.gradle/**` - `build/**` - `target/**` - `bin/**` - `obj/**` - `dist/**` - `coverage/**` - `node_modules/**` - Keep dependency lockfiles consistent with their manifests; include lockfile deltas in patchsets when manifests change. - Build/restored artifacts must be produced by generation/runtime/build commands, never maintained by state patch files. Preflight gates: ```bash bash pipeline/validate-template-version-consistency.sh bash pipeline/validate-generated-dependency-targets.sh generated/code/components generated/code/target-generated # verifies dependency key versions are consistent across generated state branches bash pipeline/validate-generated-branch-dependency-consistency.sh ``` ## Convergence CI For `C0+` Each convergence state from `C0+` must include: - `.github/workflows/build-and-publish.yml` The workflow must build and publish all containerized components present in that convergence state to GHCR. ## Non-Convergence Container CI For non-convergence states that include containerized components, generated branches must include: - `.github/workflows/build-container-images.yml` Policy: - Build all detected container images for validation. - Do **not** publish images to GHCR. - Do not generate GHCR runtime bundles for non-convergence states. - This policy applies from state `004+` onward; pre-container states (`001-003`) must not emit container build workflows. ## GHCR Namespace Policy For Convergence States Use convergence-level namespaces (not numeric state ids): - `C0`: `ghcr.io/finos/traderx-c0/<component>` - `C1`: `ghcr.io/finos/traderx-c1/<component>` - `C2`: `ghcr.io/finos/traderx-c2/<component>` - `C3`: `ghcr.io/finos/traderx-c3/<component>` Each image publish must include: - immutable tag: commit SHA - moving tag: `latest` ## State Run Bundle Requirement For `C0+` Each convergence state from `C0+` must publish a generated run bundle for consuming GHCR images without local source builds. Minimum bundle artifacts: - runtime manifest(s) that reference published GHCR images - `.env.example` or equivalent runtime config template - state run README with copyable startup commands ## Deployment Bundle Contract For Containerized Demo States Generated snapshots may include deployment bundles only for containerized states that explicitly opt in via state catalog metadata. Policy: - Pre-container states (`001-003`) MUST NOT emit deployment bundles. - Deployment bundle assets, when enabled, live under `runtime/deploy/<profile>/`. - Initial profile contract is `aws-ec2-compose` for compose-based demo rollout. - Deployment scripts MUST support `--dry-run` and use env vars for environment-specific values (for example target branch, domain/FQDN, image tag). - Deployment scripts MUST NOT contain embedded credentials/tokens/secrets. - Deployment bundles MUST include host-prerequisite setup assets (`host-setup-check.sh` and `host-setup-install.sh`) for the emitted profile. - Default generated deploy-clone instructions MUST use tokenless public repository URLs; token-authenticated clone guidance should be optional and only for private forks. - Deployment bundle generation must come from canonical templates/generator logic, not manual edits on generated-state branches. Profile matrix: - `aws-ec2-compose` (active): compose-runtime states. - `aws-ec2-k8s` (planned): Kubernetes-runtime states (`010+`) once profile-generation support lands. Recommended minimum deployment bundle assets: - `runtime/deploy/<profile>/README.md` - `runtime/deploy/<profile>/deploy.sh` - `runtime/deploy/<profile>/upgrade.sh` - `runtime/deploy/<profile>/cleanup.sh` - `runtime/deploy/<profile>/host-setup-check.sh` - `runtime/deploy/<profile>/host-setup-install.sh` - `runtime/deploy/<profile>/nginx.reverse-proxy.snippet.conf` Reverse-proxy snippet contract: - Must include websocket upgrade headers (`Upgrade`, `Connection`, HTTP/1.1) for websocket routes. - Must include websocket path mappings required by runtime messaging transport for the emitted state (for example `/nats-ws` for NATS-enabled state lineage), and avoid transport-specific stale routes that are no longer active for that state. Planned Kubernetes deploy profile contract (`aws-ec2-k8s`): - Must emit Kubernetes-specific deploy assets (for example ingress values/templates and cluster smoke checks), not Compose-only scripts. - Must include host setup check/install scripts for Kubernetes operator dependencies (`kubectl`, `helm`, and cluster access checks). - Must include dry-run deploy/upgrade/cleanup behavior consistent with compose profile expectations. - Must remain disabled in catalog (`deploy.enabled=false`) until generator support and profile smoke tests are implemented. ## Local CI Preflight Before Publish Generated branch publish runs a prepublish CI-parity gate by default via `pipeline/publish-generated-state-branch.sh`. Run the same gate manually when validating outside publish flow: ```bash bash pipeline/prepublish-generated-state-gate.sh <state-id> ``` The gate enforces: - generated-state contract validation - compile preflight for Node/Gradle/.NET modules - dependency target/version propagation checks - UI status metadata + lineage invariants - Node license scanning - container image buildability checks - dependency CVE scanning with suppression rules Emergency override remains explicit and operator-invoked: ```bash bash pipeline/publish-generated-state-branch.sh <state-id> --skip-prepublish-gate ``` ## Publish Gate Generated branch publish is blocked unless: - required CI workflows are present for the target state, - scanner/build matrices include all applicable components, - required suppression files are present and valid, - prepublish CI-parity gate checks pass. --- ## Getting Started with TraderX TraderX has multiple generated codebases representing different learning states, from a simple pre-container baseline to a more feature-rich platform with pricing, order management, and platform tooling. ## Where To Find The Code - Search generated branches directly: [code/generated-state-* branches](https://github.com/finos/traderX/branches/all?query=code%2Fgenerated-state-) - See full state mapping and links: [State Docs](/docs/spec-kit/state-docs) - See the progression and visual graph: [Learning Paths](/docs/learning-paths) - For private overlay customization: [Customizing TraderX](/docs/spec-kit/customizing-traderx) - Canonical overlay starter template: `examples/custom-overlay-template/` If you want to run demos quickly, use the generated state branches listed below. ## Generated Codebase States - [code/generated-state-001-baseline-uncontainerized-parity](https://github.com/finos/traderX/tree/code/generated-state-001-baseline-uncontainerized-parity) - [code/generated-state-002-edge-proxy-uncontainerized](https://github.com/finos/traderX/tree/code/generated-state-002-edge-proxy-uncontainerized) - [code/generated-state-003-agentic-harness-foundation](https://github.com/finos/traderX/tree/code/generated-state-003-agentic-harness-foundation) - [code/generated-state-004-containerized-compose-runtime](https://github.com/finos/traderX/tree/code/generated-state-004-containerized-compose-runtime) - [code/generated-state-005-postgres-database-replacement](https://github.com/finos/traderX/tree/code/generated-state-005-postgres-database-replacement) - [code/generated-state-006-messaging-nats-replacement](https://github.com/finos/traderX/tree/code/generated-state-006-messaging-nats-replacement) - [code/generated-state-007-observability-lgtm-compose](https://github.com/finos/traderX/tree/code/generated-state-007-observability-lgtm-compose) - [code/generated-state-008-pricing-awareness-market-data](https://github.com/finos/traderX/tree/code/generated-state-008-pricing-awareness-market-data) - [code/generated-state-009-order-management-matcher](https://github.com/finos/traderX/tree/code/generated-state-009-order-management-matcher) - [code/generated-state-010-kubernetes-runtime](https://github.com/finos/traderX/tree/code/generated-state-010-kubernetes-runtime) - [code/generated-state-011-tilt-kubernetes-dev-loop](https://github.com/finos/traderX/tree/code/generated-state-011-tilt-kubernetes-dev-loop) - [code/generated-state-012-platform-convergence-c3](https://github.com/finos/traderX/tree/code/generated-state-012-platform-convergence-c3) - [code/generated-state-013-radius-kubernetes-platform](https://github.com/finos/traderX/tree/code/generated-state-013-radius-kubernetes-platform) - [code/generated-state-014-fdc3-intent-interoperability](https://github.com/finos/traderX/tree/code/generated-state-014-fdc3-intent-interoperability) ## Why This Works These codebases are generated from specifications using GitHub Spec Kit. Specs, templates, and generation logic are the maintained source of truth; generated code branches are the runnable outputs. ## Contribution Workflow If you want to contribute new states: 1. Clone the main TraderX repository on macOS or Linux. 2. Use your preferred agentic coding assistant (for example Amp, Claude Code, or Codex). 3. Choose the parent state from the [Learning Paths](/docs/learning-paths). 4. Define the new state in specs (requirements, plan, tasks, architecture, contracts, and generator changes). 5. Implement and validate in the spec/source branch. 6. Commit spec/docs/generator changes, then publish the generated state branch. Example contribution themes: - Service mesh adoption. - Multi-temporal data support and related UI updates. - Swapping messaging or database implementations. ## Corporate Internal Extension Point If your organization needs private runtime constraints, internal-only states, or private generated branches: 1. Start with the canonical overlay starter under `examples/custom-overlay-template/`. 2. Create a separate corporate overlay repository (do not fork TraderX for corporate-only deltas). 3. Add TraderX as a pinned submodule dependency. 4. Tailor the included profile, transform, and internal-docs examples to your environment policy. 5. Publish an internal docs portal that includes only your sanctioned internal learning graph branches. For generated snapshot publishing and live-demo operations, follow [Publishing security and verified demo deployment](./publishing-security-and-deployment.md). Live redeployment requires the full generated snapshot SHA and successful checks for that exact snapshot. --- ## SpecKit Getting Started This project is built so a contributor can regenerate the baseline system from requirements and then evolve future states through SpecKit feature packs. The baseline state is intentionally **pre-Docker / pre-ingress**: - nine local processes - fixed startup order - fixed default ports - explicit cross-origin CORS behavior From this state, we generate code from specs and move forward by adding controlled FR/NFR deltas. Create a new planned state pack from template: ```bash bash pipeline/scaffold-state-pack.sh <NNN-state-name> --title "<Title>" --previous <prior-state-id> --track <prelude|baseline|architecture|nonfunctional|functional|devex> ``` Run the integrated state-change playbook: ```bash bash pipeline/state-playbook.sh --state <state-id> --publish-neighborhood --push-generated ``` ## New Contributor Path 1. Read the source-of-truth map: - `/docs/spec-kit/spec-kit-portal` 2. Run the baseline from specs: - `/docs/spec-kit/spec-kit-generation-guide` 3. Understand validation and parity gates: - `/docs/spec-kit/spec-kit-workflow` 4. Understand generated-code branch publishing: - `/docs/spec-kit/generated-state-branches` - `/docs/spec-kit/generated-state-ci` - `/docs/spec-kit/live-environments` - `/docs/spec-kit/aws-ec2-compose-prerequisites` - `/docs/spec-kit/aws-ec2-kubernetes-prerequisites` 5. Follow patch-set implementation workflow: - `/docs/spec-kit/patchset-authoring` - `/docs/spec-kit/messaging-subject-map-standard` 6. Browse state-specific architecture and flow docs: - `/docs/spec-kit/state-docs` 7. Browse developer-focused generated-code guides per state: - `/docs/learning` 8. Browse core artifacts: - `/specs` - `/specify` 9. Plan future-state transitions: - `/docs/spec-kit/spec-kit-learning-path-strategy` - `/docs/spec-kit/convergence-states` - `/docs/learning-paths` 10. Plan enterprise/private extension strategy: - `/docs/spec-kit/customizing-traderx` - `/docs/spec-kit/corporate-environments-guide` (overview) - `/docs/spec-kit/custom-overlay-architecture` (implementation contract) - `/docs/spec-kit/custom-environments-guide` (environment integration) - canonical starter: `examples/custom-overlay-template/` ## Where Is The Generated Code? All state code is available in this repository as generated-state branches/tags, while `main` keeps the spec-first source of truth. - Branch pattern: `code/generated-state-*` - Tag pattern: `generated/<state-id>/vN` - Publish mapping source: `catalog/state-catalog.json` Browse directly: - [Generated-State Branches on GitHub](https://github.com/finos/traderX/branches/all?query=code%2Fgenerated-state-) - [Generated-State Tags on GitHub](https://github.com/finos/traderX/tags?query=generated%2F) - [State Catalog (`catalog/state-catalog.json`)](https://github.com/finos/traderX/blob/main/catalog/state-catalog.json) - [Live Environments (`catalog/live-environments.json`)](https://github.com/finos/traderX/blob/main/catalog/live-environments.json) - [Generated State Branches Guide](/docs/spec-kit/generated-state-branches) - [Live Environments Guide](/docs/spec-kit/live-environments) ## What Is Canonical - SpecKit scaffold and governance: `/.specify/**` - Baseline feature pack: `/specs/001-baseline-uncontainerized-parity/**` - Learning-path definitions and state map: `/docs/learning-paths/**` and `/catalog/learning-paths.yaml` - Generation/runtime orchestration: `/pipeline/**` and `/scripts/**` ## Why This Structure Works - Requirements and stories are explicit and reviewable. - Code generation is reproducible and test-gated. - State transitions are auditable through numbered feature packs. - Learning paths stay consistent with the same baseline contracts and behaviors. - Parallel or conflicting state options can coexist with low maintenance overhead because specs are canonical and code is generated. ## Learning Path Rule Set - DevEx and non-functional paths should primarily layer NFR and operational constraints. - Functional paths may introduce FR deltas, but must preserve baseline compatibility unless explicitly versioned. - Every transition should include traceability updates and conformance checks before promotion. Use `/docs/spec-kit/spec-kit-learning-path-strategy` for the concrete transition template. --- ## Live Environments This page tracks canonical live TraderX demo environments and their mapped generated-state branches. Source of truth: - `catalog/live-environments.json` - `catalog/state-catalog.json` ## Launch Live Demos Launch Original Demo Launch Advanced Demo ## Current Live Demos | Environment | Demo Name | URL | State | Generated Branch | Notes | | --- | --- | --- | --- | --- | --- | | `demo-004` | `Original Demo` | [demo.traderx.finos.org](https://demo.traderx.finos.org) | `004-containerized-compose-runtime` | [`code/generated-state-004-containerized-compose-runtime`](https://github.com/finos/traderX/tree/code/generated-state-004-containerized-compose-runtime) | Containerized compose baseline demo | | `demo-advanced-009` | `Advanced Demo` | [demo-advanced.traderx.finos.org](https://demo-advanced.traderx.finos.org) | `009-order-management-matcher` | [`code/generated-state-009-order-management-matcher`](https://github.com/finos/traderX/tree/code/generated-state-009-order-management-matcher) | NATS + order-management demo | ## Update Process When an environment is moved to a different state: 1. Update `catalog/live-environments.json`. 2. Ensure the target state `deploy` metadata in `catalog/state-catalog.json` is accurate. 3. Republish the target generated-state branch and deploy bundle. 4. Validate with deploy-bundle dry-runs and smoke checks before cutover. --- ## Messaging Subject Map Standard States that introduce, remove, or modify message-bus subject families MUST maintain `system/messaging-subject-map.md`. ## Scope - Required for states `006+`. - The map is cumulative for the state: list all active subject families in that state (not only the delta). ## Required Sections Each file must contain: - `## Subject Families` - One bullet per subject family using the schema below. ## Entry Schema ```markdown - `<subject family>` - producer: `<component>` - consumer: `<component>` (or `consumers:` list) - delivery: `point-to-point` | `broadcast` - wildcard: `yes` | `no` (include pattern when yes) - scope: `<global|per-account|per-ticker|...>` - payload: <summary of key fields/contracts> ``` ## Authoring Rules - Keep names and wildcard patterns aligned with runtime implementation. - Keep payload summaries aligned with API/event contracts for that state. - When a state adds subject families, update this file in that state pack. - If a generated state contains `order-matcher`, the map must include: - `/accounts/<accountId>/orders` - `/orders` ## Validation Generated-state contract validation (`pipeline/validate-generated-state-contracts.sh`) enforces required map presence and order-matcher subject-map coverage rules. --- ## Patch-Set Authoring Workflow Use this workflow for any new state implementation or state update. ## Why - Keeps derived state logic as explicit diffs from parent state. - Reduces brittle shell scripts that write full file payloads. - Makes LLM changes easier to review and safer to regenerate. ## Standard Flow 1. Pick the parent state from `catalog/state-catalog.json`. 2. Implement the child-state changes (component/root changes as required). 3. Capture/refresh the patch set. 4. Regenerate via `pipeline/generate-state.sh <state-id>`. 5. Run state smoke tests and quality gates. ## Capture Commands Runtime-root states (`003+`): ```bash bash pipeline/create-state-patchset.sh <state-id> <parent-state-id> ``` Component-root overlays (state `002` pattern): ```bash bash pipeline/create-state-patchset.sh <state-id> <parent-state-id> generated/code/components ``` ## Apply Command ```bash bash pipeline/apply-state-patchset.sh <state-id> [target-root] ``` ## LLM Implementation Rules When an LLM implements a state change: 1. Modify files in generated parent output only for that target state. 2. Regenerate patch set immediately using `create-state-patchset.sh`. 3. Replace state hook logic with parent-generation + patch-apply model. 4. Never keep long heredoc file-payload scripts for derived states. 5. Update `specs/<state>/generation/generation-hook.md` with parent, patch path, and refresh command. ## Validation Run at minimum: ```bash bash pipeline/speckit/validate-root-spec-kit-gates.sh bash pipeline/verify-spec-coverage.sh ``` Then run target state smoke tests (and full stack sweep when state impacts shared contracts). --- ## Publishing security and verified demo deployment Generated snapshots retain the one-snapshot ancestry contract: regenerate from the canonical sources, reset to the configured base, and publish with force-with-lease. Security results never require an extra commit on a generated branch. ## Normal path 1. Generate the state and run `pipeline/publish-generated-state-branch.sh <state-id> --push`. Local dependency scans have independent acquisition and execution deadlines. Configure `TRADERX_SECURITY_PULL_TIMEOUT_SECONDS` (default 120) and `TRADERX_SECURITY_SCAN_TIMEOUT_SECONDS` (default 1800) as positive integers. Python 3 is required. The deadline kills the operation's process group; Docker cleanup has an additional maximum five-second allowance and targets only its unique scanner container. No Docker daemon, unrelated container, or shared scanner cache is removed. 2. Wait for **Build and Test Snapshot**, **Security Scanning**, and **Build and Publish Application Images** on the exact generated commit. Every job must succeed; missing, skipped, pending, failed, cancelled, timed-out and neutral results are ineligible. A successful image build cannot substitute for dependency security. The newest push run and its current attempt are used, never a previous successful run. Rerun the push workflow after fixing infrastructure; manually dispatched substitute workflows are not accepted. 3. Dispatch **Redeploy Live Demo** from the canonical branch, providing the target host and full generated commit SHA. The SHA must match the catalog's generated branch tip for that host. The workflow verifies expected jobs and downloads digest artifacts from that snapshot's successful publishing run and attempt. Expired or missing artifacts block deployment. 4. The host checks out that SHA, substitutes the scanned application digests, resolves infrastructure images to digests and scans those digests for HIGH/CRITICAL findings. It starts Compose with `--no-build --wait`. Any image lacking required evidence or any infrastructure scan failure stops deployment. Neither `latest` nor a subsequently changed SHA tag can change the deployed application images. 5. The workflow retains `deployment-authorization-*` as its audit artifact. The host records the deployed snapshot, run URLs, attempts, timestamp and complete resolved service image set in `~/traderx/.traderx-deploy/deployed.json` and a SHA-named history file only after Compose reports success. The directory is private because resolved Compose configuration may contain environment values. For direct generated-bundle use, authenticate `gh` with read access to repository Actions, set `TRADERX_SNAPSHOT` to the full SHA, and run `runtime/deploy/aws-ec2-compose/deploy.sh --use-ghcr`. This performs the same remote authorization. `--dry-run` validates the SHA format and describes the operation without contacting or modifying the host. Python 3, Git, Docker Compose with `--wait` support, outbound GitHub/GHCR/Trivy access and the existing demo runtime prerequisites must be available. The live workflow no longer invokes an unverified `~/redeploy.sh`; migrate host-specific settings into its environment/runtime configuration before the first deployment. The non-GHCR bundle path and clone-first scripts remain available for local development builds. They do not grant live-demo deployment eligibility. ## Failure recovery Local reports live under `ci/local-security-reports/<project>/`. `acquisition.json` and `result.json` distinguish success (exit 0), findings (10), infrastructure failure (20), timeout (124), and cancellation (130). Dependency-Check's findings exit status is mapped separately from its analyzer/update failures. The overall `ci/local-security-status.json` stays incomplete if execution stops early. For acquisition failures, restore registry/Docker access and rerun; increase the acquisition deadline only when the download needs more time. For scanner execution failures, restore its database/NVD access or adjust the execution deadline. Reusing an existing database via `TRADERX_DEPENDENCY_CHECK_NO_UPDATE=1` remains supported. Inspect the named container if cleanup could not be confirmed. Findings require remediation or reviewed, narrowly scoped canonical suppression changes, followed by regeneration and successful CI. `--skip-cve-scan` / `TRADERX_SKIP_CVE_SCAN=1` and `--skip-prepublish-gate` explicitly record **deferred_to_ci** in snapshot metadata. They allow publishing for CI evaluation; they do not waive deployment checks. Compilation and other local skips likewise cannot bypass the required remote workflows. If deployment stops before Compose starts, the previous running containers remain in place. If Compose fails partway through startup, no successful deployment record is written; inspect service health and use the last recorded digest set for incident diagnosis. Rollback through the normal workflow requires republishing the intended source as the current generated snapshot and completing its checks; the gate deliberately rejects old branch tips. Do not rerun with `latest` as a workaround. ## Exceptions There is no built-in force-deploy or security-check bypass. Any exceptional policy change must be reviewed as a canonical pull request with the incident/approval link, precise scope, owner, expiry and recovery plan. Retain that review and deployment authorization artifact for audit. An ad hoc local skip or manually edited manifest is not an approved exception. Regression coverage: `python3 scripts/test-publishing-security.py` and `bash scripts/test-generated-ci-assets.sh`. Dependency version remediation is tracked separately in issue #465; issue #466 defines these publishing/deployment guarantees. --- ## Run Generated Baseline This is the canonical runbook for the base uncontainerized generated runtime. ## Regenerate All Base Components ```bash bash pipeline/generate-state.sh 001-baseline-uncontainerized-parity ``` Generation now installs local runtime scripts into: - `generated/code/target-generated/scripts` - state-local runbook: `generated/code/target-generated/RUN_FROM_GENERATED.md` ## Start Full Overlay Stack ```bash CORS_ALLOWED_ORIGINS=http://localhost:18093 ./scripts/start-base-uncontainerized-generated.sh --build-only CORS_ALLOWED_ORIGINS=http://localhost:18093 ./scripts/start-base-uncontainerized-generated.sh ``` Optional if dependencies are already cached: ```bash TRADERSPEC_SKIP_NETWORK_CHECK=1 CORS_ALLOWED_ORIGINS=http://localhost:18093 ./scripts/start-base-uncontainerized-generated.sh --build-only CORS_ALLOWED_ORIGINS=http://localhost:18093 ./scripts/start-base-uncontainerized-generated.sh ``` ## Dry Run ```bash ./scripts/start-base-uncontainerized-generated.sh --dry-run ``` ## Smoke Test Suite ```bash ./scripts/test-reference-data-overlay.sh ./scripts/test-database-overlay.sh ./scripts/test-people-service-overlay.sh ./scripts/test-account-service-overlay.sh ./scripts/test-position-service-overlay.sh ./scripts/test-trade-feed-overlay.sh ./scripts/test-trade-processor-overlay.sh ./scripts/test-trade-service-overlay.sh ./scripts/test-web-angular-overlay.sh ``` ## Stop ```bash ./scripts/stop-base-uncontainerized-generated.sh ``` ## State 002 (Edge Proxy) Runtime Generate and start: ```bash bash pipeline/generate-state.sh 002-edge-proxy-uncontainerized ./scripts/start-state-002-edge-proxy-generated.sh --build-only ./scripts/start-state-002-edge-proxy-generated.sh ``` Smoke test: ```bash ./scripts/test-state-002-edge-proxy.sh ``` Stop: ```bash ./scripts/stop-state-002-edge-proxy-generated.sh ``` ## State 004 (Containerized Compose) Runtime Generate and start: ```bash bash pipeline/generate-state.sh 004-containerized-compose-runtime ./scripts/start-state-004-containerized-generated.sh ./scripts/start-state-004-containerized-generated.sh --skip-build ``` Ingress/UI endpoint: `http://localhost:8080` Status: ```bash ./scripts/status-state-004-containerized-generated.sh ``` Smoke test: ```bash ./scripts/test-state-004-containerized.sh ``` Stop: ```bash ./scripts/stop-state-004-containerized-generated.sh ``` --- ## Spec Kit Generation Guide This guide describes how to regenerate TraderX code from specs and evolve between states. ## Source of Truth - Spec framework: `/.specify/**` - State packs: `/specs/NNN-*` - State lineage and publish metadata: `/catalog/state-catalog.json` Refresh all derived learning/state docs: ```bash bash pipeline/refresh-state-docs.sh ``` ## Generate and Run a State Generate any state: ```bash bash pipeline/generate-state.sh <state-id> ``` Typical examples: ```bash bash pipeline/generate-state.sh 001-baseline-uncontainerized-parity bash pipeline/generate-state.sh 004-containerized-compose-runtime bash pipeline/generate-state.sh 007-observability-lgtm-compose bash pipeline/generate-state.sh 012-platform-convergence-c3 ``` ## Override Generated Output Root Generation defaults to `generated/` under the TraderX repository root. You can override that location with: ```bash TRADERX_GENERATED_ROOT=/absolute/path/to/generated bash pipeline/generate-state.sh <state-id> ``` When unset, behavior is unchanged. ## Generated Runtime Harness Requirement Every generated state output now includes a local runtime harness under: - `generated/code/target-generated/scripts` - `generated/code/target-generated/RUN_FROM_GENERATED.md` This harness is part of the generation contract and must include state-local `start/stop/status` scripts (plus state smoke tests where available) so each generated codebase is runnable with local scripts. Root scripts under `/scripts` act as wrappers and may forward to the generated local harness when present. Harness sanity check: ```bash ./scripts/test-generated-runtime-harness.sh ``` Run examples: ```bash # 001 (uncontainerized) CORS_ALLOWED_ORIGINS=http://localhost:18093 ./scripts/start-base-uncontainerized-generated.sh --build-only CORS_ALLOWED_ORIGINS=http://localhost:18093 ./scripts/start-base-uncontainerized-generated.sh # 003 (compose) ./scripts/start-state-004-containerized-generated.sh ./scripts/start-state-004-containerized-generated.sh --skip-build # 006 (compose + observability) ./scripts/start-state-007-observability-lgtm-compose-generated.sh ./scripts/start-state-007-observability-lgtm-compose-generated.sh --skip-build # 009 (kubernetes) ./scripts/start-state-010-kubernetes-runtime-generated.sh --provider kind ./scripts/start-state-010-kubernetes-runtime-generated.sh --provider kind --skip-build ``` ## Derived-State Implementation Pattern For states `002+`: 1. Generate parent state. 2. Apply patch set from `specs/<state>/generation/patches/*.patch`. 3. Regenerate architecture docs. 4. Run state smoke tests + global gates. Patch tooling: ```bash bash pipeline/apply-state-patchset.sh <state-id> [target-root] bash pipeline/create-state-patchset.sh <state-id> <parent-state-id> [target-path] ``` ## Convergence-First Guidance - Prefer starting new state work from convergence states: - `003` (`C0`) - `006` (`C1`) - `008` (`C2`) - `011` (`C3`) - Keep `previous` single-parent. - Use `dottedParents` only for convergence states. - Update `system/convergence-rationale.md` when changing convergence states. ## Validation Gates ```bash bash pipeline/refresh-state-docs.sh --check bash pipeline/validate-state-pack-artifacts.sh bash pipeline/verify-spec-coverage.sh ``` ## Publish Code Snapshot Branches ```bash bash pipeline/publish-generated-state-branch.sh <state-id> --push ``` Published branches include: - `README.md` (lineage + convergence context) - `STATE.md` - `.traderx-state/state.json` - `LEARNING.md` --- ## SpecKit Learning Path Strategy TraderX uses a convergence-first state model on top of GitHub Spec Kit artifacts in `specs/` and `.specify/`. ## Canonical Progression - Prelude onboarding: `001 -> 002` - `C0`: `003` - Architecture to `C1`: `004 -> 005 -> 006` - Functional to `C2`: `007 -> 008` - Platform to `C3`: `009 -> 010 -> 011` - Optional side branch: `012` off `009` Convergence checkpoints: - `C0`: `004-containerized-compose-runtime` - `C1`: `007-observability-lgtm-compose` - `C2`: `009-order-management-matcher` - `C3`: `012-platform-convergence-c3` ## Lineage Rules - `previous` is the single publish lineage parent (max length 1). - `dottedParents` is documentation lineage only and allowed only on convergence states. - `primaryLineageRole` is one of `prelude`, `canonical`, `optional`. ## Authoring Guidance 1. Start new work from the nearest suitable convergence state by default. 2. Create one feature pack per transition. 3. Add FR deltas only when behavior changes. 4. Add NFR deltas for runtime/platform/ops objectives. 5. Keep contract changes scoped and explicit. 6. If changing a convergence state, update `system/convergence-rationale.md`. ## Required Artifacts Per Transition - `spec.md` - `plan.md` - `tasks.md` - `system/**` (requirements, flows, architecture, topology) - `generation/generation-hook.md` - `generation/patches/*.patch` for derived-state implementation deltas - `tests/smoke/README.md` ## Validation and Promotion Loop 1. Update spec artifacts. 2. Regenerate state outputs. 3. Refresh catalog-derived docs. 4. Run gates and state smoke tests. 5. Publish generated code branch when green. Commands: ```bash bash pipeline/refresh-state-docs.sh bash pipeline/verify-spec-coverage.sh bash pipeline/publish-generated-state-branch.sh <state-id> --push ``` ## References - State catalog: `catalog/state-catalog.json` - Visual graph: `/docs/learning-paths` - Convergence policy: `/docs/spec-kit/convergence-states` --- ## Spec Kit Portal This is the single map for where Spec Kit artifacts live in TraderX and how they are used for generation. ## Published Sources The portal publishes three repository sources: 1. Root feature packs under `specs/**` 2. Root Spec Kit scaffold under `.specify/**` 3. Generated runtime/doc artifacts under `generated/**` (ephemeral outputs) ## Browse in Docusaurus - Root specs catalog: `/specs` - Baseline feature pack: `/specs/baseline-uncontainerized-parity` - Docs portal homepage feature pack: `/specs/docs-portal-homepage` - State docs map (flows + architecture): `/docs/spec-kit/state-docs` - Learning guides for generated code states: `/docs/learning` - OpenAPI API Explorer (runtime): `/api/docs` (served by running state ingress/edge) - Project ADRs: `/docs/adr` - `.specify` constitution and templates: `/specify/memory/constitution` - Generation operator guide: `/docs/spec-kit/spec-kit-generation-guide` - Generated-state branch model: `/docs/spec-kit/generated-state-branches` - Generated-state CI policy: `/docs/spec-kit/generated-state-ci` - Live demo environment registry: `/docs/spec-kit/live-environments` - EC2 Compose host prerequisites: `/docs/spec-kit/aws-ec2-compose-prerequisites` - EC2 Kubernetes host prerequisites (planned profile): `/docs/spec-kit/aws-ec2-kubernetes-prerequisites` - Learning-path visuals: `/docs/learning-paths` ## Homepage Contract The Docusaurus root route `/` is the TraderX reference architecture portal homepage. It replaces the former default Docusaurus homepage and the standalone `new_website/traderx.html` prototype. Maintaining spec: - Feature pack: `/specs/docs-portal-homepage` - Contract: `/specs/docs-portal-homepage/system/homepage-contract` Homepage source rules: - Generated state blocks are sourced from `catalog/state-catalog.json`. - Live demo blocks are sourced from `catalog/live-environments.json` and joined to the state catalog by `stateId`. - State cards must link to canonical specs, architecture, runtime docs, learning guides, generated code branches, and ADRs when declared in the catalog. - Footer source claims must distinguish catalog-backed state data from curated homepage copy. ## Official GitHub Spec Kit There is currently no official Docusaurus plugin from GitHub Spec Kit. Official references: - Spec Kit docs: [https://github.github.com/spec-kit/index.html](https://github.github.com/spec-kit/index.html) - Spec Kit quickstart: [https://github.github.com/spec-kit/quickstart.html](https://github.github.com/spec-kit/quickstart.html) - Spec Kit repository: [https://github.com/github/spec-kit](https://github.com/github/spec-kit) ## How This Repo Maps to Spec Kit Core alignment: - `.specify/` initialized and active - numbered root feature packs under `specs/NNN-*` - `spec.md` -> `plan.md` -> `tasks.md` flow in baseline pack - constitution and templates in active use - CI/root gates for feature-pack and branch/feature resolution checks Repo-specific extensions: - manifest-driven synthesis compiler and component generators - conformance packs and semantic compare harness - parity smoke-test gates for runtime validation - optional `learn` community extension for richer learning-material authoring workflows These extensions are intentional and sit on top of, not instead of, the core Spec Kit workflow. --- ## GitHub Spec Kit Workflow TraderX baseline generation now follows a root-canonical GitHub Spec Kit flow. ## Canonical Locations - `.specify/**` for constitution and templates - `specs/001-baseline-uncontainerized-parity/**` for baseline requirements, stories, plan, tasks, and contracts - `pipeline/**` for generation and validation orchestration Each state pack under `specs/NNN-*` must include: - `research.md` - `data-model.md` - `quickstart.md` Convergence states must also include: - `system/convergence-rationale.md` Browse in docs: - `/specs/baseline-uncontainerized-parity` - `/specs/baseline-uncontainerized-parity/system/system-requirements` - `/specs/baseline-uncontainerized-parity/system/user-stories` - `/specs/baseline-uncontainerized-parity/system/acceptance-criteria` - `/specs/baseline-uncontainerized-parity/system/requirements-traceability` - `/specs/baseline-uncontainerized-parity/system/end-to-end-flows` - `/specs/baseline-uncontainerized-parity/system/architecture` - `/docs/spec-kit/state-docs` - `/docs/learning` - runtime API explorer route: `/api/docs` (via running state ingress/edge) - `/specify` ## Input Evidence For Requirements - `specs/001-baseline-uncontainerized-parity/system/architecture.model.json` - `specs/001-baseline-uncontainerized-parity/system/end-to-end-flows.md` - `specs/001-baseline-uncontainerized-parity/system/system-requirements.md` - `docs/README.md` - `README.md` ## Baseline Generation Flow 1. Validate Spec Kit readiness and requirement coverage. 2. Compile component manifests from Spec Kit artifacts. 3. Synthesize generated components from manifest + templates. 4. Start generated overlays and run smoke tests/parity checks. ## Derived-State Generation Flow For states `002+`, generation follows: 1. Generate parent state output. 2. Apply ordered patch set from `specs/<state>/generation/patches/*.patch`. 3. Regenerate architecture docs and run state smoke checks. ## Convergence-State Policy - Prefer new state proposals from the nearest suitable convergence state (`C0/C1/C2/C3`). - Keep `previous` single-parent for publish lineage. - Use `dottedParents` only for convergence states. - If convergence-state metadata/content changes, update that state's `system/convergence-rationale.md`. - Reference: [`/docs/spec-kit/convergence-states`](/docs/spec-kit/convergence-states) ## LLM Feature-Add Implementation Contract When implementing a new state delta with an LLM: 1. Generate the parent state (`bash pipeline/generate-state.sh <parent-state-id>`). 2. Make only the intended child-state edits against generated output. 3. Capture patch set (`bash pipeline/create-state-patchset.sh <state-id> <parent-state-id> [target-path]`). 4. Keep state hook as parent-generation + `apply-state-patchset.sh` only. 5. Update `specs/<state>/generation/generation-hook.md` with parent, patch path, and refresh commands. 6. Do not introduce or keep large file-payload heredoc generators for derived states. 7. Keep patch payloads clean: exclude build/restored artifacts (`.gradle`, `build`, `target`, `bin`, `obj`, `dist`, `coverage`, `node_modules`) and template-owned wrapper artifacts; include lockfile deltas only when Node manifests change. ## Generation Concurrency Run generation sequentially by default. `pipeline/generate-state.sh` writes to shared output roots (`generated/code/target-generated`, `generated/code/components`), so parallel invocations can race. Use isolated `TRADERX_GENERATED_ROOT` directories only when intentionally running multiple generation sessions in separate workspaces. ## Validation Commands ```bash bash pipeline/refresh-state-docs.sh --check bash pipeline/validate-state-doc-consistency.sh bash pipeline/validate-state-pack-artifacts.sh bash pipeline/validate-template-version-consistency.sh bash pipeline/validate-generated-branch-dependency-consistency.sh bash pipeline/validate-generated-dependency-targets.sh generated/code/components generated/code/target-generated bash pipeline/validate-sail-pin-contract.sh ./pipeline/speckit/validate-speckit-readiness.sh ./pipeline/speckit/verify-spec-expressiveness.sh bash pipeline/speckit/compile-all-component-manifests.sh ./pipeline/validate-regeneration-readiness.sh ./pipeline/verify-spec-coverage.sh ``` ## Generated-State CI Preflight For generated-code branch publish, prepublish CI-parity gating is executed by default by `pipeline/publish-generated-state-branch.sh`. Run it manually when you need a standalone check: ```bash bash pipeline/prepublish-generated-state-gate.sh <state-id> ``` Use `--skip-prepublish-gate` only as an explicit temporary override. See: [`/docs/spec-kit/generated-state-ci`](/docs/spec-kit/generated-state-ci) ## Docusaurus CI Compatibility When changing Docusaurus, docs plugins, or website build configuration: 1. Keep `website/package-lock.json` committed and updated with the dependency change. 2. Use `npm --prefix website ci --no-audit --no-fund` (not `npm install`) for CI parity. 3. Validate docs build with GitHub Pages settings: ```bash DOCUSAURUS_URL=https://traderx.finos.org \ DOCUSAURUS_BASE_URL=/ \ bash pipeline/refresh-state-docs.sh DOCUSAURUS_URL=https://traderx.finos.org \ DOCUSAURUS_BASE_URL=/ \ npm --prefix website run clear DOCUSAURUS_URL=https://traderx.finos.org \ DOCUSAURUS_BASE_URL=/ \ npm --prefix website run build ``` `TRADERX_SITE_ROOT` controls the docs-route prefix used when regenerating `/docs/learning-paths` Mermaid click links. Keep it empty for root deployment (`/`) and set it only when publishing under a subpath. 4. If dependency changes alter transitive Webpack behavior, pin and re-lock before merge. 5. Run AFDocs checks for agent-friendliness: ```bash # local preview (while docusaurus start runs) npx afdocs check http://localhost:3000 --format scorecard npx afdocs check http://localhost:3000 --format json --score # published docs URL npx afdocs check https://traderx.finos.org/ --format scorecard ``` Use AFDocs scorecards to prioritize high-impact fixes. Prefer improvements via docs build/pipeline/plugins (for example llms outputs) before broad manual content rewrites. Learning-path catalog policy: - `catalog/state-catalog.json` is the source for state lineage. - Derived artifacts are regenerated together by: - `bash pipeline/refresh-state-docs.sh` - Derived artifacts include: - `catalog/learning-paths.yaml` - `catalog/learning-paths.md` - `/docs/learning-paths` - `/docs/spec-kit/state-docs` - `/docs/learning/index` - `/docs/learning/state-*.md` ## Generation Commands Generate any state: ```bash bash pipeline/generate-state.sh <state-id> ``` These commands also regenerate state architecture docs from `specs/*/system/architecture.model.json`. State docs (`/docs/spec-kit/state-docs`) are generated from `catalog/state-catalog.json`. Scaffold a new planned state pack: ```bash bash pipeline/scaffold-state-pack.sh <NNN-state-name> --title "<Title>" --previous <prior-state-id> --track <prelude|baseline|architecture|nonfunctional|functional|devex> ``` Run the state-change playbook (refresh docs -> gates -> generate -> optional publish/push): ```bash bash pipeline/state-playbook.sh --state <state-id> --publish-neighborhood --push-generated ``` Dependency maintenance refresh contract: 1. Start from the earliest implemented state impacted by the dependency change. 2. Run forward through downstream implemented states. 3. Require smoke-test pass evidence before publish. Sail-specific maintenance checks (state `014`): ```bash # local pin contract check bash pipeline/validate-sail-pin-contract.sh # upstream drift check (fails if tracking ref moved beyond the pinned commit) bash pipeline/check-sail-pin-drift.sh --fail-on-drift ``` Sail pin metadata source: - `specs/014-fdc3-intent-interoperability/generation/sail-pin.env` Generated outputs: - `generated/code/components/*-specfirst` - `generated/manifests/*.manifest.json` - `generated/code/target-generated/**` (assembled at startup) Run generated baseline stack: ```bash CORS_ALLOWED_ORIGINS=http://localhost:18093 ./scripts/start-base-uncontainerized-generated.sh --build-only CORS_ALLOWED_ORIGINS=http://localhost:18093 ./scripts/start-base-uncontainerized-generated.sh ``` Run generated containerized stack (state `004`): ```bash ./scripts/start-state-004-containerized-generated.sh ./scripts/start-state-004-containerized-generated.sh --skip-build ./scripts/status-state-004-containerized-generated.sh ./scripts/test-state-004-containerized.sh ./scripts/stop-state-004-containerized-generated.sh ``` Ingress endpoint: `http://localhost:8080` ## Full Parity Gate Run end-to-end parity validation (generation + startup + all baseline smoke tests): ```bash bash pipeline/speckit/run-full-parity-validation.sh ``` ## API Explorer API explorer generation is runtime-centric and decoupled from the docs portal. It is installed during state generation by: ```bash bash pipeline/install-generated-api-explorer.sh <state-id> ``` ## Compare Generation Output To compare a single component generated output between a legacy script revision and current Spec Kit-driven generation: ```bash bash pipeline/speckit/compare-component-generation.sh <component-id> <legacy-ref> ``` Example: ```bash bash pipeline/speckit/compare-component-generation.sh reference-data HEAD ``` ## Iterating Learning-Path States After baseline parity is green: 1. add FR/NFR deltas in the next feature pack under `specs/NNN-*` 2. update contracts if interfaces change 3. update and capture state patch set: ```bash bash pipeline/create-state-patchset.sh <state-id> <parent-state-id> [target-path] ``` 4. rerun conformance + parity gates This keeps each learning-path state reproducible from requirements instead of from copied source. --- ## State Docs This page is generated from `catalog/state-catalog.json` and links the most important per-state artifacts. For progression context, see [Visual Learning Paths](/docs/learning-paths). ## State Catalog | State | Status | Convergence | Docs | Architecture | Specs | Generated Code Branch | Compare To Previous | ADR | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `001-baseline-uncontainerized-parity` | released | `none` | [Learning Guide](/docs/learning/state-001-baseline-uncontainerized-parity)[Quick Start](/specs/baseline-uncontainerized-parity/quickstart) | [Architecture](/specs/baseline-uncontainerized-parity/system/architecture)[Flows](/specs/baseline-uncontainerized-parity/system/end-to-end-flows)[Data Model](/specs/baseline-uncontainerized-parity/data-model) | [Spec Pack](/specs/baseline-uncontainerized-parity)[Research](/specs/baseline-uncontainerized-parity/research) | [code/generated-state-001-baseline-uncontainerized-parity](https://github.com/finos/traderX/tree/code/generated-state-001-baseline-uncontainerized-parity) | `n/a` | `n/a` | | `002-edge-proxy-uncontainerized` | implemented | `none` | [Learning Guide](/docs/learning/state-002-edge-proxy-uncontainerized)[Quick Start](/specs/edge-proxy-uncontainerized/quickstart) | [Architecture](/specs/edge-proxy-uncontainerized/system/architecture)[Flows](/specs/edge-proxy-uncontainerized/system/runtime-topology)[Data Model](/specs/edge-proxy-uncontainerized/data-model) | [Spec Pack](/specs/edge-proxy-uncontainerized)[Research](/specs/edge-proxy-uncontainerized/research) | [code/generated-state-002-edge-proxy-uncontainerized](https://github.com/finos/traderX/tree/code/generated-state-002-edge-proxy-uncontainerized) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-001-baseline-uncontainerized-parity...code%2Fgenerated-state-002-edge-proxy-uncontainerized) | `n/a` | | `003-agentic-harness-foundation` | implemented | `none` | [Learning Guide](/docs/learning/state-003-agentic-harness-foundation)[Quick Start](/specs/agentic-harness-foundation/quickstart) | [Architecture](/specs/agentic-harness-foundation/system/architecture)[Flows](/specs/agentic-harness-foundation/system/runtime-topology)[Data Model](/specs/agentic-harness-foundation/data-model) | [Spec Pack](/specs/agentic-harness-foundation)[Research](/specs/agentic-harness-foundation/research) | [code/generated-state-003-agentic-harness-foundation](https://github.com/finos/traderX/tree/code/generated-state-003-agentic-harness-foundation) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-002-edge-proxy-uncontainerized...code%2Fgenerated-state-003-agentic-harness-foundation) | `n/a` | | `004-containerized-compose-runtime` | implemented | `C0` | [Learning Guide](/docs/learning/state-004-containerized-compose-runtime)[Quick Start](/specs/containerized-compose-runtime/quickstart) | [Architecture](/specs/containerized-compose-runtime/system/architecture)[Flows](/specs/containerized-compose-runtime/system/runtime-topology)[Data Model](/specs/containerized-compose-runtime/data-model) | [Spec Pack](/specs/containerized-compose-runtime)[Research](/specs/containerized-compose-runtime/research) | [code/generated-state-004-containerized-compose-runtime](https://github.com/finos/traderX/tree/code/generated-state-004-containerized-compose-runtime) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-003-agentic-harness-foundation...code%2Fgenerated-state-004-containerized-compose-runtime) | `n/a` | | `005-postgres-database-replacement` | implemented | `none` | [Learning Guide](/docs/learning/state-005-postgres-database-replacement)[Quick Start](/specs/postgres-database-replacement/quickstart) | [Architecture](/specs/postgres-database-replacement/system/architecture)[Flows](/specs/postgres-database-replacement/system/runtime-topology)[Data Model](/specs/postgres-database-replacement/data-model) | [Spec Pack](/specs/postgres-database-replacement)[Research](/specs/postgres-database-replacement/research) | [code/generated-state-005-postgres-database-replacement](https://github.com/finos/traderX/tree/code/generated-state-005-postgres-database-replacement) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-004-containerized-compose-runtime...code%2Fgenerated-state-005-postgres-database-replacement) | [link](/docs/adr/006-state-005-use-postgres-for-database-replacement) | | `006-messaging-nats-replacement` | implemented | `none` | [Learning Guide](/docs/learning/state-006-messaging-nats-replacement)[Quick Start](/specs/messaging-nats-replacement/quickstart) | [Architecture](/specs/messaging-nats-replacement/system/architecture)[Flows](/specs/messaging-nats-replacement/system/runtime-topology)[Data Model](/specs/messaging-nats-replacement/data-model) | [Spec Pack](/specs/messaging-nats-replacement)[Research](/specs/messaging-nats-replacement/research) | [code/generated-state-006-messaging-nats-replacement](https://github.com/finos/traderX/tree/code/generated-state-006-messaging-nats-replacement) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-005-postgres-database-replacement...code%2Fgenerated-state-006-messaging-nats-replacement) | [link](/docs/adr/005-state-006-use-nats-for-messaging-replacement) | | `007-observability-lgtm-compose` | implemented | `C1` | [Learning Guide](/docs/learning/state-007-observability-lgtm-compose)[Quick Start](/specs/observability-lgtm-compose/quickstart) | [Architecture](/specs/observability-lgtm-compose/system/architecture)[Flows](/specs/observability-lgtm-compose/system/runtime-topology)[Data Model](/specs/observability-lgtm-compose/data-model) | [Spec Pack](/specs/observability-lgtm-compose)[Research](/specs/observability-lgtm-compose/research) | [code/generated-state-007-observability-lgtm-compose](https://github.com/finos/traderX/tree/code/generated-state-007-observability-lgtm-compose) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-006-messaging-nats-replacement...code%2Fgenerated-state-007-observability-lgtm-compose) | [link](/docs/adr/007-state-007-adopt-lgtm-observability-stack) | | `008-pricing-awareness-market-data` | implemented | `none` | [Learning Guide](/docs/learning/state-008-pricing-awareness-market-data)[Quick Start](/specs/pricing-awareness-market-data/quickstart) | [Architecture](/specs/pricing-awareness-market-data/system/architecture)[Flows](/specs/pricing-awareness-market-data/system/runtime-topology)[Data Model](/specs/pricing-awareness-market-data/data-model) | [Spec Pack](/specs/pricing-awareness-market-data)[Research](/specs/pricing-awareness-market-data/research) | [code/generated-state-008-pricing-awareness-market-data](https://github.com/finos/traderX/tree/code/generated-state-008-pricing-awareness-market-data) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-007-observability-lgtm-compose...code%2Fgenerated-state-008-pricing-awareness-market-data) | `n/a` | | `009-order-management-matcher` | implemented | `C2` | [Learning Guide](/docs/learning/state-009-order-management-matcher)[Quick Start](/specs/order-management-matcher/quickstart) | [Architecture](/specs/order-management-matcher/system/architecture)[Flows](/specs/order-management-matcher/system/runtime-topology)[Data Model](/specs/order-management-matcher/data-model) | [Spec Pack](/specs/order-management-matcher)[Research](/specs/order-management-matcher/research) | [code/generated-state-009-order-management-matcher](https://github.com/finos/traderX/tree/code/generated-state-009-order-management-matcher) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-008-pricing-awareness-market-data...code%2Fgenerated-state-009-order-management-matcher) | `n/a` | | `010-kubernetes-runtime` | implemented | `none` | [Learning Guide](/docs/learning/state-010-kubernetes-runtime)[Quick Start](/specs/kubernetes-runtime/quickstart) | [Architecture](/specs/kubernetes-runtime/system/architecture)[Flows](/specs/kubernetes-runtime/system/runtime-topology)[Data Model](/specs/kubernetes-runtime/data-model) | [Spec Pack](/specs/kubernetes-runtime)[Research](/specs/kubernetes-runtime/research) | [code/generated-state-010-kubernetes-runtime](https://github.com/finos/traderX/tree/code/generated-state-010-kubernetes-runtime) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-009-order-management-matcher...code%2Fgenerated-state-010-kubernetes-runtime) | `n/a` | | `011-tilt-kubernetes-dev-loop` | implemented | `none` | [Learning Guide](/docs/learning/state-011-tilt-kubernetes-dev-loop)[Quick Start](/specs/tilt-kubernetes-dev-loop/quickstart) | [Architecture](/specs/tilt-kubernetes-dev-loop/system/architecture)[Flows](/specs/tilt-kubernetes-dev-loop/system/runtime-topology)[Data Model](/specs/tilt-kubernetes-dev-loop/data-model) | [Spec Pack](/specs/tilt-kubernetes-dev-loop)[Research](/specs/tilt-kubernetes-dev-loop/research) | [code/generated-state-011-tilt-kubernetes-dev-loop](https://github.com/finos/traderX/tree/code/generated-state-011-tilt-kubernetes-dev-loop) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-010-kubernetes-runtime...code%2Fgenerated-state-011-tilt-kubernetes-dev-loop) | `n/a` | | `012-platform-convergence-c3` | implemented | `C3`Dotted: [009](/docs/learning/state-009-order-management-matcher) | [Learning Guide](/docs/learning/state-012-platform-convergence-c3)[Quick Start](/specs/platform-convergence-c3/quickstart) | [Architecture](/specs/platform-convergence-c3/system/architecture)[Flows](/specs/platform-convergence-c3/system/runtime-topology)[Data Model](/specs/platform-convergence-c3/data-model) | [Spec Pack](/specs/platform-convergence-c3)[Research](/specs/platform-convergence-c3/research) | [code/generated-state-012-platform-convergence-c3](https://github.com/finos/traderX/tree/code/generated-state-012-platform-convergence-c3) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-011-tilt-kubernetes-dev-loop...code%2Fgenerated-state-012-platform-convergence-c3) | [link](/docs/adr/008-convergence-state-model) | | `013-radius-kubernetes-platform` | implemented | `none` | [Learning Guide](/docs/learning/state-013-radius-kubernetes-platform)[Quick Start](/specs/radius-kubernetes-platform/quickstart) | [Architecture](/specs/radius-kubernetes-platform/system/architecture)[Flows](/specs/radius-kubernetes-platform/system/runtime-topology)[Data Model](/specs/radius-kubernetes-platform/data-model) | [Spec Pack](/specs/radius-kubernetes-platform)[Research](/specs/radius-kubernetes-platform/research) | [code/generated-state-013-radius-kubernetes-platform](https://github.com/finos/traderX/tree/code/generated-state-013-radius-kubernetes-platform) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-012-platform-convergence-c3...code%2Fgenerated-state-013-radius-kubernetes-platform) | `n/a` | | `014-fdc3-intent-interoperability` | implemented | `none` | [Learning Guide](/docs/learning/state-014-fdc3-intent-interoperability)[Quick Start](/specs/fdc3-intent-interoperability/quickstart) | [Architecture](/specs/fdc3-intent-interoperability/system/architecture)[Flows](/specs/fdc3-intent-interoperability/system/runtime-topology)[Data Model](/specs/fdc3-intent-interoperability/data-model) | [Spec Pack](/specs/fdc3-intent-interoperability)[Research](/specs/fdc3-intent-interoperability/research) | [code/generated-state-014-fdc3-intent-interoperability](https://github.com/finos/traderX/tree/code/generated-state-014-fdc3-intent-interoperability) | [link](https://github.com/finos/traderX/compare/code%2Fgenerated-state-012-platform-convergence-c3...code%2Fgenerated-state-014-fdc3-intent-interoperability) | `n/a` | ## API Explorer by State - Runtime API explorer route: `/api/docs` - Current scope: `001-baseline-uncontainerized-parity` - Source contracts: `specs/001-baseline-uncontainerized-parity/contracts/**/openapi.yaml` - Runtime metadata source: `catalog/state-catalog.json` → `apiCatalog` ## How This Page Is Maintained - Source catalog: `catalog/state-catalog.json` - Regenerate docs pages: ```bash bash pipeline/refresh-state-docs.sh ``` - State architecture docs are generated from: - `specs/<state>/system/architecture.model.json` - Generate state architecture docs: ```bash bash pipeline/generate-state-architecture-doc.sh <state-id> ``` - Generate all state architecture docs: ```bash bash pipeline/generate-all-architecture-docs.sh ``` --- ## Multi-State Generation Plan This plan defines how TraderX evolves through numbered states with explicit convergence checkpoints. ## Objectives - keep each transition reproducible from specs, - keep generated branches compareable and publishable, - isolate FR/NFR deltas per state, - preserve a clear set of recommended jump-off points (`C0-C3`). ## Explicit State Inheritance Policy - Every state inherits all functional and non-functional behavior from its `previous` lineage by default. - A state may diverge from inherited behavior only when the feature pack declares an explicit conflict, replacement, or deprecation requirement. - If no explicit conflict/replacement/deprecation is declared, generation, runtime scripts, smoke tests, and docs must keep inherited capabilities intact. - Validation should verify inherited capabilities in downstream states to prevent accidental feature loss during runtime transitions. ## Containerization Boundary - States `001-003` are pre-container states: published generated branches should not include Docker/Compose artifacts. - Container build workflows begin at `004+`. - Only convergence states (`C0+`) publish container images. ## Transition Mechanics 1. Update target state pack (`spec.md`, `plan.md`, `tasks.md`, `system/**`). 2. Generate parent state. 3. Apply ordered patch set (`specs/<state>/generation/patches/*.patch`) for derived changes. 4. Regenerate architecture docs. 5. Run state smoke tests + global gates. 6. Publish code snapshot branch. ## Generation Concurrency Invariant - Default generation is sequential. The pipeline writes to shared roots under `generated/**`, so parallel runs targeting the same root are a race condition. - `pipeline/generate-state.sh` enforces this with a top-level generation lock and fails fast on concurrent shared-root invocations. - Parallel generation is allowed only when each run uses a distinct `TRADERX_GENERATED_ROOT` (for example separate workspaces or CI job sandboxes). - This invariant applies to upstream and overlay pipelines that mirror upstream generation behavior. ## Current Convergence-First Model ```mermaid flowchart LR S001["001 Prelude"] --> S002["002 Prelude"] S002 --> C0["003 C0"] C0 --> A004["004 Postgres"] A004 --> A005["005 NATS"] A005 --> C1["006 C1 Observability"] C1 --> F007["007 Pricing"] F007 --> C2["008 C2 Order Mgmt"] C2 --> P009["009 Kubernetes"] P009 --> P010["010 Tilt"] P010 --> C3["011 C3 Platform Convergence"] P009 --> O012["012 Radius Optional"] ``` ## Publish Model - Canonical authoring stays in this branch (`specs/**`, `.specify/**`, pipeline/docs). - Generated code snapshots are published to `code/generated-state-*` branches. - Publish ancestry follows `previous` only. - Dotted-line parents are docs lineage only. ## Required Governance for Convergence States - `catalog/state-catalog.json` must carry convergence metadata. - `system/convergence-rationale.md` must exist and be updated when convergence state content/metadata changes. - CI gates must pass before publish. - Convergence states `C0+` must include image build/publish CI and GHCR run-bundle artifacts (see `/docs/spec-kit/generated-state-ci`). ## Reference Commands ```bash bash pipeline/refresh-state-docs.sh bash pipeline/verify-spec-coverage.sh bash pipeline/publish-generated-state-branch.sh <state-id> --push ``` --- ## TraderX v2.0 Go-Live TODO Status: in progress. Created: 2026-05-21 ## Decisions (Current Direction) - Hosting: retire Netlify hosting for docs and use GitHub Pages as the single docs host. - Domain strategy: prefer custom domain on top of GitHub Pages (recommended: `traderx.finos.org`) instead of exposing raw `finos.github.io/traderX` as the canonical URL. - Generated docs branch links: add an override mechanism for authoring branch links in generated docs. Default should come from a configured default branch value, with explicit override for preview/PR builds. ## Go-Live Checklist - [x] 1) Docs cleanup to reference `main` as canonical authoring branch. - [x] 1.1) Remove/replace the renovation banner with launch messaging and engineering-blog link. - [x] 1.3) Update generator default branch reference used by learning/state docs. - [x] 1.4) Regenerate derived docs and validate no stale branch refs remain in generated learning/state docs. - [x] 2.0) Create the major PR from `feature/agentic-renovation` to `main` ([#354](https://github.com/finos/traderX/pull/354)). - [ ] 2.1) Merge the major PR from `feature/agentic-renovation` to `main`. - [x] 2.2) Confirm final pre-merge gates pass. - [x] 3.1) Switch Pages deploy trigger to `main`. - [ ] 3.2) Remove Netlify dependency from release path (including external project settings). - [x] 3.3) Decide canonical docs domain strategy (`traderx.finos.org` on GitHub Pages). - [x] 3.4) Set `DOCUSAURUS_URL` and `DOCUSAURUS_BASE_URL` to match the final domain choice. - [ ] 3.5) Validate published docs link behavior on final domain after deploy. - [ ] 4) Protect generated branches (`code/generated-state-*`) so only maintainers can push. - [ ] 5) Add CI-driven generated-state publication pipeline (maintainer manual regen/push is interim model). - [ ] 6) Improve contributor model documentation for spec-first + generated-branch workflow. - [ ] 7) Add dynamic target-branch support for generated code/docs links to improve local/PR preview simulation. ## Generator Override Proposal (Implementation Target) - Add optional env var `TRADERX_SOURCE_AUTHORING_BRANCH` and optional CLI flag `--source-authoring-branch <branch>` to `pipeline/generate-state-docs-from-catalog.mjs`. - Resolution order: 1. CLI flag 2. `TRADERX_SOURCE_AUTHORING_BRANCH` 3. default value from repo config (target: `main`) - Keep current behavior deterministic in CI by explicitly setting one value in workflow env. --- ## Learning Paths (Moved) The visual learning graph and state map were consolidated into a single canonical page: - [/docs/learning-paths](/docs/learning-paths) Use that page for the current convergence graph, swimlanes, and state-to-artifact mapping. --- ## Why TraderX Was Reworked Around SpecKit ## Mission TraderX was reworked around SpecKit to make multi-state learning paths practical to maintain. The primary goal is to support multiple parallel and sometimes conflicting state progressions (for example, different platform, DevOps, or messaging choices) while still giving developers clear, runnable jump-off points. Browse the state landscape directly: - [Visual Learning Paths](/docs/learning-paths) - [Spec Catalog](/specs) ## Problem With The Old Code-First Model When code was the main maintained artifact, each new state incarnation increased maintenance overhead and drift risk. - Parallel tracks were hard to keep aligned. - Conflicting directions were expensive to review and merge. - It was difficult to preserve clear relationships between states. - Maintainers spent too much effort on code deltas instead of intent and outcomes. ## What SpecKit Changes TraderX now treats specs as the system of record and code as a generated by-product. - Canonical intent, requirements, stories, and governance live in: - `.specify/**` - `specs/**` - State transitions are captured as explicit spec deltas. - Generation pipelines synthesize runnable code from those deltas. - Conformance and parity checks verify generated output against spec intent. This gives us an understandable and documented relationship between states while reducing overhead for maintainers. ## Main Consequence For Contribution Flow Feedback into the project is modeled as a spec/state change first, not as ad-hoc source edits first. - Proposed improvements become state-oriented spec updates. - Review focuses on requirements, contracts, and traceability. - Code is regenerated from accepted spec changes. In short: we preserve specs; the code follows. ## What A Developer Needs To Know 1. Start from baseline state `001-baseline-uncontainerized-parity`. 2. Read system context, requirements, and flows under `specs/001-*/system/**`. 3. Regenerate code using `pipeline/generate-state.sh <state-id>` (or component generators where needed). 4. Run with state runtime scripts under `scripts/`. 5. Validate with SpecKit/conformance gates under `pipeline/speckit/**`. ### Where Is All The Code? All TraderX state code lives in this same repository. The canonical source is specs in `main`, and runnable code snapshots are published as generated-state branches and tags. - [Generated State Branches Guide](/docs/spec-kit/generated-state-branches) - [Generated-State Branches on GitHub](https://github.com/finos/traderX/branches/all?query=code%2Fgenerated-state-) - [Generated-State Tags on GitHub](https://github.com/finos/traderX/tags?query=generated%2F) - [State Catalog (`catalog/state-catalog.json`)](https://github.com/finos/traderX/blob/main/catalog/state-catalog.json)