Ingests supplier catalogs, deduplicates products into single sellable items, computes the retail price, and routes orders and returns. Every event is written to an immutable audit log. Every capability is exposed over an HTTP API.
The shape of the system
A quick read: data flows left → right (normalize → dedup → price → publish), orders & returns flow back, and every event lands in an immutable audit log. Full annotated diagram lives in the Architecture tab.
Design principles
A change in one module cannot reach into another. Enforced by import-boundary tests in CI, not convention.
Modules communicate only through per-domain contract packages. common holds primitives only.
Every external event carries an idempotency key; retries are safe. Domain change and event emit commit in one transaction via an outbox.
Unit, property-based, integration, contract, and end-to-end tests gate every merge. Decimal money throughout.
OpenAPI / Swagger generated from the app. Every service is an HTTP tool any agent (Claude or OpenAI) can call. No MCP dependency.
Runs as a modular monolith or split into per-module services without code changes. Plain containers.
Decisions locked for v1
One FastAPI app, one Postgres, one worker, one scheduler. Strict in-code boundaries so any module can later become its own service.
Contracts live in catalog / pricing / orders / suppliers / marketplaces packages. common holds IDs, errors, config, logging, and the event envelope only.
UPC/GTIN and Brand+MPN auto-merge. Fuzzy Tier 3 goes to a human review queue at launch.
When MAP/MSRP clamping erases margin, suppress the listing or hold for review. Do not silently accept a bad margin.
The landed-cost field is in the model but defaults off until supplier shipping data is reliable.
Primary storefront connector first, then eBay / Amazon / Walmart. Ops actions (including manual merge/split) exposed as Swagger APIs in v1.
System architecture · data flow
The full picture. Suppliers feed in on the left; the core normalizes, merges, prices, and publishes; marketplaces receive listings on the right; orders & returns flow back; the system of record captures every event underneath.
Reading it: an offer is normalized to the canonical model, merged into a Master Product, priced once by the central engine, then published to every active marketplace. No connector ever computes price. Sourcing is decided at order time from the freshest cost & inventory.
The isolation guarantee
common, never fatOnly true primitives: IDs, base DTO patterns, error types, config, logging, auth, the event envelope, protocol helpers. Nothing domain-specific.
Each domain publishes its interface in contracts/<domain>. Modules depend on contracts, not on each other's code.
In monolith mode, contracts resolve to in-process calls. Split a module out and the same contract is backed by an HTTP client. No rewrite.
Separate Postgres schemas per module. No module reads another's tables — only its contract or its events.
An automated test fails the build if, say, pricing imports catalog internals. The boundary is enforced, not hoped for.
# Dependency direction — always inward, never sideways modules/pricing │ depends on ▼ contracts/pricing # interfaces + events contracts/catalog # reads offers via contract │ ▼ common # primitives only # ❌ what CI forbids from ridewell.modules.catalog._internal import ... # ✅ what's allowed from ridewell.contracts.catalog import CatalogQuery from ridewell.common.money import Money
Repository layout
src/ridewell/ common/ # primitives: ids, errors, config, logging, money, event envelope, secrets jobs/ # shared job envelope + status + retry / idempotency metadata contracts/ # interfaces + domain events — the only cross-module language catalog/ pricing/ orders/ suppliers/ marketplaces/ modules/ # isolated domains — each owns its tables, job handlers, DESIGN.md, tests supplier_adapters/ catalog/ pricing/ orders/ marketplace_connectors/ system_of_record/ scheduler/ apps/ # thin process entry points wiring modules together api/ # FastAPI + auto-generated Swagger / OpenAPI worker/ # executes queued jobs → dispatches to the owning module's handler scheduler/ # decides WHEN: polling + recompute triggers (enqueues only) tests/ # unit · property · integration · contract · e2e · import-boundary deploy/ # Dockerfile · docker-compose docs/ # this page + per-module DESIGN.md
Modules
Each module owns its tables and behaviour and ships its own DESIGN.md. The depends on column lists the only packages it may import — enforced in CI.
Money (Decimal), event envelope, secrets provider, protocol helpers. common/jobs holds the job envelope, status, and retry/idempotency metadata.SupplierOffer. Raw payloads are landed before normalize so feeds can be replayed.FOR UPDATE SKIP LOCKED) — it does not execute them. The worker process pulls jobs and dispatches to each module's registered handler; retry/backoff and dead-letter live in common/jobs.Documentation
Defined before implementation so every module is documented the same way. Each section is what an engineer — or an AI agent — needs to work on the module without reading the whole codebase.
# <module_name> ## Purpose # what this module is for, in two sentences ## Owns # tables, behavior, and decisions it is responsible for ## Does Not Own # explicit non-responsibilities (prevents scope creep) ## Public Contracts # interfaces it exposes in contracts/<domain> ## Events Emitted # event types + idempotency key source ## Events Consumed # event types it handles + inbox dedup rule ## Tables Owned # schema + ownership boundary ## External APIs Called # supplier / marketplace endpoints, auth, rate limits ## Config / Secrets # env + SecretsProvider keys it needs ## Invariants # statements that must always hold (encoded as tests) ## Failure Modes # what can go wrong and the expected behavior ## Retry / Idempotency # keys, backoff, dead-letter conditions ## Metrics # what it emits for observability ## Tests Required # unit / property / integration / contract / golden flow ## AI Usage Notes # how an agent should call it; safe vs unsafe operations
Sourcing
Multiple suppliers carry the same product. Each is kept as a distinct Supplier Offer under one Master Product. Sourcing uses the most recent cost and inventory, not the values at listing time.
Default offer is the active offer with the lowest cost and sufficient inventory.
If out of stock or short, fall to the next-lowest-cost offer that can fulfil.
Field exists in the model; defaults to item cost until supplier shipping data is reliable.
Chosen offer and reason (lowest_cost / fallback) are stored on the order.
Pricing engine
Price is computed once per Master Product per marketplace. Connectors publish this price; they do not compute their own. Every stage is recorded so a price can be reconstructed.
Take the cost of the offer that would actually source the product — the lowest-cost offer.
Resolve the most-specific matching rule across category × marketplace × supplier. A global default always exists so every product is priceable.
Raise to MAP floor if below, lower to MSRP ceiling if above, respect price tiers. Checked against the offer actually used for sourcing.
Optional per-platform rounding (.99 endings) and fee-aware adjustment for marketplace commissions.
If a constraint erases margin below the floor, flag and apply the configured fallback: suppress or hold for review (locked default) — never publish an invalid price.
Markup resolution
| Dimension | Example | Purpose |
|---|---|---|
| Category | Tires +22% · Apparel +40% | Reflect category margin norms |
| Marketplace | Amazon +3% · eBay +1% | Absorb platform commission differences |
| Supplier | Turn14 +18% · AutoDist +20% | Reflect supplier-specific economics |
Helmets + Amazon + Turn14 overrides Helmets + Amazon overrides global default | ||
Data model
Each entity lives in its owning module's Postgres schema. PK primary key · FK foreign key · UQ unique. Types are indicative; final DDL is the engineering team's call.
| master_product_id | uuid | PK |
| canonical_name | text | |
| brand | text | |
| mpn | text | |
| upc | text | UQ |
| description | text | |
| category | text | |
| merged_images | jsonb | |
| fitment_data | jsonb | |
| match_tier | smallint | |
| match_confidence | numeric | |
| is_active | boolean | |
| manual_override_flag | boolean |
| offer_id | uuid | PK |
| master_product_id | uuid | FK |
| supplier_id | uuid | FK |
| source_sku | text | |
| mpn | text | |
| upc | text | |
| cost | numeric(12,2) | |
| map_price | numeric(12,2) | |
| msrp_ceiling | numeric(12,2) | |
| price_tier | text | |
| quantity_available | integer | |
| source_method | enum(api,ftp) | |
| policy_effective_date | date | |
| last_synced_at | timestamptz | |
| is_active | boolean |
| rule_id | uuid | PK |
| category | text null | |
| marketplace | text null | |
| supplier_id | uuid null | FK |
| markup_type | enum(percent,fixed,target_margin) | |
| markup_value | numeric | |
| specificity_score | integer | |
| min_margin_override | numeric null | |
| effective_date | date | |
| is_active | boolean |
| price_id | uuid | PK |
| master_product_id | uuid | FK |
| marketplace | text | |
| cost_basis | numeric(12,2) | |
| markup_applied | numeric | |
| pre_clamp_price | numeric(12,2) | |
| constraint_applied | enum(none,map,msrp,tier) | |
| final_price | numeric(12,2) | |
| margin | numeric | |
| fallback_action | text | |
| computed_at | timestamptz |
| listing_id | uuid | PK |
| master_product_id | uuid | FK |
| marketplace | text | |
| platform_listing_id | text | |
| status | text | |
| listed_price | numeric(12,2) | |
| last_pushed_at | timestamptz |
| order_id | uuid | PK |
| marketplace | text | |
| platform_order_id | text | UQ |
| status | enum | |
| supplier_submission_id | text | |
| tracking_number | text | |
| created_at | timestamptz | |
| updated_at | timestamptz |
| line_item_id | uuid | PK |
| order_id | uuid | FK |
| master_product_id | uuid | FK |
| sourced_offer_id | uuid | FK |
| supplier_id | uuid | FK |
| quantity | integer | |
| unit_price | numeric(12,2) | |
| sourcing_reason | enum(lowest_cost,fallback) |
| return_id | uuid | PK |
| order_id | uuid | FK |
| fulfilling_supplier_id | uuid | FK |
| marketplace_return_id | text | |
| reason | text | |
| status | enum | |
| submitted_to_supplier_at | timestamptz | |
| resolved_at | timestamptz | |
| refund_amount | numeric(12,2) |
| event_id | uuid | PK |
| event_type | text | |
| source_system | text | |
| target_system | text | |
| payload_snapshot | jsonb | |
| status | text | |
| error_detail | text null | |
| created_at | timestamptz |
Testing
Every layer below is a CI gate. The invariants on the right are encoded as tests and must hold on every build.
Every pure domain rule — markup math, merge attribute rules, state transitions.
Pricing and merge invariants checked across generated inputs.
Real Postgres, real migrations. No mocked database for data-layer tests.
Every module interface verified both sides — provider and consumer — so a contract change can't silently break a caller.
Critical flows: ingest → merge → price → publish, and order → source → submit → return.
Fail the build if any module couples directly to another's internals.
Stack
FastAPI generates the OpenAPI / Swagger spec. Agents (Claude or OpenAI) call the platform over HTTP from that spec. No MCP dependency.
Simulators
All four suppliers are still TBD on API-vs-FTP. To build and test the whole architecture before credentials arrive, every external integration has a fake that implements the same contract as the real one — so a fake is a drop-in swap, not a separate code path.
Serves canned inventory responses and webhooks against the supplier contract.
A directory of CSV/XML feed files (including malformed ones) the FTP adapter polls.
Accepts listing pushes and replays order/return events against the marketplace contract.
Emits a realistic order event to drive the orders → sourcing → submission flow end to end.
Process roles
The same container runs in one of three roles, selected by an entrypoint flag. Monolith today; any module can be split into its own service later without code changes, because the contracts already abstract in-process versus HTTP calls.
FastAPI + Swagger. Serves the HTTP API and inbound webhooks. Stateless — scale horizontally.
Claims jobs from the Postgres queue (FOR UPDATE SKIP LOCKED) and dispatches to the owning module's handler. Retry with backoff, dead-letter after N attempts.
Decides when: polling intervals and recompute triggers. Enqueues jobs only — it never executes them.
Reliability
Applied at async and external boundaries — anything that crosses the job queue or calls an external system. Synchronous in-process contract calls inside one request do not need this.
A module updates its own tables and appends the event to its outbox in the same DB transaction. Either both commit or neither — no "DB updated but event lost".
External events use a natural key (feed file hash, webhook ID, marketplace order ID, return ID). Internal triggers (recompute, publish) use a deterministic derived key so repeats collapse.
A worker claims the job, dispatches to the owning module's registered handler, retries with exponential backoff, and dead-letters the unresolvable with an audit entry.
The consumer records the key in its inbox; a replay of the same key is a no-op. Exactly-once effect without distributed transactions.
Cross-cutting
System of record and the job queue, in one instance. Per-module schemas; no module reads another's tables.
Supplier and marketplace credentials via a SecretsProvider abstraction (env/local in dev, pluggable to a vault later). Encrypted at rest; HTTPS/SFTP only.
Structured logs (structlog) and OpenTelemetry traces, per-integration health endpoints, error alerting to ops.
Non-functional targets
| Requirement | Target |
|---|---|
| Availability | 99.9% uptime on the order-processing path |
| Price sync latency | API < 15 min · FTP within scheduled poll interval |
| Order routing latency | < 60 s from marketplace event to supplier submission |
| Pricing correctness | 100% of published prices satisfy active constraints — violations block publish |
| Scalability | Add suppliers & marketplaces with no architectural change |
| Fault tolerance | Exponential-backoff retries · dead-letter queue for unresolvable failures |
| Auditability | Full event history queryable · pricing & merge decisions reconstructable · no post-write mutation |
Build sequence
Each phase is independently shippable and fully tested before the next begins. Supplier and marketplace integrations are deliberately not touched until the foundation, audit spine, and ingestion framework exist.
uv setup · FastAPI shell · module template · testing harness · Docker · Postgres · Alembic · structured logging · event envelope · import-boundary enforcement.
Immutable event log · job table · retry/backoff · dead-letter queue · health checks. Everything built after this logs to it.
Adapter interface · FTP & API adapter bases · CSV/XML parsing · canonical Supplier Offer · raw landing zone for replay · a fake supplier for tests.
Master Product · Supplier Offer · Tier 1/2 auto-merge first · manual override model · then Tier 3 fuzzy → review queue.
Markup rule resolution · MAP/MSRP/tier clamp · margin-floor fallback · full computed-price audit breakdown.
Connector interface · BigCommerce first (primary storefront) · then eBay / Amazon / Walmart · inventory buffers · drift reconciliation.
Order state machine · lowest-cost sourcing · supplier submission interface (API/FTP) · RMA routing back to the fulfilling supplier.
Full E2E flows · performance checks · observability · admin/ops APIs · production deployment & monitoring.
Plugin interface · Google Merchant feed export using centrally-computed price · optional performance ingestion.