MODULAR MONOLITH · PYTHON 3.12 / uv · POSTGRES

Integration layer between suppliers and marketplaces.

Ingests supplier catalogs, deduplicates products into single sellable items, computes the retail price, and routes orders and returns. Every event is written to an immutable audit log. Every capability is exposed over an HTTP API.

9isolated modules
4 + 4suppliers + marketplaces
15 minprice sync SLA
100%constraint-safe prices

The shape of the system

One hub — suppliers in, marketplaces out

A quick read: data flows left → right (normalize → dedup → price → publish), orders & returns flow back, and every event lands in an immutable audit log. Full annotated diagram lives in the Architecture tab.

Suppliers
AutoDist
Turn14
PartsUnlimited
WesternPowersports
in
Core Middleware
Normalize
↓
Catalog · Dedup / Merge
↓
Pricing Engine
↓
Orders & Returns
out
Marketplaces
BigCommercegoridewell.com
eBay
Amazon
Walmart
⬓ SYSTEM OF RECORD — immutable, append-only event & audit log · every module writes here

Design principles

Constraints the codebase enforces

Module isolation

A change in one module cannot reach into another. Enforced by import-boundary tests in CI, not convention.

Contract-driven

Modules communicate only through per-domain contract packages. common holds primitives only.

Idempotency & outbox

Every external event carries an idempotency key; retries are safe. Domain change and event emit commit in one transaction via an outbox.

Testing standard

Unit, property-based, integration, contract, and end-to-end tests gate every merge. Decimal money throughout.

API-first

OpenAPI / Swagger generated from the app. Every service is an HTTP tool any agent (Claude or OpenAI) can call. No MCP dependency.

One image, two topologies

Runs as a modular monolith or split into per-module services without code changes. Plain containers.

Decisions locked for v1

What we agreed

Modular monolith first

One FastAPI app, one Postgres, one worker, one scheduler. Strict in-code boundaries so any module can later become its own service.

Thin common + per-domain contracts

Contracts live in catalog / pricing / orders / suppliers / marketplaces packages. common holds IDs, errors, config, logging, and the event envelope only.

Dedup: auto-merge Tier 1/2 only

UPC/GTIN and Brand+MPN auto-merge. Fuzzy Tier 3 goes to a human review queue at launch.

Pricing fallback = suppress / hold

When MAP/MSRP clamping erases margin, suppress the listing or hold for review. Do not silently accept a bad margin.

Item cost now, landed-cost later

The landed-cost field is in the model but defaults off until supplier shipping data is reliable.

BigCommerce first · admin APIs, no UI

Primary storefront connector first, then eBay / Amazon / Walmart. Ops actions (including manual merge/split) exposed as Swagger APIs in v1.

System architecture · data flow

How a product becomes a priced, sellable listing

The full picture. Suppliers feed in on the left; the core normalizes, merges, prices, and publishes; marketplaces receive listings on the right; orders & returns flow back; the system of record captures every event underneath.

RideWell Middleware

Integration Hub · Canonical Data Flow
RW
Inbound · Suppliers
AutoDistAPI / FTP — TBD
Turn14API / FTP — TBD
PartsUnlimitedAPI / FTP — TBD
WesternPowersportsAPI / FTP — TBD
Adapters: API · FTP · Normalizer
normalize
Core · Middleware
Normalizersupplier schema → canonical model
↓
Catalog + Dedup / MergeMaster Product · Supplier Offers
↓
Pricing Enginecost → markup → MAP/MSRP clamp
↓
Orders & Returnslowest-cost sourcing
publish
Outbound · Marketplaces
BigCommercegoridewell.com · primary
eBayREST API
AmazonSP-API
WalmartMarketplace API
Connectors translate canonical → platform
◀ Orders & returns flow back from marketplaces → validated, sourced to the lowest-cost supplier, RMAs routed to the fulfilling supplier
⬓ SYSTEM OF RECORD — immutable, append-only event & audit log · every module writes here · queryable by SKU, order, supplier, marketplace, date

Reading it: an offer is normalized to the canonical model, merged into a Master Product, priced once by the central engine, then published to every active marketplace. No connector ever computes price. Sourcing is decided at order time from the freshest cost & inventory.

The isolation guarantee

Why a change in one module can't break another

1
Thin common, never fat

Only true primitives: IDs, base DTO patterns, error types, config, logging, auth, the event envelope, protocol helpers. Nothing domain-specific.

2
Explicit contract packages

Each domain publishes its interface in contracts/<domain>. Modules depend on contracts, not on each other's code.

3
Dependency inversion

In monolith mode, contracts resolve to in-process calls. Split a module out and the same contract is backed by an HTTP client. No rewrite.

4
Per-module table ownership

Separate Postgres schemas per module. No module reads another's tables — only its contract or its events.

5
Import-boundary tests in CI

An automated test fails the build if, say, pricing imports catalog internals. The boundary is enforced, not hoped for.

# Dependency direction — always inward, never sideways

  modules/pricing
      │  depends on
      ▼
  contracts/pricing   # interfaces + events
  contracts/catalog   # reads offers via contract
      │
      ▼
  common              # primitives only

# ❌ what CI forbids
from ridewell.modules.catalog._internal import ...

# ✅ what's allowed
from ridewell.contracts.catalog import CatalogQuery
from ridewell.common.money import Money

Repository layout

One monorepo, clear physical structure

src/ridewell/
  common/              # primitives: ids, errors, config, logging, money, event envelope, secrets
    jobs/              # shared job envelope + status + retry / idempotency metadata
  contracts/           # interfaces + domain events — the only cross-module language
    catalog/  pricing/  orders/  suppliers/  marketplaces/
  modules/             # isolated domains — each owns its tables, job handlers, DESIGN.md, tests
    supplier_adapters/  catalog/  pricing/  orders/
    marketplace_connectors/  system_of_record/  scheduler/
  apps/                # thin process entry points wiring modules together
    api/        # FastAPI + auto-generated Swagger / OpenAPI
    worker/     # executes queued jobs → dispatches to the owning module's handler
    scheduler/  # decides WHEN: polling + recompute triggers (enqueues only)
tests/                # unit · property · integration · contract · e2e · import-boundary
deploy/               # Dockerfile · docker-compose
docs/                 # this page + per-module DESIGN.md

Modules

Nine modules, one shared contract layer

Each module owns its tables and behaviour and ships its own DESIGN.md. The depends on column lists the only packages it may import — enforced in CI.

common
depends on: nothing
Shared primitives only: IDs, base DTOs, error types, config, structured logging, Money (Decimal), event envelope, secrets provider, protocol helpers. common/jobs holds the job envelope, status, and retry/idempotency metadata.
contracts/*
common
Per-domain interface and event packages — the only language modules use to talk to each other. Pydantic models and Protocols; no heavy dependencies.
supplier_adapters
commonsuppliers
API and FTP adapter bases, per-supplier driver plugins, CSV/XML parsing, the Normalizer that emits canonical SupplierOffer. Raw payloads are landed before normalize so feeds can be replayed.
rapidfuzz · polars · defusedxml / lxml
catalog
commoncatalogsuppliers
Master Product store, SKU dedup/merge engine (Tier 1/2/3), attribute-merge rules, manual merge/split overrides that survive re-sync.
pricing
commonpricingcatalog
Five-stage pricing pipeline, markup resolution by specificity, MAP/MSRP/tier clamp, margin-floor fallback, full computed-price breakdown for audit.
orders
commonorderscatalog
Purchase lifecycle state machine, lowest-cost sourcing decision, supplier submission (API/FTP), RMA routing back to the fulfilling supplier.
marketplace_connectors
commonmarketplacespricing
BigCommerce, eBay, Amazon, Walmart connectors. Canonical-to-platform translation, inventory buffer, publish-only price, drift reconciliation loop.
httpx · tenacity
system_of_record
common
Immutable, append-only event and audit log. Owns audit records only. Queryable by SKU, master product, order, supplier, marketplace, date, and status.
scheduler
common
Decides when work runs: polling intervals and recompute triggers. Enqueues jobs onto the Postgres-backed queue (FOR UPDATE SKIP LOCKED) — it does not execute them. The worker process pulls jobs and dispatches to each module's registered handler; retry/backoff and dead-letter live in common/jobs.
tenacity
ads_social fast follow
commoncatalogpricing
Plugin interface for Google Ads / Meta. Exports the catalog in Google Merchant feed format using the centrally-computed price. Performance ingestion optional.

Documentation

Per-module DESIGN.md template

Defined before implementation so every module is documented the same way. Each section is what an engineer — or an AI agent — needs to work on the module without reading the whole codebase.

# <module_name>

## Purpose            # what this module is for, in two sentences
## Owns               # tables, behavior, and decisions it is responsible for
## Does Not Own       # explicit non-responsibilities (prevents scope creep)
## Public Contracts   # interfaces it exposes in contracts/<domain>
## Events Emitted     # event types + idempotency key source
## Events Consumed    # event types it handles + inbox dedup rule
## Tables Owned       # schema + ownership boundary
## External APIs Called # supplier / marketplace endpoints, auth, rate limits
## Config / Secrets    # env + SecretsProvider keys it needs
## Invariants         # statements that must always hold (encoded as tests)
## Failure Modes      # what can go wrong and the expected behavior
## Retry / Idempotency # keys, backoff, dead-letter conditions
## Metrics            # what it emits for observability
## Tests Required     # unit / property / integration / contract / golden flow
## AI Usage Notes     # how an agent should call it; safe vs unsafe operations

Sourcing

Sourcing is decided at order time

Multiple suppliers carry the same product. Each is kept as a distinct Supplier Offer under one Master Product. Sourcing uses the most recent cost and inventory, not the values at listing time.

Lowest cost

Default offer is the active offer with the lowest cost and sufficient inventory.

Fallback chain

If out of stock or short, fall to the next-lowest-cost offer that can fulfil.

Landed cost

Field exists in the model; defaults to item cost until supplier shipping data is reliable.

Recorded

Chosen offer and reason (lowest_cost / fallback) are stored on the order.

Pricing engine

Five stages, in order

Price is computed once per Master Product per marketplace. Connectors publish this price; they do not compute their own. Every stage is recorded so a price can be reconstructed.

1
Select cost basis

Take the cost of the offer that would actually source the product — the lowest-cost offer.

2
Apply dynamic markup

Resolve the most-specific matching rule across category × marketplace × supplier. A global default always exists so every product is priceable.

3
Apply constraint clamp

Raise to MAP floor if below, lower to MSRP ceiling if above, respect price tiers. Checked against the offer actually used for sourcing.

4
Marketplace rounding & fees

Optional per-platform rounding (.99 endings) and fee-aware adjustment for marketplace commissions.

5
Validate & publish

If a constraint erases margin below the floor, flag and apply the configured fallback: suppress or hold for review (locked default) — never publish an invalid price.

Markup resolution

Most-specific rule wins

DimensionExamplePurpose
CategoryTires +22% · Apparel +40%Reflect category margin norms
MarketplaceAmazon +3% · eBay +1%Absorb platform commission differences
SupplierTurn14 +18% · AutoDist +20%Reflect supplier-specific economics
Helmets + Amazon + Turn14  overrides  Helmets + Amazon  overrides  global default

Data model

Entities, types, and keys

Each entity lives in its owning module's Postgres schema. PK primary key · FK foreign key · UQ unique. Types are indicative; final DDL is the engineering team's call.

master_productcatalog
master_product_iduuidPK
canonical_nametext
brandtext
mpntext
upctextUQ
descriptiontext
categorytext
merged_imagesjsonb
fitment_datajsonb
match_tiersmallint
match_confidencenumeric
is_activeboolean
manual_override_flagboolean
supplier_offercatalog
offer_iduuidPK
master_product_iduuidFK
supplier_iduuidFK
source_skutext
mpntext
upctext
costnumeric(12,2)
map_pricenumeric(12,2)
msrp_ceilingnumeric(12,2)
price_tiertext
quantity_availableinteger
source_methodenum(api,ftp)
policy_effective_datedate
last_synced_attimestamptz
is_activeboolean
pricing_rulepricing
rule_iduuidPK
categorytext null
marketplacetext null
supplier_iduuid nullFK
markup_typeenum(percent,fixed,target_margin)
markup_valuenumeric
specificity_scoreinteger
min_margin_overridenumeric null
effective_datedate
is_activeboolean
computed_pricepricing
price_iduuidPK
master_product_iduuidFK
marketplacetext
cost_basisnumeric(12,2)
markup_appliednumeric
pre_clamp_pricenumeric(12,2)
constraint_appliedenum(none,map,msrp,tier)
final_pricenumeric(12,2)
marginnumeric
fallback_actiontext
computed_attimestamptz
marketplace_listingconnectors
listing_iduuidPK
master_product_iduuidFK
marketplacetext
platform_listing_idtext
statustext
listed_pricenumeric(12,2)
last_pushed_attimestamptz
orderorders
order_iduuidPK
marketplacetext
platform_order_idtextUQ
statusenum
supplier_submission_idtext
tracking_numbertext
created_attimestamptz
updated_attimestamptz
order_line_itemorders
line_item_iduuidPK
order_iduuidFK
master_product_iduuidFK
sourced_offer_iduuidFK
supplier_iduuidFK
quantityinteger
unit_pricenumeric(12,2)
sourcing_reasonenum(lowest_cost,fallback)
returnorders
return_iduuidPK
order_iduuidFK
fulfilling_supplier_iduuidFK
marketplace_return_idtext
reasontext
statusenum
submitted_to_supplier_attimestamptz
resolved_attimestamptz
refund_amountnumeric(12,2)
event_log  · append-onlysystem_of_record
event_iduuidPK
event_typetext
source_systemtext
target_systemtext
payload_snapshotjsonb
statustext
error_detailtext null
created_attimestamptz

Foreign keys

supplier_offer.master_product_id → master_product.master_product_id
computed_price.master_product_id → master_product.master_product_id
marketplace_listing.master_product_id → master_product.master_product_id
order_line_item.order_id → order.order_id
order_line_item.sourced_offer_id → supplier_offer.offer_id
return.order_id → order.order_id
order.status: received → validated → sourced → submitted → acknowledged → shipped → completed

Testing

Test layers and invariants

Every layer below is a CI gate. The invariants on the right are encoded as tests and must hold on every build.

U
Unit tests

Every pure domain rule — markup math, merge attribute rules, state transitions.

P
Property-based (Hypothesis)

Pricing and merge invariants checked across generated inputs.

I
Integration (testcontainers)

Real Postgres, real migrations. No mocked database for data-layer tests.

C
Contract tests

Every module interface verified both sides — provider and consumer — so a contract change can't silently break a caller.

E
End-to-end

Critical flows: ingest → merge → price → publish, and order → source → submit → return.

B
Import-boundary tests

Fail the build if any module couples directly to another's internals.

Critical invariants — always true

✓Marketplace connectors never compute price.
✓Every published price satisfies active supplier constraints.
✓A MAP/MSRP conflict blocks publish — never an invalid price.
✓Order sourcing uses the latest available offer, not a stale one.
✓Manual merge/split overrides survive re-sync.
✓Audit events are append-only — no mutation post-write.
✓Failed jobs retry, then dead-letter; handlers are idempotent.
✓Every external event carries an idempotency key — replays are no-ops.
✓Domain change + event emit commit in one transaction (outbox).
✓Money is always Decimal, never float.

Stack

Libraries

Runtime

Python 3.12+uvFastAPIPydantic v2

Data

PostgreSQLSQLAlchemy 2Alembicpolars

Integration

httpxtenacityrapidfuzzdefusedxml / lxml

Quality & ops

pytestHypothesistestcontainersstructlogOpenTelemetry
API surface

FastAPI generates the OpenAPI / Swagger spec. Agents (Claude or OpenAI) call the platform over HTTP from that spec. No MCP dependency.

Simulators

Fakes that implement the real contracts

All four suppliers are still TBD on API-vs-FTP. To build and test the whole architecture before credentials arrive, every external integration has a fake that implements the same contract as the real one — so a fake is a drop-in swap, not a separate code path.

Fake supplier API

Serves canned inventory responses and webhooks against the supplier contract.

Fake supplier FTP

A directory of CSV/XML feed files (including malformed ones) the FTP adapter polls.

Fake marketplace

Accepts listing pushes and replays order/return events against the marketplace contract.

Fake BigCommerce order

Emits a realistic order event to drive the orders → sourcing → submission flow end to end.

Process roles

One image, three process roles

The same container runs in one of three roles, selected by an entrypoint flag. Monolith today; any module can be split into its own service later without code changes, because the contracts already abstract in-process versus HTTP calls.

api

FastAPI + Swagger. Serves the HTTP API and inbound webhooks. Stateless — scale horizontally.

worker

Claims jobs from the Postgres queue (FOR UPDATE SKIP LOCKED) and dispatches to the owning module's handler. Retry with backoff, dead-letter after N attempts.

scheduler

Decides when: polling intervals and recompute triggers. Enqueues jobs only — it never executes them.

Reliability

Outbox, inbox, and idempotency

Applied at async and external boundaries — anything that crosses the job queue or calls an external system. Synchronous in-process contract calls inside one request do not need this.

1
Domain write + outbox insert (one transaction)

A module updates its own tables and appends the event to its outbox in the same DB transaction. Either both commit or neither — no "DB updated but event lost".

2
Enqueue with an idempotency key

External events use a natural key (feed file hash, webhook ID, marketplace order ID, return ID). Internal triggers (recompute, publish) use a deterministic derived key so repeats collapse.

3
Worker executes the handler

A worker claims the job, dispatches to the owning module's registered handler, retries with exponential backoff, and dead-letters the unresolvable with an audit entry.

4
Inbox dedup on consume

The consumer records the key in its inbox; a replay of the same key is a no-op. Exactly-once effect without distributed transactions.

Cross-cutting

Data, secrets, and observability

PostgreSQL

System of record and the job queue, in one instance. Per-module schemas; no module reads another's tables.

Secrets

Supplier and marketplace credentials via a SecretsProvider abstraction (env/local in dev, pluggable to a vault later). Encrypted at rest; HTTPS/SFTP only.

Observability

Structured logs (structlog) and OpenTelemetry traces, per-integration health endpoints, error alerting to ops.

Non-functional targets

Targets

RequirementTarget
Availability99.9% uptime on the order-processing path
Price sync latencyAPI < 15 min · FTP within scheduled poll interval
Order routing latency< 60 s from marketplace event to supplier submission
Pricing correctness100% of published prices satisfy active constraints — violations block publish
ScalabilityAdd suppliers & marketplaces with no architectural change
Fault toleranceExponential-backoff retries · dead-letter queue for unresolvable failures
AuditabilityFull event history queryable · pricing & merge decisions reconstructable · no post-write mutation

Build sequence

Foundation-first. No big bang.

Each phase is independently shippable and fully tested before the next begins. Supplier and marketplace integrations are deliberately not touched until the foundation, audit spine, and ingestion framework exist.

P0 Foundation

Repo skeleton and tooling

uv setup · FastAPI shell · module template · testing harness · Docker · Postgres · Alembic · structured logging · event envelope · import-boundary enforcement.

greenapp boots in all 3 roles; a sample job round-trips the queue.failurean illegal cross-module import fails the boundary test in CI.
commoncontractsappsCI
P1 Spine

System of Record + Scheduler

Immutable event log · job table · retry/backoff · dead-letter queue · health checks. Everything built after this logs to it.

greenan event is written and queryable; a job retries then dead-letters.failurea duplicate event key is ignored (inbox dedup).
system_of_recordscheduler
P2 Ingestion

Supplier ingestion framework

Adapter interface · FTP & API adapter bases · CSV/XML parsing · canonical Supplier Offer · raw landing zone for replay · a fake supplier for tests.

greenCSV feed lands, normalizes, stores supplier offers, logs event.failuremalformed CSV dead-letters with an audit entry.
supplier_adapters
P3 Catalog

Master Product & dedup

Master Product · Supplier Offer · Tier 1/2 auto-merge first · manual override model · then Tier 3 fuzzy → review queue.

greentwo offers with the same UPC merge into one Master Product.failurea Tier 3 fuzzy match is held for review, not auto-merged.
catalog
P4 Pricing

Pricing engine

Markup rule resolution · MAP/MSRP/tier clamp · margin-floor fallback · full computed-price audit breakdown.

greencost + markup + MAP clamp produces a valid published price.failureMAP > MSRP blocks publish and flags for review.
pricing
P5 Publish

Marketplace publishing

Connector interface · BigCommerce first (primary storefront) · then eBay / Amazon / Walmart · inventory buffers · drift reconciliation.

greena price/inventory change reaches BigCommerce within the sync SLA.failuredrift (live ≠ published) is detected and corrected.
marketplace_connectors
P6 Orders

Orders & returns

Order state machine · lowest-cost sourcing · supplier submission interface (API/FTP) · RMA routing back to the fulfilling supplier.

greena marketplace order routes to the lowest-cost in-stock supplier.failurelowest-cost offer out of stock → falls back to the next offer.
orders
P7 Hardening

End-to-end & launch

Full E2E flows · performance checks · observability · admin/ops APIs · production deployment & monitoring.

greenfull ingest→price→publish and order→return pass end-to-end.failurea forced supplier outage degrades gracefully (dead-letter + alert).
launch
P8 Fast follow

Ad & social platform integrations

Plugin interface · Google Merchant feed export using centrally-computed price · optional performance ingestion.

ads_social