Skip to content

Performance Implementation Backlog - 2026-02-08

Purpose

Turn the performance baseline into an execution backlog for Day 1 migration delivery.

This plan is constrained by existing architecture decisions:

  1. Max 5 deployables (webshop, stock-import, platform-export, vehicle-data, erp-middleware).
  2. Max 3 internal shared packages (atraxion/contracts, atraxion/integration-sdk, atraxion/foundation).
  3. Projection-first reads for stock, fitment, and wheel sets.
  4. Async-first write propagation with outbox/inbox and idempotency.

References: - docs/migration/02-island-boundaries-and-performance.md - docs/migration/webshop/02-products.md - docs/migration/vehicle-data/02-vehicle-product-mapping.md - docs/migration/stock-import/00-stock-import-overview.md - docs/migration/platform-export/00-platform-export-overview.md

Scope And Targets

Data scale assumptions:

  • ~285,000 products
  • ~20,000 vehicle variants
  • ~50 suppliers
  • ~60 platforms
  • composite sets without native SKU

Day 1 SLI targets:

  1. Product/search read p95: < 200 ms
  2. Vehicle-to-product lookup p95: < 250 ms
  3. Set list/detail + wheel-set API p95: < 300 ms
  4. Projection freshness lag (stock/fitment/sets): < 5 min
  5. Projection rebuild failure rate: < 0.5% daily

Tooling Decisions (Day 1)

  1. Queue and async compute: Symfony Messenger + RabbitMQ transport.
  2. Cache and locks: Redis (cache + lock store).
  3. Persistence: MySQL/MariaDB (single family across islands).
  4. Profiling and performance regression checks: Blackfire + Symfony profiler.
  5. Metrics/alerting: Prometheus metrics + Grafana dashboards + Sentry for exception telemetry.

Defer until metrics prove need:

  1. OpenSearch/Elasticsearch for read/search offload.
  2. Additional deployables for compute-only services.

Work Packages

ID Priority Island(s) Deliverable Dependencies
PERF-001 H All Performance instrumentation baseline and dashboards none
PERF-002 H Webshop, Vehicle Data Projection schema and index hardening PERF-001
PERF-003 H Stock Import, Webshop Stock projection pipeline (incremental + reconcile) PERF-002
PERF-004 H Vehicle Data Fitment projection pipeline (incremental) PERF-002
PERF-005 H Webshop, Vehicle Data Wheel-set projection pipeline + component reverse index PERF-002, PERF-004
PERF-006 H All integration islands Queue topology, retry/backoff, DLQ, replay commands PERF-001
PERF-007 M All Worker sizing baseline + autoscaling policy PERF-006
PERF-008 H Webshop, Vehicle Data Dual-read/dual-write validation and drift detection PERF-003, PERF-004, PERF-005
PERF-009 M Webshop API/query optimization and cache policy hardening PERF-008
PERF-010 M Shared Ops Failure runbooks, replay tooling, reconciliation SOPs PERF-006, PERF-008
PERF-011 M All Load tests and failure-injection tests PERF-007, PERF-008
PERF-012 H All Cutover gates and go/no-go check PERF-008, PERF-011

Schema And Index Specs

Use narrow projection tables with explicit compound indexes.

1) article_availability_projection

CREATE TABLE article_availability_projection (
    article_number VARCHAR(50) PRIMARY KEY,
    sellable_qty INT NOT NULL DEFAULT 0,
    best_price DECIMAL(10,2) NOT NULL,
    next_delivery_date DATE NULL,
    source_batch_id VARCHAR(64) NULL,
    updated_at DATETIME(3) NOT NULL,
    INDEX idx_aap_qty_updated (sellable_qty, updated_at),
    INDEX idx_aap_updated (updated_at)
);

2) product_vehicle_mappings

Use composite primary key to avoid surrogate-id lookups on hot paths.

CREATE TABLE product_vehicle_mappings (
    vehicle_variant_id INT NOT NULL,
    article_number VARCHAR(50) NOT NULL,
    product_type VARCHAR(10) NOT NULL,
    position VARCHAR(10) NOT NULL DEFAULT 'both',
    is_original_equipment BOOLEAN NOT NULL DEFAULT FALSE,
    match_score SMALLINT NOT NULL,
    computed_at DATETIME(3) NOT NULL,
    PRIMARY KEY (vehicle_variant_id, article_number),
    INDEX idx_pvm_article_type_score (article_number, product_type, match_score),
    INDEX idx_pvm_vehicle_type_score (vehicle_variant_id, product_type, match_score)
);

3) wheel_set_projection

CREATE TABLE wheel_set_projection (
    set_key CHAR(64) PRIMARY KEY,
    vehicle_variant_id INT NOT NULL,
    season VARCHAR(20) NOT NULL,
    mounting_type VARCHAR(20) NOT NULL,
    wheel_front_article_number VARCHAR(50) NOT NULL,
    wheel_rear_article_number VARCHAR(50) NULL,
    tyre_front_article_number VARCHAR(50) NOT NULL,
    tyre_rear_article_number VARCHAR(50) NULL,
    tpms_article_number VARCHAR(50) NULL,
    sellable_quantity INT NOT NULL,
    set_price DECIMAL(10,2) NOT NULL,
    computed_at DATETIME(3) NOT NULL,
    INDEX idx_wsp_vehicle_season_qty_price (vehicle_variant_id, season, sellable_quantity, set_price),
    INDEX idx_wsp_computed (computed_at)
);

4) wheel_set_component_index

Reverse index to update impacted sets when one component changes.

CREATE TABLE wheel_set_component_index (
    component_article_number VARCHAR(50) NOT NULL,
    component_role VARCHAR(20) NOT NULL,
    set_key CHAR(64) NOT NULL,
    PRIMARY KEY (component_article_number, component_role, set_key),
    INDEX idx_wsci_set_key (set_key)
);

5) projection_job_checkpoint

CREATE TABLE projection_job_checkpoint (
    job_name VARCHAR(100) PRIMARY KEY,
    scope_key VARCHAR(100) NOT NULL,
    last_success_cursor VARCHAR(255) NULL,
    last_success_batch_id VARCHAR(64) NULL,
    last_success_at DATETIME(3) NULL,
    updated_at DATETIME(3) NOT NULL,
    UNIQUE KEY uq_pjc_job_scope (job_name, scope_key)
);

Compute Strategy For Vehicle Mapping And Sets

Vehicle Mapping (Tyre/Wheel -> Vehicle)

Do not do full Cartesian recomputes (all vehicles x all products).

Incremental approach:

  1. Normalize vehicle rules into fitment buckets:
  2. tyre_bucket = width:aspect:diameter:position
  3. wheel_bucket = pcd:diameter:center_bore_floor:offset_range
  4. Normalize products into the same bucket keys.
  5. Maintain bucket membership tables for vehicles and products.
  6. Recompute only affected key intersections when either side changes.

Trigger sources:

  1. Vehicle source sync updates.
  2. Manual fitment changes.
  3. Product spec changes.

Wheel Set Composition

  1. Build candidate sets from fitment-compatible wheel/tyre buckets only.
  2. Generate deterministic set_key.
  3. Compute quantity via projection stock snapshot:
  4. standard: floor(min(wheel_qty, tyre_qty) / 4)
  5. staggered: min(floor(min(front)/2), floor(min(rear)/2))
  6. Store set in wheel_set_projection and component links in wheel_set_component_index.
  7. On component stock/price change, fetch affected set_key values via reverse index and recompute only those sets.

Queue Topology And Policies

Queue classes remain high-priority and bulk, with explicit per-flow queues.

Suggested queue names

Queue Class Purpose
webshop.high-priority high-priority order/payment/customer critical updates
webshop.bulk bulk set/stock projection recompute jobs
vehicle-data.high-priority high-priority lookup-critical fitment refresh notifications
vehicle-data.bulk bulk mapping recompute/backfill
stock-import.high-priority high-priority incremental supplier corrections
stock-import.bulk bulk full supplier feed loads
platform-export.high-priority high-priority order import/tracking updates
platform-export.bulk bulk stock export generation
*.dlq dlq exhausted retries for operator replay

Retry defaults

  1. Temporary errors: exponential backoff (15s, 60s, 300s, 900s, 1800s).
  2. Permanent validation errors: no blind retry, route to DLQ/review.
  3. Max attempts: 5 (override by connector if required).
  4. Idempotency key required on all cross-island write events/messages.

Replay and reconciliation commands

  1. bin/console queue:replay --queue=<dlq> --from=<ts> --to=<ts> [--scope=<id>]
  2. bin/console projection:reconcile --domain=stock|fitment|sets --since=<ts>
  3. bin/console projection:rebuild --domain=stock|fitment|sets --scope=<id|all>

Worker Sizing Baseline (Initial)

Initial production baseline (before autoscaling):

Island Queue Workers Concurrency/worker Notes
Webshop webshop.high-priority 4 1 protect latency-sensitive domain writes
Webshop webshop.bulk 8 1 projection recompute throughput
Vehicle Data vehicle-data.high-priority 2 1 fast fitment signal propagation
Vehicle Data vehicle-data.bulk 8 1 fitment mapping recompute
Stock Import stock-import.high-priority 3 1 urgent correction batches
Stock Import stock-import.bulk 6 1 supplier full imports
Platform Export platform-export.high-priority 3 1 order/tracking pushes
Platform Export platform-export.bulk 6 1 export feed generation

Autoscaling triggers (per queue):

  1. p95 queue wait time > 60s for 10 min: scale out +2 workers.
  2. queue depth > 10,000 for 15 min: scale out +4 workers.
  3. error rate > 2% for 10 min: freeze scale-out, page on-call, inspect DLQ.

Observability And SLI Dashboard

Required metrics:

  1. API latency: p50/p95/p99 by endpoint and tenant.
  2. Projection freshness lag: event time to projection commit time.
  3. Queue lag: enqueue-to-start and enqueue-to-ack.
  4. Recompute throughput: rows/s by projection domain.
  5. Recompute failure rate and DLQ depth.
  6. Duplicate suppression rate by idempotency key.
  7. Stock-set consistency drift count.

Minimum dashboards:

  1. perf-storefront-latency
  2. perf-projection-freshness
  3. perf-queue-health
  4. perf-fitment-and-sets-compute
  5. perf-connector-runtime

Alert thresholds:

  1. set API p95 > 300 ms for 15 min.
  2. fitment lookup p95 > 250 ms for 15 min.
  3. projection lag > 5 min for 10 min.
  4. any DLQ depth > 500 for 10 min.
  5. daily rebuild failure rate > 0.5%.

Delivery Phases And Gates

Phase P0 - Baseline And Instrumentation

Includes: PERF-001, partial PERF-006

Gate G0:

  1. Dashboards live in staging.
  2. All message producers emit idempotency keys.
  3. Queue lag and API latency metrics verified with test traffic.

Phase P1 - Projection Storage Hardening

Includes: PERF-002

Gate G1:

  1. Projection tables/indexes migrated.
  2. Explain plans for top 10 read queries are within budget.
  3. No full table scan on critical read endpoints.

Phase P2 - Incremental Compute Pipelines

Includes: PERF-003, PERF-004, PERF-005, PERF-006

Gate G2:

  1. Incremental triggers implemented for stock/fitment/sets.
  2. Reverse component index for set recompute active.
  3. Replay tooling validated against synthetic failure cases.

Phase P3 - Dual-Read Validation

Includes: PERF-008

Gate G3:

  1. Dual-read enabled for stock/fitment/set APIs.
  2. Drift reports under thresholds:
  3. stock mismatch < 0.1%
  4. fitment mismatch < 0.5%
  5. set quantity mismatch < 0.5%
  6. Drift reconciliation job and runbook validated.

Phase P4 - Tuning And Load Validation

Includes: PERF-007, PERF-009, PERF-011

Gate G4:

  1. Load tests at >= 1.5x expected peak pass SLI targets.
  2. Queue backlog drains within 30 min after peak bursts.
  3. No unbounded memory growth in long-running workers.

Phase P5 - Cutover

Includes: PERF-010, PERF-012

Gate G5 (Go/No-Go):

  1. All H-priority work packages complete.
  2. SLIs stable for 7 consecutive days in pre-prod/prod canary.
  3. DLQ replay and rollback drills completed.
  4. Architecture gates still pass (deployable/package caps unchanged).

Backlog Details (Definition Of Done)

PERF-001 Instrumentation Baseline

DoD:

  1. Metrics and traces emitted from all islands.
  2. Correlation id propagated across sync+async flows.
  3. SLI dashboards and alerts configured.

PERF-002 Projection Schema Hardening

DoD:

  1. Projection DDL applied with indexes.
  2. Query plans captured and documented.
  3. Migration rollback script tested.

PERF-003 Stock Projection Pipeline

DoD:

  1. Stock events update article_availability_projection incrementally.
  2. Reconciliation command catches and fixes drift.
  3. p95 product search meets < 200 ms in load tests.

PERF-004 Fitment Projection Pipeline

DoD:

  1. Bucket-based incremental recompute implemented.
  2. Vehicle and product change triggers wired.
  3. p95 fitment lookup meets < 250 ms.

PERF-005 Wheel Set Projection Pipeline

DoD:

  1. Set projection uses deterministic key and reverse component index.
  2. Standard and staggered quantity logic covered by tests.
  3. p95 set endpoints meet < 300 ms.

PERF-006 Queue Reliability Contract

DoD:

  1. Retry/backoff and DLQ behavior implemented for all connector classes.
  2. Replay command operational with scope filters.
  3. Duplicate-delivery tests pass.

PERF-007 Worker Sizing And Scaling

DoD:

  1. Baseline worker counts configured per queue.
  2. Scaling rules implemented and tested under load.
  3. Worker saturation runbook written.

PERF-008 Dual-Read And Drift Validation

DoD:

  1. API dual-read comparison runs continuously.
  2. Drift reports emitted daily.
  3. Drift thresholds are below gate limits.

PERF-009 Read Path Optimization

DoD:

  1. API and SQL hot paths profiled and optimized.
  2. Cache strategy documented per endpoint.
  3. No critical endpoint exceeds p95 target in soak test.

PERF-010 Runbooks

DoD:

  1. DLQ replay runbook per island exists.
  2. Projection reconciliation runbook exists.
  3. On-call escalation matrix is documented.

PERF-011 Load/Failure Tests

DoD:

  1. Import burst tests, compute burst tests, and outage tests pass.
  2. Failure scenarios validated (broker outage, DB failover, upstream 429/5xx).
  3. Results attached to release evidence.

PERF-012 Cutover Governance

DoD:

  1. Go/no-go checklist signed by domain and ops owners.
  2. Rollback steps tested and time-bounded.
  3. First 48-hour hypercare staffing confirmed.

Risks And Mitigations

Risk Impact Mitigation
Projection drift during high write bursts wrong stock/fitment/set results dual-read drift monitor + reconcile command + replay
Queue saturation in bulk windows stale data autoscaling + separate high-priority queues + backpressure
Expensive set recompute fan-out high CPU and lag reverse component index + bucketed candidate generation
Connector retry storms cascading failures capped retries, jittered backoff, circuit-break policy
Package/service sprawl pressure architecture erosion enforce ADR gates from 03-architecture-review-checklist-template.md

Acceptance Scenarios (Gherkin)

Feature: Performance migration backlog execution

  Scenario: Component stock update recomputes only impacted sets
    Given component article "WHEEL-123" is linked in wheel_set_component_index
    When stock for "WHEEL-123" changes
    Then only linked set_key projections should be recomputed
    And unrelated sets should not be recalculated

  Scenario: Fitment recompute runs incrementally by bucket
    Given a fitment rule change only affects tyre bucket "225:45:17:both"
    When fitment recompute is triggered
    Then only product and vehicle records in that bucket are recomputed

  Scenario: Queue backlog scales workers
    Given bulk queue wait time exceeds 60 seconds for 10 minutes
    When autoscaling policy evaluates queue health
    Then worker count should scale out according to policy

  Scenario: Dual-read gate blocks cutover on excessive drift
    Given set quantity drift is above 0.5 percent
    When cutover gate G3 is evaluated
    Then cutover should be blocked
    And reconciliation actions should be required