Module 17: The Proving Grounds (Capstone Architectural Simulation)

PREREQUISITE STATEMENT: Read this module after completing all prior 16 modules. Having mastered code-level design, database internals, caching topologies, distributed routing, and architectural governance, you are ready to apply this knowledge to solve three complex enterprise system designs.

1. Introduction: The Capstone Challenge

Architectural competence is not measured by memorizing terminology or multiple-choice exams. It is proven by design execution. The Proving Grounds present three real-world enterprise scenarios. You must select one, model its end-to-end topology in the interactive sandbox editor, and draft a Software Design Specification (SDS) documenting your trade-offs, constraints, and failure recovery playbooks.

Your submission will be evaluated against five architectural metrics:

1. Correctness: Does the design satisfy all business rules and consistency requirements?
2. Resiliency: How does the system handle component failures, network splits, and resource saturation?
3. Scale Capability: Is the infrastructure partitioned horizontally to avoid single-point scaling limits?
4. Cost Efficiency (FinOps): Are resources budgeted appropriately, avoiding over-provisioned cluster configurations?
5. Design Clarity: Are service boundaries, data directions, and protocol handshakes clearly documented?

2. Capstone Project Choices

Select and complete one of the following three industrial scenarios:

Challenge 17.1: The Greenfield Ticket Multi-Processor

[ Client Traffic ] ---> [ Edge Rate Limiter ] ---> [ Ingestion Queue (Kafka) ] 
                                                          |
                                                    (Worker Pool)
                                                          |
                                                          v
                                            [ Redis Distributed Lock ]
                                                          |
                                                (Write to Postgres DB)

1. The Scenario & Scale

An entertainment platform is launching sales for a global stadium concert.

• Scale: 100,000+ requests per second are expected at the millisecond sales open.
• Inventory: The stadium has a capacity of exactly 80,000 seats.
• Constraint: Zero overallocations or double-bookings are permitted. The system must remain available to clients under peak load, degrade gracefully, and enforce strict correctness.

2. Architectural Blueprint Requirements

• Traffic Ingress: Place an Edge Rate Limiter to drop malicious DDoS and duplicate bots.
• Ingestion Queue Gate: Do not allow client connections to access the database directly. Route requests into an Ingestion Queue (e.g. Apache Kafka or AWS SQS) to buffer traffic, converting a high-concurrency write spike into a flat, predictable consumer load.
• Distributed Inventory Lock: Implement a Redis Distributed Lock (using the Redlock algorithm or optimistic concurrency locking via Lua scripts) to validate seat availability before executing database writes:
  • Lock key format: lock:seat:<seat_number>.
  • Set a short Time-To-Live (TTL) on the lock (e.g., 10 minutes) representing the user checkout safety window. If checkout is not completed, the lock expires automatically, returning the seat to inventory.
• Finalized Purchase Ledger: Use a relational database (PostgreSQL/Postgres with transaction isolation levels set to REPEATABLE READ or SERIALIZABLE) to store finalized checkout records and ledger balances, ensuring ACID compliance.

Challenge 17.2: The Brownfield Banking Ledger Decoupling

                      [ Strangler Routing Gateway ]
                               /         \
                 (Legacy Path)             (Decoupled Path)
                     /                             \
        [ Legacy Core Monolith ]            [ Account Microservice ]
                                                   |
                                            (Saga Orchestrator)
                                                   |
                                            [ Transaction Service ]

1. The Scenario & Scale

A commercial bank is modernizing its core banking ledger.

• Scale: The system processes 5,000 transactions per second.
• Constraints: The legacy monolith database must remain online during the migration. The new architecture must decouple the system into autonomous microservices (AccountService, TransactionService, AuditLedgerService) while guaranteeing 100% data consistency and zero lost operations.

2. Architectural Blueprint Requirements

• Strangler Fig Routing: Deploy a Routing Gateway in front of the banking core. Intercept incoming queries and route new account features to the decoupled services while leaving old savings operations on the legacy core.
• Orchestrated Saga Pattern: Implement a Saga Orchestrator to coordinate distributed funds transfers between decoupled services:
  1. Action 1: AccountService debits Account A (Pending state).
  2. Action 2: TransactionService credits Account B (Pending state).
  3. Action 3: AuditLedgerService logs the transfer.
  4. If all actions succeed, transition account balances to Committed.
  5. If Action 2 fails (e.g., Account B is closed), the orchestrator triggers Compensating Transactions (executes a credit back to Account A to restore state).
• Anti-Corruption Layer (ACL): Build an ACL to translate binary EBCDIC mainframe data structures from the legacy core into clean JSON/Protobuf domain models used by the new microservices.

Challenge 17.3: The Global Video Streaming Delivery Fabric

[ Upload Client ] ---> [ S3 Ingestion ] ---> [ Event Lambda ] ---> [ Transcoding Workers ]
                                                                          |
                                                                    (Output to S3)
                                                                          |
[ Stream Client ] <--- [ Edge CDN Cache ] <-------------------------------+

1. The Scenario & Scale

A media streaming service processes uploaded video files and streams them globally to millions of concurrent users.

• Scale: Petabytes of data, global distribution.
• Constraints: Videos must start playing in sub-second latency anywhere in the world, automatically adjusting to client bandwidth variations (Adaptive Bitrate Streaming).

2. Architectural Blueprint Requirements

• Ingestion & Notification pipeline: Uploads are written to an S3 Ingestion Bucket. File creation events trigger Serverless Lambdas that insert jobs into a Transcoding Worker queue.
• Multi-Bitrate Transcoding: Dynamic workers transcode raw video into HLS (HTTP Live Streaming) and MPEG-DASH formats at different resolutions (1080p, 720p, 480p), segmenting video into 2-second .ts chunks.
• Multi-Tier Edge CDN Caching: Route client streaming traffic through a Global Content Delivery Network (CDN) like CloudFront. Store video chunks at edge caches globally to bypass light-propagation fiber latency.
• Distributed Metadata persistent store: Store user profiles, play history, and catalog search data in a globally replicated database (such as DynamoDB Global Tables or CockroachDB) to maintain low read latencies.

3. Software Design Specification (SDS) Document Outline

To document your Capstone architecture, your Software Design Specification (SDS) must adhere to the following outline:

# Software Design Specification: [Capstone Title]

## 1. System Requirements & Boundary Constraints
*   Document the functional limits (e.g. maximum transactions/second) and constraints.
*   Define the target SLA goals (e.g. RTO, RPO, Latency percentiles).

## 2. Target System Architecture
*   Explain the role of each service, gateway, database, queue, and cache.
*   Document the communication protocols used (REST, gRPC, AMQP).

## 3. Distributed Transactions & Consistency Strategy
*   Detail how data consistency is maintained (e.g. Sagas, Redis locks, quorums).
*   Specify database isolation levels and write replication paths.

## 4. Failure Modes & Self-Healing Playbook
*   Analyze potential failure modes (network splits, crashed nodes, slow databases).
*   Define the circuit breaker thresholds and fallback behaviors.

4. Hands-on Architecture Challenge

Scenario Description

Refactor the starting sandbox layout to represent the complete, detailed end-to-end architecture topology for Challenge 17.1: The Greenfield Ticket Multi-Processor.

Your Goal:

1. Model the complete request flow:
   • Client $\rightarrow$ APIGateway (with rate limiter).
   • APIGateway $\rightarrow$ IngestionQueue (Kafka Topic).
   • IngestionQueue $\rightarrow$ WorkerPool (Multiple parallel consumers).
   • WorkerPool $\rightarrow$ RedisLockStore (Validating seat reservations).
   • WorkerPool $\rightarrow$ PostgresLedgerDB (Persisting finalized ticket purchases).
2. Use Mermaid's flowchart (graph TD or similar) syntax.
3. Ensure all database nodes, brokers, gateways, and worker boundaries are represented.

5. Practice Challenge Template

Use this template in your sandbox to model the Ticketing architecture:

graph TD
    Client[Client App] -->|1. Submit Purchase| APIGateway[Edge API Gateway]
    
    subgraph Gateway_Ingress [Gateway Boundary]
        APIGateway -->|2. Check Limit| RateLimiter[Token Bucket Limiter]
    end
    
    RateLimiter -->|3. Accept & Queue| KafkaBroker[Kafka Ingestion: ticket-sales-topic]
    
    subgraph Consumer_Processing [Worker Pool Boundary]
        KafkaBroker -.->|4. Pull Job| Worker1[Worker Node 1]
        KafkaBroker -.->|4. Pull Job| Worker2[Worker Node 2]
        
        Worker1 -->|5a. Acquire Lock| RedisLock[Redis Distributed Lock]
        Worker2 -->|5a. Acquire Lock| RedisLock
        
        Worker1 -->|5b. Write Ticket| RelationalDB[(PostgreSQL ACID DB)]
        Worker2 -->|5b. Write Ticket| RelationalDB
    end
    
    style Client fill:#9f9,stroke:#333,stroke-width:2px
    style APIGateway fill:#9ff,stroke:#333,stroke-width:3px
    style KafkaBroker fill:#9ff,stroke:#333,stroke-width:2px
    style Worker1 fill:#9f9,stroke:#333,stroke-width:2px
    style Worker2 fill:#9f9,stroke:#333,stroke-width:2px
    style RedisLock fill:#9ff,stroke:#333,stroke-width:2px
    style RelationalDB fill:#9f9,stroke:#333,stroke-width:3px

CONGRATULATIONS: Complete this Capstone challenge in your sandbox, finalize your Software Design Specification, and submit your work for evaluation. Upon successful validation, the system will issue your public, verified Macro Patterns Consortium Systems Architect Certificate, signaling your design competency to global recruitment partners.