Module 18: Cloud Architecture Fundamentals

1. Module Title & Overview

• Title: Module 18: Cloud Architecture Fundamentals
• Overview: This module covers the translation of distributed systems principles onto public cloud providers (AWS, Azure, GCP). Students learn to design system topologies by evaluating cloud-native primitives, analyzing managed vs. self-managed trade-offs, and mapping CAP/PACELC constraints to virtualized network, storage, and compute boundaries.

2. Learning Objectives

• Assess Managed vs. Self-Managed Services: Quantify the trade-offs between operational overhead, execution limits, cost structures, and vendor lock-in for compute and storage.
• Design Multi-Region Replication Topologies: Configure active-active and active-passive network topologies utilizing cloud-native databases, global traffic routing, and private backend links.
• Optimize Egress and Transfer Costs: Evaluate the pricing models of cloud networks to design cost-aware architectures that minimize cross-AZ and cross-region traffic fees.
• Implement Tenant Isolation and Security Patterns: Structure logical network isolation (VPC peering, private link networks, NATs) and compute boundaries using cloud identity and tenant resource partitioning.

3. Prerequisite Statement

Students must complete Module 7 (Distributed Computing Realities & CAP) and Module 14 (Fault Tolerance & Resiliency) to understand network partition behaviors and fault containment zones (circuit breakers, bulkheads) before translating these concepts into virtualized cloud regions.

4. Content Outline

Section 18.1: Compute Primitives & Virtualization Models

• Concepts: Bare metal vs. Hypervisor VMs vs. Container Engines vs. Serverless FaaS.
• Deep Dive: Execution lifetimes, cold start latency mechanics, thread limits, memory-to-CPU scaling, and ephemeral local storage limits.
• Architectural Trade-offs: Serverless FaaS (e.g., AWS Lambda) offers automated horizontal scaling but introduces cold start delays (up to several seconds for cold runtime initializations) and execution time limits (e.g., 15 minutes). Virtual Machines (e.g., EC2) provide persistent compute and complete OS control but require manual autoscaling group configuration and patch management.
• Physical Constraints: Resource limits, hypervisor noisy-neighbor effects, CPU instruction set variance, and networking limits (packet-per-second constraints on specific virtual NIC types).

Section 18.2: Cloud Network Topology & Routing Mechanics

• Concepts: Virtual Private Clouds (VPCs), subnets, routing tables, NAT Gateways, VPC Peering, Transit Gateways, and PrivateLink.
• Deep Dive: Under-the-hood routing pathways of Software-Defined Networks (SDN). Unicast vs. Anycast IP allocation, regional load-balancing topologies (ALB/NLB), and global traffic routing policies.
• Architectural Trade-offs: VPC Peering yields direct, low-latency connection lines between two VPCs but operates on non-overlapping CIDR blocks and scales quadratically. Transit Gateways centralize hub-and-spoke networking but introduce processing hops and extra cost per GB processed.
• Physical Constraints: Subnet size planning, NAT Gateway throughput limits (e.g., 45 Gbps scale-out limit), and cross-Availability Zone (AZ) latency variance (typically 0.5–1.5ms).

Section 18.3: Storage Primitives & Persistence Topologies

• Concepts: Block Storage (EBS), Object Storage (S3), Shared File Storage (EFS), and Managed NoSQL/SQL databases.
• Deep Dive: IOPS scaling limits, read/write throughput quotas, storage durability tiering, eventual consistency mechanics of object store metadata, and local instance store ephemeral architectures.
• Architectural Trade-offs: EBS offers high-speed, low-latency block access for single VMs but is regionally bound and cannot be naturally attached across distinct instances. S3 provides high durability and unlimited scalability but incurs higher latency per request (first-byte latency of 10–50ms) and operates on eventual consistency models for overwrites.
• Physical Constraints: IOPS limits per volume, throughput-to-capacity ratios, regional boundaries, and storage replication latencies.

Section 18.4: Managed vs. Self-Managed Database Mechanics

• Concepts: Databases on VMs (EC2 PostgreSQL) vs. Managed Database Services (AWS RDS/Aurora PostgreSQL).
• Deep Dive: Read-replica scaling lag, automated backups, regional failover automation (multi-AZ replication), patching windows, and parameter group configurations.
• Architectural Trade-offs: Self-managed DBs on VMs allow custom OS kernels, extensions, and cost-efficiency for large disks, but demand deep DBA ops, manual replication scripts, and complex database verifications. Managed DBs handle failovers, backup snapshots, and minor engine upgrades automatically, but limit administrative access and introduce premium licensing/compute markups (often 1.5x–2.0x raw VM costs).
• Physical Constraints: Storage auto-growth rates, read replica replication lag (dependent on write throughput and network distance), and lock escalation blocks during automated patching.

Section 18.5: Multi-Region Active-Active Topologies

• Concepts: Multi-region active-active vs. active-passive, latency-based routing, database global tables, cross-region replication, and split-brain resolution.
• Deep Dive: Conflict-Free Replicated Data Types (CRDTs), last-write-wins (LWW) conflict resolution, cross-region network latency (e.g., 70ms transcontinental round trips), and global traffic management DNS routing tables.
• Architectural Trade-offs: Active-active multi-region systems deliver near-instantaneous global failover and extremely low latency for users, but present massive sync challenges, write collision risks, and high cost overhead. Active-passive failover models simplify database consistency but suffer from larger Recovery Time Objectives (RTO) and Recovery Point Objectives (RPO).
• Physical Constraints: Speed of light in fiber optics (limiting minimum ping times), write lock delays, and cross-region replication bandwidth bottlenecks.

Section 18.6: Cloud Cost Dynamics & Data Transfer Mechanics

• Concepts: Compute reservation models, data transfer billing (cross-region, cross-AZ, public egress), service endpoint optimization, and cost-aware architectural designs.
• Deep Dive: AWS/Azure/GCP data transfer pricing structures, cost-aware container and microservice placement, VPC Gateway Endpoints vs. Interface Endpoints, and lifecycle policy execution on object stores.
• Architectural Trade-offs: Distributing microservices across multiple AZs maximizes availability but increases cross-AZ data transfer fees (e.g., $0.01 per GB in both directions). Consolidating services in a single AZ eliminates transfer costs but creates a single point of failure (AZ outage).
• Physical Constraints: Cloud billing cycles, network interface speeds, and IP routing table size restrictions.

5. Key Concepts

• Availability Zone (AZ): One or more discrete data centers with redundant power, networking, and connectivity within a Region.
• Region: A physical location around the world where a cloud provider clusters multiple Availability Zones.
• Cold Start: The latency delay incurred when a serverless compute platform spins up a new instance of a function container to execute a request.
• Egress Traffic: Data traveling out of a cloud network boundary to the public internet or external services, which incurs the highest network cost.
• VPC Peering: A networking connection between two VPCs that enables routing traffic between them using private IPv4 or IPv6 addresses.
• PrivateLink: AWS/Azure private network connectivity that exposes services privately inside a VPC without traversing public internet routes.
• Gateway Endpoint: A free VPC routing resource that provides private connectivity to Object Storage (S3) or Key-Value databases (DynamoDB) without using internet gateways.
• Active-Active Replication: A database topology where write operations can be performed concurrently across multiple regions, requiring eventual conflict resolution.
• Split-Brain Scenario: A failure state where two parts of a partitioned system independently assume they are the primary coordinator, leading to data divergence.
• IOPS (Input/Output Operations Per Second): A metric measuring the performance of storage devices, defining the maximum rate of discrete read/write tasks.
• Shared Responsibility Model: The security framework dividing tasks between the cloud provider (security of the cloud) and the customer (security in the cloud).

6. Practice Section Description

• Practice Exercise: Designing a Global, Low-Latency User Profile Service.
• Scenario: A gaming platform requires sub-100ms global latency for user profiles. It must support high write concurrency (15,000 writes/sec) and survive the complete loss of an entire cloud region without losing session state.
• Challenge: Using a cloud topology diagram (via the diagram editor), students must layout a multi-region active-active architecture. They must specify where database replication occurs, configure route policies (latency-based routing), locate private connections (VPC peering/transit paths), place edge cache networks, and outline the network pathway.
• Constraints: Cross-region data transfer must be optimized. Write conflict resolution must be clearly defined (e.g., LWW or CRDTs). Network egress must bypass public internet via private backbones where possible.

flowchart TB
    subgraph Global Traffic Layer
        DNS[Route 53 Latency DNS]
        GA[Anycast Global Accelerator]
    end

    subgraph Region A [us-east-1]
        ALB_A[Application Load Balancer]
        App_A[User Service Tasks - ECS/EKS]
        Cache_A[(Redis Cache Cluster)]
        DB_A[(DynamoDB Global Table - Region A)]
    end

    subgraph Region B [eu-west-1]
        ALB_B[Application Load Balancer]
        App_B[User Service Tasks - ECS/EKS]
        Cache_B[(Redis Cache Cluster)]
        DB_B[(DynamoDB Global Table - Region B)]
    end

    %% Network Flow
    DNS -->|Geo-Routing| GA
    GA -->|Private Route - Primary| ALB_A
    GA -->|Private Route - Secondary| ALB_B

    ALB_A --> App_A
    ALB_B --> App_B

    App_A --> Cache_A
    App_A --> DB_A
    
    App_B --> Cache_B
    App_B --> DB_B

    %% Replication
    DB_A <===>|Async Global Replication - Conflict Resolution| DB_B

7. Deliverable/Documentation

• Deliverable Name: Cloud Architecture Decision Matrix (CADM)
• Description: A comprehensive markdown specification containing:
  1. A structural block-diagram of the global infrastructure.
  2. A cost-impact spreadsheet evaluating egress data volumes, cross-AZ traffic, and database replication costs.
  3. A detailed table assessing the trade-offs of using Managed SQL (e.g., Aurora Postgres Multi-Region) vs. Managed NoSQL (e.g., DynamoDB Global Tables) for this specific scenario.
  4. An operational recovery plan mapping RTO and RPO metrics for three failure modes: single host failure, AZ outage, and complete regional network isolation.

Code Snippet: C# Resilience Policy using Polly for Cloud Database Connections

using System;
using System.Net.Http;
using System.Threading.Tasks;
using Polly;
using Polly.Retry;

public class CloudConnectionManager
{
    private readonly AsyncRetryPolicy _connectionRetryPolicy;
    private readonly AsyncRetryPolicy _fallbackRegionPolicy;
    
    public CloudConnectionManager()
    {
        // Define transient connection error retry policy (exponential backoff with jitter)
        _connectionRetryPolicy = Policy
            .Handle<HttpRequestException>()
            .Or<TimeoutException>()
            .WaitAndRetryAsync(3, 
                retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)) 
                                + TimeSpan.FromMilliseconds(new Random().Next(0, 100)),
                (exception, timeSpan, retryCount, context) => 
                {
                    Console.WriteLine($"Database connection failed. Retrying in {timeSpan.TotalSeconds}s... (Attempt {retryCount})");
                });

        // Define region fallback policy
        _fallbackRegionPolicy = Policy
            .Handle<Exception>()
            .RetryAsync(1, (exception, retryCount) =>
            {
                Console.WriteLine("Primary region connection exhausted. Failing over network to secondary backup endpoint...");
            });
    }

    public async Task<string> ExecuteDatabaseQueryAsync(Func<string, Task<string>> queryExecutor, string primaryEndpoint, string secondaryEndpoint)
    {
        // Wrap execution in the fallback and retry policy chain
        return await _fallbackRegionPolicy.ExecuteAsync(async () =>
        {
            try
            {
                return await _connectionRetryPolicy.ExecuteAsync(() => queryExecutor(primaryEndpoint));
            }
            catch (Exception)
            {
                // Trigger failover attempt to secondary regional endpoint
                return await _connectionRetryPolicy.ExecuteAsync(() => queryExecutor(secondaryEndpoint));
            }
        });
    }
}

8. Integration Notes

• Curriculum Placement: Extends Module 7 (Distributed Computing Realities & CAP) by grounding database replication latency and network partitioning in actual AWS/Azure topologies.
• hiring_signal: "Can design cost-efficient, resilient, multi-region infrastructures that balance latency targets with cloud egress budgets without relying blindly on vendor managed magic."