System Design Concepts & Interactive Labs

Distributes incoming client requests across a pool of backend servers to prevent single-node bottlenecks and maximize throughput.

Sits in front of internal servers to terminate TLS, compress responses (Gzip/Brotli), cache static assets, and shield internal IP addresses.

Unified entry point for microservices handling authentication (JWT), token bucket rate limiting, protocol translation (REST to gRPC), and request routing.

Translates human-readable domain names into IP addresses using hierarchical nameservers (Root -> TLD -> Authoritative). Anycast routes packets to the topologically closest PoP.

Vertical scaling (Scale Up) upgrades CPU/RAM on a single box. Horizontal scaling (Scale Out) adds more stateless worker nodes behind a load balancer.

Partitions large database tables horizontally across independent physical servers using a shard key (e.g., hash(user_id) % N or Consistent Hash Ring).

Master node accepts write transactions and replicates Write-Ahead Logs (WAL) to multiple read replicas. Read replicas serve high-volume read traffic.

B-Trees (PostgreSQL/MySQL) organize data into balanced tree pages on disk, optimizing for fast in-place reads. LSM-Trees (Cassandra/RocksDB) append writes sequentially to MemTable and SSTables, maximizing write throughput.

Guarantees Atomicity, Consistency, Isolation, and Durability. Isolation levels (Read Committed -> Repeatable Read -> Serializable) trade concurrency for phantom read protection.

Application code queries the cache first. On cache hit, data returns in <2ms. On cache miss, application queries database and writes data into cache for future requests.

Write-Through writes to cache and DB simultaneously. Write-Behind (Write-Back) writes immediately to cache and acknowledges the client, flushing asynchronously to DB in batches.

When a popular cached key expires, thousands of concurrent requests miss simultaneously and flood the primary database, crashing storage servers.

A space-efficient bit array that tests whether an element is definitely NOT in a set or possibly in a set in O(k) time without disk I/O.

Monitors downstream service failure rates. If errors exceed a threshold (e.g. 50%), the breaker trips to OPEN, instantly short-circuiting calls and preventing thread pool exhaustion.

Controls request velocity per IP/User to prevent DDoS attacks, credential stuffing, and noisy-neighbor compute starvation. Drops excess calls with HTTP 429.

Clients attach a unique UUID in the Idempotency-Key header. Servers lock the key in Redis and cache the completed response, guaranteeing zero duplicate payments on retry.

Isolates resources (thread pools, connection pools, memory) into distinct compartments. An outage in an analytics service cannot exhaust threads allocated for user checkout.

Manages distributed transactions across microservices without 2-Phase Locking. Each service executes a local transaction; if a step fails, compensating transactions undo earlier changes.

Atomic consensus across multiple database nodes. Phase 1 (Prepare): coordinator asks all nodes if they can commit. Phase 2 (Commit): coordinator issues final commit or rollback.

Stores an immutable log of events instead of current state. CQRS separates Command (mutating write models) from Query (read-optimized materialized views).

Guarantees database update and message publishing commit together. Writes event to an `outbox` table in the same DB transaction; Debezium tails the WAL to publish to Kafka.

WebSockets provide full-duplex bidirectional TCP communication. SSE provides lightweight unidirectional streaming from server to browser over standard HTTP/2.

Combines multiple microservice GraphQL schemas into a single unified supergraph. Apollo Gateway coordinates sub-queries and resolves entities across independent services.

Retrieval-Augmented Generation enriches prompts with private data. Text chunks are embedded into dense vectors, searched via HNSW Cosine Similarity, and injected into LLM context.

Manages Transformer Key-Value activations in GPU VRAM like OS virtual memory pages. Allocates non-contiguous memory blocks on-demand, cutting VRAM waste from 70% to <4%.

Reverse proxy for LLM endpoints providing semantic caching (sub-20ms cache hit for similar queries), automatic model fallback, rate limiting, and prompt injection filters.

Autonomous AI agent execution pattern combining reasoning traces and action calls. Agents iteratively reason, invoke tools (web search, SQL, Python interpreter), and observe outputs.