System Design Fundamentals

Overview

System design interviews test your ability to think like a senior engineer, someone who can decompose a vague problem into concrete components, reason about trade-offs, and communicate a coherent architecture under time pressure. Unlike coding interviews where there is a correct solution, system design is evaluated on your thinking process, your communication clarity, and your ability to justify decisions.

Every system design problem, whether it is a URL shortener, a ride-sharing platform, or a distributed message queue, shares the same underlying building blocks. A Load Balancer distributes traffic. Application servers process requests. A cache reduces database load. A message queue decouples producers from consumers. Understanding these building blocks and how they connect is the foundation for everything else in this curriculum.

This lesson introduces the components, the mental models, and the structured interview framework that appear throughout all practice problems in Nalar Linuwih. By the end, you will have the vocabulary and the approach to walk into any system design interview and start from a position of confidence rather than blank-page panic.

The structured approach matters because interviewers at top-tier companies are not just looking for the right answer, they are looking for evidence that you can work through ambiguity systematically. A candidate who jumps straight to "we need Kafka and Cassandra" signals shallow thinking. A candidate who starts with "let me clarify the requirements and scale before proposing a design" signals seniority.

Core Concepts

Client-Server Architecture and the Request Lifecycle

Every internet-facing system begins with a client making a request to a server. The client might be a browser, a mobile app, or another service. The server processes the request and returns a response.

The request lifecycle matters because each hop introduces latency and potential failure. When you design a system, you are deciding how many hops exist between a user action and the response, where data is stored, and what happens when any component fails.

A typical web request lifecycle:

  1. The client sends an HTTP request to a domain name.
  2. DNS resolves the domain to an IP address.
  3. The request reaches a Load Balancer.
  4. The Load Balancer forwards the request to one of several Application Servers.
  5. The Application Server checks a Cache for the requested data.
  6. On a cache miss, the Application Server queries the Database.
  7. The Database returns data; the Application Server populates the Cache.
  8. The Application Server returns the response to the Load Balancer.
  9. The Load Balancer forwards the response to the client.

Understanding this lifecycle lets you identify where bottlenecks form, where caching helps, and where failures can cascade.

Load Balancers and Reverse Proxies

A load balancer distributes incoming requests across multiple servers so no single server becomes overwhelmed. Load balancers operate at Layer 4 (TCP/UDP) or Layer 7 (HTTP). Layer 7 load balancers can route based on the request path, headers, or content, useful for routing /api/* requests to one cluster and /assets/* to another.

Load balancing algorithms include:

  • Round Robin: requests are distributed in rotation. Simple, but ignores server load.
  • Least Connections: sends new requests to the server with the fewest active connections. Better for variable-duration requests.
  • IP Hash: routes a given client's requests consistently to the same server. Useful for session affinity.

A reverse proxy sits in front of servers and intercepts requests on their behalf. Load balancers are a type of reverse proxy, but reverse proxies also perform SSL termination, caching, and compression. Nginx and HAProxy are common choices. In cloud environments, managed load balancers (AWS ALB, GCP Cloud Load Balancing) handle this layer.

In interviews, mention load balancers whenever your design has more than one application server. This signals that you understand horizontal scaling.

Application Servers and Stateless Design

Application servers run your business logic, they validate inputs, coordinate calls to the database and cache, and compute responses. The key design principle is stateless design: each application server should be able to handle any request without relying on local state from previous requests.

Stateless servers allow horizontal scaling, you can add or remove servers freely because no session state is tied to a specific machine. If you need to maintain user session state, store it externally in a shared store such as Redis, not on the application server itself.

When your application server needs to do work that should not block the request (sending an email, resizing an image, updating counters), defer it to a Message Queue and process it asynchronously with background workers.

Databases: Relational vs. NoSQL

The database is the most consequential component choice in most systems. The choice between relational (SQL) and NoSQL databases depends on the data model, query patterns, and consistency requirements.

Relational databases (PostgreSQL, MySQL) store data in structured tables with rows and columns. They enforce schemas, support ACID transactions, and use SQL for queries. They are the right choice when:

  • Data has complex relationships (foreign keys, joins)
  • You need strong consistency and transaction support
  • Your query patterns are known upfront

NoSQL databases come in several varieties:

  • Document stores (MongoDB, DynamoDB): store JSON-like documents. Flexible schema, good for hierarchical data.
  • Key-value stores (Redis, DynamoDB): extremely fast lookups by key. Used for caching, sessions, and leaderboards.
  • Wide-column stores (Cassandra, HBase): designed for massive write throughput and time-series or event data.
  • Graph databases (Neo4j): model relationship-heavy data such as social graphs.

The choice matters in interviews. Saying "I'll use Postgres" is fine for most problems. Saying "I'll use Cassandra because we have high write throughput and can tolerate eventual consistency" demonstrates deeper reasoning.

Caching Layers

Caching stores the results of expensive computations or database queries so that subsequent requests can be served faster without repeating the work.

Where caching fits in the stack:

  • Client-side cache: the browser caches static assets (images, JS, CSS). Controlled via HTTP Cache-Control headers.
  • CDN cache: a geographically distributed cache for static and semi-static content. Reduces latency by serving content from an edge node near the user.
  • Application-layer cache: a shared in-memory cache (Redis, Memcached) that sits between the application server and the database. Reduces database load for frequently read data.
  • Database query cache: some databases support query result caching internally, though this is less commonly relied on in high-scale designs.

Cache invalidation, deciding when to evict stale data, is one of the hardest problems in distributed systems. Common strategies include:

  • TTL-based eviction: data expires after a fixed time-to-live.
  • Write-through: the cache is updated at the same time as the database.
  • Cache-aside (lazy loading): data is loaded into the cache on the first cache miss and served from cache on subsequent reads.

Message Queues and Asynchronous Processing

A message queue (Kafka, RabbitMQ, SQS) decouples the producer of work from the consumer of work. Instead of calling a downstream service directly and waiting for a response, the producer publishes a message to a queue and continues. A consumer (worker) picks up the message and processes it independently.

Benefits of message queues:

  • Decoupling: producers and consumers scale independently
  • Buffering: absorbs traffic spikes by queuing work instead of dropping it
  • Retry semantics: failed messages can be retried without the producer knowing
  • Fan-out: one message can be consumed by multiple consumers

In interviews, introduce a message queue whenever you have a write-heavy operation that does not need to be synchronous (sending notifications, generating thumbnails, updating search indices, logging events).

CDNs and Static Asset Delivery

A Content Delivery Network (CDN) is a globally distributed network of edge servers that cache and serve content from locations close to users. Examples include Cloudflare, Akamai, and AWS CloudFront.

CDNs are used for:

  • Static assets: JavaScript bundles, CSS, images, and videos that do not change per-request
  • Cacheable API responses: GET responses with predictable TTLs can be CDN-cached
  • Edge computing: running lightweight logic at the edge to reduce origin server load

In a system design, introduce a CDN when you need to serve global users with low latency or when you have a high read-to-write ratio on static content. For a video platform, the CDN is not optional, it is the entire delivery mechanism.

Mermaid Diagram 1: Canonical Multi-Tier Web Architecture

graph TD
    Client["Client\n(Browser / Mobile App)"]
    CDN["CDN\n(Edge Cache, static assets)"]
    LB["Load Balancer\n(Layer 7, distributes HTTP)"]
    App1["App Server 1\n(stateless business logic)"]
    App2["App Server 2\n(stateless business logic)"]
    Cache["Cache Layer\n(Redis, hot data)"]
    DB["Primary Database\n(Postgres, source of truth)"]
    DBreplica["Read Replica\n(Postgres, read-heavy queries)"]
    MQ["Message Queue\n(Kafka / SQS, async work)"]
    Worker["Background Workers\n(notifications, indexing)"]

    Client -->|"static assets"| CDN
    Client -->|"API requests"| LB
    LB --> App1
    LB --> App2
    App1 -->|"read hot data"| Cache
    App2 -->|"read hot data"| Cache
    Cache -->|"cache miss"| DB
    App1 -->|"writes"| DB
    App2 -->|"writes"| DB
    DB -->|"replication"| DBreplica
    App1 -->|"write-heavy async work"| MQ
    App2 -->|"write-heavy async work"| MQ
    MQ --> Worker
    Worker -->|"DB writes"| DB

Mermaid Diagram 2: Structured Approach to a System Design Interview

flowchart TD
    A["Step 1: Clarify Requirements\n(5 min)\nFunctional: what does the system do?\nNon-functional: scale, latency, consistency?"]
    B["Step 2: Estimate Scale\n(3 min)\nDAU, QPS, storage, bandwidth\nback-of-the-envelope"]
    C["Step 3: Define API\n(3 min)\nKey endpoints, request/response shapes\nidempotency, pagination"]
    D["Step 4: High-Level Design\n(10 min)\nComponents, data flow\ncore read and write paths"]
    E["Step 5: Deep Dive\n(10 min)\nScale bottlenecks, storage schema\ncritical component internals"]
    F["Step 6: Trade-offs\n(5 min)\nWhy this design over alternatives?\nWhat fails under pressure?"]
    G["Step 7: Summary\n(2 min)\nRecap choices, highlight open questions\nask for feedback"]

    A --> B --> C --> D --> E --> F --> G

Interview Application

The First 5 Minutes Are Decisive

Interviewers evaluate whether you behave like a senior engineer. In the first 5 minutes, a senior engineer does three things:

  1. Clarifies requirements before drawing boxes. Ask functional questions: "What are the core use cases?" Ask non-functional questions: "What is the expected scale, daily active users and requests per second?" Ask constraint questions: "Are there geographic or regulatory requirements?" This behavior signals that you do not jump to solutions before understanding the problem.

  2. Acknowledges what you do not know. If a domain is unfamiliar, say so and ask a targeted question. "I'm less familiar with ticketing reservation flows, can you tell me whether seat holds are temporary or permanent?" This is far better than guessing silently.

  3. States your assumptions out loud. "I'll assume this is read-heavy, around 10:1 read-to-write ratio, does that match your expectation?" Stating assumptions invites the interviewer to correct you early, preventing you from designing the wrong system for 30 minutes.

Using Estimation as a Segue

After clarifying requirements, move into back-of-the-envelope estimation. This is not a math test, it is a tool for scoping component choices. If your estimation reveals 50,000 QPS at peak, you immediately know that a single database server is a bottleneck and that you need read replicas or a cache. If your estimation shows 1 TB of new data per day, you know that a single-node Postgres instance is insufficient and you need to discuss sharding or a data warehouse.

State your calculations out loud, use round numbers, and connect them to architecture decisions. "100 million DAU, 10 requests per user per day, that's 1 billion requests per day, roughly 12,000 QPS on average, call it 60,000 QPS at peak. That tells me we need horizontal scaling for app servers and a cache to keep the database load manageable."

Demonstrating Trade-off Reasoning

The most important differentiator between a junior and a senior candidate is trade-off reasoning. A junior candidate says "I'll use Kafka because it's scalable." A senior candidate says "I'll use Kafka here because the notification delivery is write-heavy and asynchronous, and Kafka's consumer group model lets us add more workers if the queue depth grows. The trade-off is operational complexity, if the team is small, SQS would be simpler with slightly less flexibility."

Every major component choice should be accompanied by a brief trade-off statement. Practice the pattern: "I chose X because [reason], and the trade-off is [cost/limitation]."

Common Mistakes

Jumping into components before clarifying requirements. The most common mistake candidates make is drawing a diagram in the first 30 seconds. Without knowing the scale, consistency requirements, or core use cases, any diagram is guesswork. Interviewers notice and penalize this immediately.

Over-engineering for scale before establishing the basic flow. Start simple. Get a working end-to-end flow on the whiteboard before adding complexity. Many candidates jump to sharding, multi-region replication, and event sourcing before explaining how a basic write operation works. This signals anxiety rather than seniority.

Ignoring failure modes. Every component can fail. What happens when the cache is cold? What happens when the database primary fails? What happens when a message queue consumer crashes mid-processing? Proactively mentioning failure handling, even briefly, demonstrates the kind of operational maturity interviewers look for at senior levels.

Treating system design as having a single right answer. There is no "correct" system design. There are trade-offs. An interviewer who pushes back on your choice ("why not use MongoDB?") is testing whether you can defend your decision and acknowledge alternatives. Treat pushback as a conversation, not a judgment.

Not connecting NFRs to component choices. If you said the system needs 99.99% availability in the requirements phase, then your design needs to explain how that availability is achieved, redundancy, failover, geographic distribution. If you stated a sub-100ms latency target, your design needs a cache in the read path. Every NFR you surface should appear as a constraint that shapes a component choice.

Key Vocabulary

Latency, The time elapsed between a client sending a request and receiving a response. Measured in milliseconds. In interviews, distinguish between p50 (median), p95, and p99 latency, the 99th percentile is what your slowest users experience. Example: "Our latency target is p99 under 200ms, which means we cannot afford a synchronous call to a slow external API in the critical path."

Throughput, The number of requests or operations a system can process per unit of time, typically expressed as queries per second (QPS) or requests per second (RPS). Throughput and latency are related: high throughput at low latency requires efficient parallelism. Example: "At 50,000 QPS, a single Postgres instance will be a bottleneck, we need either read replicas or a caching layer."

Availability, The fraction of time a system is operational and accessible. Expressed as a percentage, 99.9% ("three nines") equals about 8.7 hours of downtime per year; 99.99% ("four nines") equals about 52 minutes per year. Example: "This is a payment flow, so we need 99.99% availability, that means active-active redundancy across two regions."

Scalability, The ability of a system to handle increased load by adding resources. Horizontal scalability means adding more machines; vertical scalability means upgrading a single machine. Example: "Our application servers are stateless, so scaling horizontally is straightforward, we add servers behind the load balancer."

Load Balancer, A component that distributes incoming requests across multiple servers to prevent any single server from becoming a bottleneck. Operates at Layer 4 (TCP) or Layer 7 (HTTP). Example: "With a Layer 7 load balancer, we can route /api/ requests to the backend cluster and /static/ requests to the CDN directly."

Reverse Proxy, A server that intercepts requests on behalf of backend servers. Provides SSL termination, compression, caching, and load balancing. Example: "Nginx acts as our reverse proxy, it handles SSL termination so our application servers only see plain HTTP."

Horizontal Scaling, Adding more instances of a component (machines, pods, containers) to handle more load. Preferred for stateless components. Example: "When peak QPS doubles during the holiday sale, we scale our application servers horizontally from 10 to 20 instances."

Vertical Scaling, Upgrading a single machine with more CPU, RAM, or disk. Has a physical ceiling and causes downtime during the upgrade. Example: "We can vertically scale the database temporarily, but long-term we will need read replicas or sharding."

Stateless Service, A service where each request carries all the information needed to process it; no session state is stored on the server. Enables horizontal scaling without session affinity. Example: "Because our API servers are stateless, any server can handle any request, the load balancer can round-robin freely."

CDN (Content Delivery Network), A globally distributed network of edge servers that cache and deliver content from locations close to users. Reduces latency for static assets and cacheable API responses. Example: "We serve all images and video thumbnails from a CDN, users in Jakarta get served from the Singapore edge node rather than our origin in us-east-1."

Summary

System design is a skill that compounds. The building blocks, load balancers, application servers, caches, databases, message queues, CDNs, are the same regardless of whether you are designing a URL shortener or a distributed video platform. What changes is how you combine them, which trade-offs you prioritize, and how clearly you communicate your reasoning.

The structured interview approach, Clarify, Estimate, API, High-Level, Deep Dive, Trade-offs, Summary, gives you a repeatable framework that prevents blank-page panic and signals seniority to interviewers. Every subsequent lesson in the Fundamentals track adds depth to one or more of these steps. Non-functional Requirements goes deep on Step 1. Resource Estimation goes deep on Step 2. API Design goes deep on Step 3.

Use this lesson as your reference point. Every time you encounter a new component in a practice problem, connect it back to the canonical architecture: where does it sit in the stack? What does it replace or augment? What is the trade-off of including it versus leaving it out?

The strongest system design candidates are not the ones who know the most component names, they are the ones who can explain, under pressure, why they chose the components they did.

Next: Non-functional Requirements →