Resource Estimation

Overview

Back-of-the-envelope estimation is a core skill that separates senior engineers from junior candidates in system design interviews. It is not a math test. It is a structured reasoning exercise that demonstrates you understand how to scope a system before proposing components. Interviewers who probe estimation are asking: "Does this candidate know how big the system is, and does their design reflect that understanding?"

The consequences of skipping estimation are severe. A candidate who jumps from requirements to architecture without estimating scale might propose a single database instance for a system handling 100 million daily active users, a solution that would fail at day one of production. A candidate who estimates first will notice that 100 million DAU at 10 requests per user per day yields approximately 60,000 queries per second at peak load, which immediately signals the need for horizontal scaling, caching, and careful database choice.

Estimation is also the bridge between the requirements phase (Step 1: Problem Framing and Step 2: NFR Clarification) and the design phase. The numbers you compute in estimation become the constraints that justify every major component you introduce: "Our storage grows by 5 TB per year, which is why we need a sharding strategy" or "At 80,000 peak QPS, a single origin database would be overwhelmed, that justifies our caching layer."

This lesson teaches you the estimation pipeline: from daily active users to queries per second to storage requirements to bandwidth. It provides the common reference numbers every engineer should know, explains how to reason about read-to-write ratio and growth projections, and shows you how to present estimates confidently in an interview without wasting time on precision.

Core Concepts

QPS Estimation: DAU to Peak QPS

The most important estimation pathway: from daily active users (DAU) to peak queries per second (QPS).

Step 1: Daily Active Users (DAU)
This is typically given in the problem statement or established during requirements clarification. Common interview scales: 1M DAU (early-stage product), 10M DAU (growing product), 100M DAU (large consumer product), 1B DAU (global platform like YouTube or WhatsApp).

Step 2: Requests per User per Day
Estimate how many requests a typical active user generates per day. For a URL shortener: a user might create 1 short URL per day and click 10-20. For Twitter: a user might post 2-3 tweets and scroll through 200 tweets per day. Keep this estimate simple, it does not need to be exact.

Step 3: Total Requests per Day

Total requests/day = DAU × requests/user/day

Example: 100M DAU × 10 req/day = 1 billion requests/day

Step 4: Average QPS

Average QPS = Total requests/day ÷ 86,400 seconds/day

Example: 1 billion ÷ 86,400 ≈ 11,600 QPS average. Round to ~12,000 QPS.

Step 5: Peak QPS
Traffic is not uniform, it peaks during certain hours (morning commute, lunch, evening). A common rule of thumb is that peak QPS is 2-5x the average. Use 3x as a default unless the problem suggests a specific spike pattern.

Peak QPS = Average QPS × peak factor (typically 3-5x)

Example: 12,000 × 3 = ~36,000 QPS peak. Round to 40,000 QPS for comfortable planning.

Step 6: Read vs. Write QPS
Most systems have a read-heavy ratio. Separate your QPS estimate into reads and writes using the read-to-write ratio established in requirements.

Example for a URL shortener at 40,000 peak QPS with 100:1 read-to-write ratio:

• Write QPS: 40,000 ÷ 101 ≈ 400 write QPS
• Read QPS: 40,000 - 400 ≈ 39,600 read QPS

Storage Estimation: Record Size × Growth Rate × Retention

Storage estimation follows a straightforward formula:

Daily storage growth = Write QPS × record size × 86,400 seconds
Annual storage = Daily storage × 365 × (1 + overhead factor)

Step 1: Write QPS
From your QPS estimation pipeline above.

Step 2: Estimate record size
Think through each field stored per record. Round up generously:

• User record: user_id (8B) + username (50B) + email (100B) + timestamps (16B) + metadata (100B) ≈ 300 bytes
• Tweet: tweet_id (8B) + user_id (8B) + content (280B in UTF-8) + timestamps (16B) + metadata (50B) ≈ 362 bytes
• URL mapping: short_code (8B) + destination_url (200B) + user_id (8B) + created_at (8B) + click_count (8B) ≈ 232 bytes

For media (images, video), storage estimates are dominated by the binary content, not metadata. A typical profile photo thumbnail: 100KB. A 1080p video: 2-5 GB per hour of content.

Step 3: Daily storage growth

URL shortener: 400 write QPS × 232 bytes × 86,400 ≈ 8 GB/day

Step 4: Annual storage and retention
Multiply daily growth by 365. Add 20% overhead for indices, replication overhead, and system files.

8 GB/day × 365 = 2.9 TB/year × 1.2 overhead = ~3.5 TB/year

Step 5: Multi-year projection
Scale the data by your growth assumptions. A product growing 50% year-over-year will need 1.5× more storage each year:

• Year 1: 3.5 TB
• Year 2: 5.25 TB
• Year 3: 7.9 TB (approaching 10 TB, often a trigger for sharding discussions)

Bandwidth Estimation: QPS × Payload Size

Inbound bandwidth (ingress):

Ingress bandwidth = Write QPS × average request payload size

Example: 400 write QPS × 5 KB per URL creation request = 2,000 KB/s ≈ 2 MB/s inbound

Outbound bandwidth (egress):

Egress bandwidth = Read QPS × average response payload size

Example: 39,600 read QPS × 0.5 KB per redirect response = 19,800 KB/s ≈ 20 MB/s outbound

For a media-heavy system, egress bandwidth dominates. A video platform serving 50,000 concurrent streams at 5 Mbps each = 250 Gbps of egress, this is why video platforms need CDNs.

Memory Estimation: Cache Sizing with the 80/20 Rule

The 80/20 rule (Pareto principle) states that roughly 80% of traffic hits 20% of the data. This means you only need to cache 20% of the working dataset to serve 80% of reads from cache.

Working set size:

Working set = total records × average record size × 0.20

Example for URL shortener:

• Total URL mappings after 1 year: 400 writes/s × 86,400 × 365 ≈ 12.6 billion rows
• But "hot" URLs (recently created, frequently accessed) might only be 10 million URLs
• Working set: 10M × 232 bytes = 2.32 GB

A single Redis instance comfortably holds 32-64 GB. This cache would fit on a modest Redis cluster. The cache hit rate should be 90%+ for a typical URL shortener read pattern.

Cache server sizing:

Number of cache servers = total cache size ÷ memory per server

Example: 2.32 GB working set fits in a single Redis node with plenty of headroom.

Common Reference Numbers

Every engineer should internalize these numbers. They let you do estimation without a calculator:

Powers of 2 (for storage scale):

Latency numbers every engineer should know (approximate):

These numbers tell you why caching matters: reading from RAM (100 ns) is 100,000x faster than reading from SSD (100 µs) and 100,000,000x faster than spinning disk. They also explain why cross-region calls are expensive and why you should minimize them in the critical path.

Read-to-Write Ratio and Its Impact on Architecture

The read-to-write ratio (R:W) is one of the most important inputs to architecture decisions. Most consumer applications are read-heavy:

Implications of high read-to-write ratio:

• Caching is highly effective, hot reads can be served from cache, dramatically reducing database load
• Read replicas pay off, you can scale reads independently by adding read replicas without increasing write capacity
• Denormalization is worth it, duplicating data to optimize for reads reduces join complexity at query time

Implications of low read-to-write ratio (write-heavy):

• Caching is less effective, write-through cache strategies add overhead without proportional benefit
• Write throughput becomes the bottleneck, consider sharding by write key (user ID, event ID)
• Consider append-only write patterns with periodic compaction (Cassandra LSM-tree model)

Growth Projections: 1-Year, 3-Year, 5-Year Horizons

Interviewers often ask you to design for the next 1-5 years. Use simple compound growth:

Data at year N = Initial data × (1 + growth rate)^N

Common growth rates for interview scenarios:

• Conservative: 30% year-over-year
• Moderate: 50% year-over-year
• Aggressive: 100% year-over-year (2x per year)

Design triggers by data volume:

• Under 10 TB: single-region Postgres with read replicas is viable
• 10-100 TB: evaluate partitioning or sharding
• 100 TB+: data lake / data warehouse architecture; consider distributed SQL (Spanner, CockroachDB)
• Petabyte scale: columnar storage (BigQuery, Redshift, Snowflake) for analytics; custom storage for serving

Mermaid Diagram 1: The Estimation Pipeline

flowchart TD
    A["100M DAU\n(given in problem)"]
    B["Active Users at Peak\n100M × 0.10 = 10M users\nduring peak hour"]
    C["Requests per Day\n100M DAU × 10 req/user = 1B req/day"]
    D["Average QPS\n1B ÷ 86,400 sec ≈ 12,000 QPS"]
    E["Peak QPS\n12,000 × 3 (peak factor) = 36,000 QPS\nround to 40,000 QPS"]
    F["Write QPS\n40,000 ÷ 101 ≈ 400 writes/sec\n(100:1 read:write ratio)"]
    G["Read QPS\n40,000 - 400 ≈ 39,600 reads/sec"]
    H["Daily Storage Growth\n400 writes/s × 232B × 86,400 ≈ 8 GB/day"]
    I["Annual Storage\n8 GB × 365 × 1.2 overhead ≈ 3.5 TB/year"]
    J["Egress Bandwidth\n39,600 reads/s × 0.5 KB = 20 MB/s"]

    A --> B
    A --> C
    C --> D
    D --> E
    E --> F
    E --> G
    F --> H
    H --> I
    G --> J

Mermaid Diagram 2: Estimation Results Drive Architecture Decisions

graph LR
    HighQPS["High Peak QPS\n40,000+ QPS"]
    LargeStorage["Large Storage Growth\n3.5 TB/year"]
    HighReadWrite["High Read:Write Ratio\n100:1"]
    LowLatency["Low Latency Target\np99 < 100ms"]

    HorizontalScale["Horizontal Scaling\nMultiple App Server Instances\nbehind Load Balancer"]
    Sharding["Database Sharding\nor Partitioning\n(when > 10 TB)"]
    ReadReplica["Read Replicas\n(scale reads independently)"]
    Caching["Cache Layer\n(Redis, 90%+ hit rate)"]
    CDN["CDN\n(static assets, edge caching)"]

    HighQPS -->|"single server bottleneck"| HorizontalScale
    LargeStorage -->|"single node limit"| Sharding
    HighReadWrite -->|"read bottleneck"| ReadReplica
    HighReadWrite -->|"hot data"| Caching
    LowLatency -->|"reduce DB round-trips"| Caching
    LowLatency -->|"global users"| CDN

Interview Application

Presenting Estimation Confidently Without Over-Investing Time

Estimation should take 3-5 minutes maximum in an interview. The goal is order-of-magnitude accuracy, not exactness. Follow this pacing:

1. State your assumptions out loud (30 seconds): "I'll assume 100M DAU, 10:1 read-to-write ratio, and an average record size of about 250 bytes. Does that match your expectation?"

2. Run the pipeline quickly (2 minutes): DAU → daily requests → QPS → peak QPS. Write the numbers on the whiteboard as you speak.

3. Call out the key implications (1 minute): "At 40,000 peak QPS with a 100:1 read ratio, we have about 400 writes per second and 39,600 reads. That tells me a single Postgres instance can handle the writes, but we need a caching layer in front to serve the reads without hitting the database on every request."

4. Transition to design (30 seconds): "Now that I have a sense of the scale, let me sketch the high-level architecture."

You do not need to estimate everything. Focus on the numbers that drive component decisions: peak QPS (determines horizontal scaling need), storage growth (determines sharding timeline), cache working set size (determines whether caching is cost-effective).

Using Estimation as a Transition to High-Level Design

Estimation creates a natural transition to the design phase. The key insight is that your estimation results directly motivate your first architectural decisions:

• "40,000 peak QPS means we need multiple app servers, so I'll put a load balancer in front."
• "3.5 TB of data per year means we'll hit ~10 TB in 3 years, I'll design the storage layer with future sharding in mind, even if we start with a single shard."
• "90% of reads can be served from a 2.3 GB cache, so I'll add Redis in front of the database to handle the URL resolution hot path."

This is the difference between a candidate who says "I'll add Redis" and a candidate who says "Given that our read-to-write ratio is 100:1 and the hot working set fits in under 3 GB, Redis is highly cost-effective here, a single cache instance should achieve 90%+ hit rate, reducing database load by an order of magnitude."

Handling Pushback: "What If Your Numbers Are Off By 10x?"

This is a test of estimation maturity, not a gotcha. The right response:

"If the numbers are off by 10x, the architecture doesn't change fundamentally, it changes the sizing. If DAU is 10x higher at 1 billion, our peak QPS would be 400,000 instead of 40,000. That changes the number of app server instances and forces us to think about database sharding earlier, but the core components, load balancer, app servers, cache, database, remain the same. I'd revisit the sharding strategy and potentially the caching tier size."

This response shows that you understand the difference between order-of-magnitude changes (10x) that affect component sizing vs. fundamental architecture changes.

Connection to the QPS Estimator Tool

In the Nalar Linuwih practice workspace, the QPS Estimator tool at the top of each practice problem mirrors exactly this estimation pipeline. When you fill in DAU, request distribution, and record size in the workspace, you are practicing the same mental model, DAU → QPS → storage → bandwidth, that you will run verbally in a real interview. Use the practice workspace to build intuition for how changes in scale affect the numbers.

Common Mistakes

Spending too long on exact numbers instead of order-of-magnitude accuracy. If you spend 10 minutes computing 12,673 QPS instead of "roughly 12,000 QPS," you are wasting interview time. Round to the nearest order of magnitude and move on. The interviewer is evaluating whether you can reason about scale, not whether you can do arithmetic.

Forgetting to account for peak vs. average load. Always apply a peak factor to average QPS. A 3-5x multiplier is safe for most consumer applications. Systems with promotional events (flash sales, sporting events) may have 10-20x spikes. Your infrastructure must be sized for peak, not average, or it will fail exactly when it matters most.

Estimating storage without a retention policy. A system that stores every event forever will run out of storage eventually. Always ask: "How long do we retain data?" For a URL shortener, links might never expire. For an event log, you might retain 30 days of hot data and archive the rest to cold storage. The retention period dramatically changes your storage estimate and your storage tier choices.

Not converting between units correctly. Common mistakes:

• Confusing GB (gigabytes) with Gb (gigabits). Bandwidth is often expressed in Gbps (gigabits per second); storage in GB. There is an 8x difference.
• Confusing GiB (gibibytes, 2^30) with GB (gigabytes, 10^9). For rough estimation, treat them as equivalent.
• Forgetting that 86,400 seconds/day ≈ 10^5 seconds. A useful shortcut: 10,000 QPS × 10^5 seconds/day = 10^9 requests/day = 1 billion.

Ignoring read vs. write separately. Estimating only total QPS without splitting reads and writes leads to incorrect component sizing. Reads and writes have very different cost profiles: a database can handle far more reads (especially with read replicas and caching) than writes. Always decompose QPS by read/write ratio.

Key Vocabulary

QPS (Queries Per Second), The number of queries or requests a system processes each second. The primary metric for sizing compute and database capacity. In interviews: "At 40,000 peak QPS, a single application server becomes a bottleneck, we need horizontal scaling."

DAU (Daily Active Users), The number of unique users who engage with the system in a given day. The starting point for all QPS and storage estimation pipelines. In interviews: "I'll assume 100M DAU, if that's wrong by a factor of 10, the architecture changes in sizing but not in structure."

Peak QPS, The maximum QPS the system experiences, typically 3-5x the average during high-traffic periods. Your infrastructure must be sized for peak QPS, not average QPS. In interviews: "Average QPS of 12,000 means peak QPS is roughly 36,000-60,000, I'll design for 50,000 to give ourselves headroom."

Read-to-Write Ratio, The ratio of read operations to write operations in a system. Most consumer applications are read-heavy (10:1 to 100:1). Determines whether caching, read replicas, and denormalization are cost-effective. In interviews: "With a 100:1 read-to-write ratio, caching is extremely effective, 80% of reads can be served from a small working set."

Working Set, The subset of data that is accessed most frequently. For a caching strategy, you need only cache the working set to achieve high hit rates. Typically follows the 80/20 rule: 20% of data accounts for 80% of reads. In interviews: "The hot working set for URL resolution is about 2 GB, that fits comfortably in a single Redis instance."

Back-of-the-Envelope, An informal, approximate calculation using round numbers to establish order-of-magnitude estimates. Not expected to be exact, within 10x of the true value is acceptable. In interviews: "This is a back-of-the-envelope estimate, I'm rounding aggressively, but the order of magnitude should be correct."

Order of Magnitude, A factor of 10. Back-of-the-envelope estimates are considered good if they are within one order of magnitude of the true value. In interviews: "Even if my estimate is off by 2-3x, the architectural choices remain the same, the sizing changes but not the components."

Capacity Planning, The process of determining the resources (servers, storage, bandwidth) needed to meet future demand. In interviews: "Based on 50% annual growth, we'll hit 10 TB of data in 3 years, I'll design the storage layer to shard at that threshold."

Horizontal Scaling, Adding more instances of a component to handle more load, as opposed to upgrading a single instance. Preferred for stateless components. In interviews: "At 40,000 peak QPS, we need at least 4-5 application server instances behind a load balancer."

Sharding, Partitioning data across multiple database nodes by a shard key (e.g., user ID, geographic region). Allows write throughput and storage to scale horizontally. In interviews: "I'll design with future sharding in mind, we can start with a single shard and add more when we hit 10 TB at year 3."

Summary

Resource estimation is the bridge between requirements and architecture. Without it, your design is a guess. With it, every component you propose is justified by a concrete scale requirement.

The estimation pipeline, DAU → daily requests → average QPS → peak QPS → storage growth → bandwidth, is the same regardless of the system. Once you have internalized the pipeline, you can run it in 3 minutes for any problem and immediately know whether you are designing a small, medium, or large system.

The most important outcomes from estimation in an interview are:

1. Peak QPS, determines whether you need horizontal scaling and how many instances
2. Storage growth rate, determines whether sharding is needed and when
3. Working set size, determines whether caching is cost-effective and how to size it
4. Bandwidth, determines whether a CDN is justified

Connect every estimation result to an architecture decision, and you demonstrate exactly the kind of quantitative thinking that senior engineers use when making real infrastructure choices.