System Design Hub

Clarify: read-heavy (100:1 read/write), need custom aliases? analytics?
Write path: generate unique 7-char Base62 ID (counter + Base62 encode, or MD5+truncate), store short_id → long_url in a DB (PostgreSQL or DynamoDB).
Read path: GET /{id} → check Redis cache first → on miss, DB lookup → HTTP 301/302 redirect. Use Redis with TTL for hot URLs (cache-aside).
Scale: Read replicas for DB, CDN for the redirect response, consistent hashing to shard the cache.
Trade-offs: 301 (permanent) vs 302 (temporary) — 301 is cached by browser (fewer hits, less analytics); 302 is not cached (more server hits, full analytics).

Clarify: per user, per IP, per API key? Global or per-service? Hard limit or soft?
Algorithm choice: Sliding Window Counter (most accurate, no boundary spike) — store user:timestamp → count in Redis using sorted sets.
Redis implementation: ZADD user_id:requests timestamp timestamp, then ZREMRANGEBYSCORE to remove old entries, ZCARD to count. If count > limit → 429 Too Many Requests.
Spring Boot: Implement as a HandlerInterceptor or Spring Cloud Gateway filter with Bucket4j + Redis backend.
Edge cases: What if Redis is down? Fail open (allow) or fail closed (block)? Fail open is safer for availability.

Clarify: channels (push, email, SMS), volume (10M/day), priority (transactional > promotional), delivery guarantee (at-least-once).
Architecture: API Service → Kafka topics per channel → Channel Workers → 3rd party (FCM/APNs for push, SendGrid for email, Twilio for SMS).
Reliability: Idempotency key on each notification to prevent duplicates. Dead-letter topic for failed messages. Retry with exponential back-off.
Fan-out at scale: If user has 10M followers, don't send 10M DB writes synchronously — use a fan-out worker that reads the follower list and publishes to Kafka in batches.
Priority: Separate Kafka topics (high-priority, low-priority) with different consumer lag SLAs.

A distributed database across two data centers is separated by a network partition (P is unavoidable). Now you must choose:
CP (Consistent + Partition Tolerant): Return an error or wait until the network heals rather than return stale data. Example: a bank transfer — stale balance data could cause double-spend. Use: ZooKeeper, HBase, etcd.
AP (Available + Partition Tolerant): Return the best available data even if stale. Example: a shopping cart — showing a slightly stale cart is acceptable. Use: Cassandra, DynamoDB (eventually consistent reads), CouchDB.
Nuance: CAP is binary but PACELC adds latency: even without partitions, there's a latency-consistency trade-off. Modern systems like DynamoDB let you choose consistency level per request.

Step-by-step progression (don't jump straight to sharding):
1. Query optimisation — proper indexes, avoid N+1 queries, use EXPLAIN ANALYZE.
2. Connection pooling — HikariCP, tune pool size to 2×CPU+disk spindles.
3. Caching — Redis cache-aside for frequently read, rarely changing data.
4. Read replicas — route all reads to replicas; reduces primary load by 80% for most apps.
5. Vertical scaling — bigger instance (quick win, limited ceiling).
6. Sharding — partition by user ID or region. Adds complexity; do this last.
7. NoSQL migration — if access pattern is key-value or time-series, move that data to a purpose-built store.

A distributed transaction (e.g., place order → reserve inventory → charge payment) cannot use a single DB transaction across services. Saga breaks it into local transactions with compensating actions on failure.
Choreography-based Saga: Each service publishes an event when its step succeeds. Next service listens and acts. On failure, the service publishes a failure event and each prior service runs its compensating action. Simple, loose coupling, hard to track overall state.
Orchestration-based Saga: A central orchestrator (a Spring Boot service) calls each step and handles failures by calling compensating endpoints. Easier to debug and monitor. Recommended when the workflow is complex.
Spring Boot: Implement with Spring Kafka (choreography) or as a Spring State Machine (orchestration). Library: Axon Framework provides built-in saga support.