## Notes [YouTube System Design - YouTube](https://www.youtube.com/watch?v=jPKTo1iGQiE) ## Functional Requirements - Upload videos - Watch videos ## Non-functional Requirements - Reliability - No video corruption on upload - Scale - 1 video can have 1,000 concurrent users - 1 billion viewers daily - 100 users watching vs 1 video upload - 10,000,000 million uploads - Low views on the majority of videos - Availability vs Consistency - Better to have availability - Updates don't really have to be consistent - Latency - Lowest latency as possible - Videos/ads should play as soon as possible ## Design ### Upload - Loadbalancer in front of a collection of servers - Upload workflow - Video - Use object storage for raw video - Use a queue to encode the raw video - Takes a while - Can be done asynchronously - Metadata - Title, description, file type, user ID - NoSQL like MongoDB, lots of reads, store reference to video file and user ID ### Watching - Introduce caching so that watches can be fast - CDN pulls video from Encoded Object Store - Server caches video metadata for newly uploaded and popular - Don't need to buffer whole video - Only need to buffer segments - Most of the time will watch from the beginning - Can use HTTP to fetch pre-buffered parts of the video - Protocol to load videos? - Live videos - Use UDP because interruptions should be skipped instead of replayed (sports game) - Regular videos - Use TCP for reliability because each moment in a movie is more important ### Scaling - Encoding service - 10 million videos per day - Assume 1 worker per video - *ASSUME*: 1 video means 1 minute to encode - 500 videos per second - Can't have 500 workers, would eventually need 30k workers because they take 1 minute per video - Users in different geographical regions will upload at different times - Can encode in chunks and make them available ### Recommendation/Search - Track user history - Compare metadata ## Actual - Used MySQL for metadata - MongoDB didn't exist at the time - Initially added read-only replicas - Then sharded - Then migrated to [[Vitess]] to decouple application layer from database layer because the application server should not know details about shards/partitions - *Vitess* handles sharding logic