Origin Story
- This began a year of product improvement and protocol iteration.
- HLS, HTTP Live Streaming
- RTMP (Real-Time Messaging Protocol), a TCP based protocol.
- MPEG-DASH (Dynamic Adaptive Streaming over HTTP).
Live Video Is Different And That Causes Problems
- The traffic pattern of the Watermelon video mentioned earlier:
- A very steep initial rise
- Then traffic dropped like a rock.
- traffic is spiky.
- Caching Problems
- A lot of people may want to watch a live video at the same time. This is your classic Thundering Herd problem.
- Global Load Balancing Problem
- Facebook has points of presence (PoPs) distributed around the world. Facebook traffic is globally distributed.
Big Picture Architecture
How Does It Scale?
- There is one point of multiplication between the datacenter cache and the many PoP caches. Users access PoP caches, not the datacenter, and there are many PoP caches distributed around the world.
- Another multiplication factor is within each PoP.
Protecting The Datacenter From The Thundering Herd
- Request Coalescing. The number of requests is reduced by adding request coalescing to the PoP cache. Only the first request is sent to the datacenter. The other requests are held until the first response arrives and the data is sent to all the viewers.
- New caching layer is added to the proxy to avoid the Hot Server problem.
Lessons learned
- Investigating a push mechanism rather than the request-pull mechanism, leveraging HTTP/2 to push to the PoPs before segments have been requested.
Problems
- Millions of simultaneous streams.
- Millions of simultaneous viewers on a stream, as well as seamless streams across different devices and service providers around the world.
- Live video is different than normal videos: it causes spiky traffic patterns.
- Caching Problems
- Global Load Balancing Problem
Research on frequently seen technology.
- A content delivery network or content distribution network (CDN) is a globally distributed network of proxy servers deployed in multiple data centers.
- The goal of a CDN is to serve content to end-users with high availability and high performance.