Locality-Aware Request Distribution in Cluster-Based Web Servers
Locality-Aware Request Distribution in Cluster-Based Web Servers
Paper [1]: “Locality-Aware Request Distribution in Cluster-Based Network Services” by V. Pai et al. (ASPLOS, 1998)
Paper [2]: “Scalable Content-Aware Request Distribution in Cluster-Based Network Servers” by M. Aron et al. (USENIX, 2000)
TL;DR
These two papers tackle the problem of intelligently distributing requests across web server clusters. The first introduces LARD, which routes requests to maximize backend cache locality while maintaining load balance. The second paper identifies LARD’s scalability limitations and proposes a new architecture where a simple layer-4 switch handles distribution and backends themselves forward requests to peers based on content, achieving significantly better scalability.
Summary
The first paper introduces LARD – Locality-Aware Request Distribution – a content-aware request distribution strategy for web server clusters. With LARD, the front-end distributes incoming requests in a manner that achieves high locality in the backends’ main memory caches while also maintaining load balance. The paper also introduces a TCP handoff protocol that can hand off an established TCP connection in a client-transparent manner. The authors validate their approach through both simulation and a working prototype, demonstrating that LARD outperforms the existing Weighted Round Robin (WRR) method.
The second paper identifies the drawbacks and limitations of LARD and proposes a new architecture where the frontend is a layer-4 switch that distributes incoming requests to a number of backends. In this design, the switch does not perform any content-aware distribution – it simply uses weighted round robin. Each backend server is connected to other backends and may forward an incoming request to another backend based on the requested content. The authors experimentally show that this new method scales much better than existing approaches across a variety of workloads.
Strengths
- The first paper validates its ideas both through simulation and with a real working prototype. This is a strong point because the prototype confirms the simulation results in practice.
- The second paper clearly explains why LARD’s limitations cannot be overcome by simply adding multiple front-end nodes. This analysis is crucial as it establishes the motivation and research foundation for the second paper.
- The discussion of batching in the second paper is valuable. Batching is an effective concept for reducing network traffic between backend servers.
Weaknesses
- The first paper should have discussed hashing strategies in more depth, since hashing is important for effective load balancing.
- The first paper mentions a “training phase” to set the T_low value but provides no further description of how this training works.
- In the second paper, the authors state that multiple frontend nodes are needed for some workloads but fail to give any concrete examples. Supporting claims with specific workload examples would have strengthened the argument.
Discussion Questions
- The first paper mentions caching on a whole-file basis. What happens when a file is larger than the cache capacity? The authors have not conducted any experiments to demonstrate this effect.
- The first paper fixes T_low = 25 and T_high = 65. A graph showing throughput under different low and high threshold values would have been helpful, since these thresholds determine how many nodes are required to handle incoming requests.
- The testing data in the first paper should have been more comprehensive. The authors only use a Rice University trace for testing. It would be interesting to see results across different workloads – small files, very large files, and dynamic content that takes a long time to generate.
- In the second paper, Figures 2 and 3 show the splicing front-end using persistent connections, while the TCP handoff data (taken from the first paper) uses HTTP 1.0. It is unclear whether these data points can be directly compared. The authors do not mention re-running the TCP handoff experiments with persistent connections.
- The second paper uses a software switch implementing WRR. It would be interesting to see whether a hardware switch can deliver the same performance. Ideally, the authors should have compared software versus hardware switch results, since their entire argument relies on using switches to make clustering efficient.
This review was written as part of CMU’s 18-845: Internet Services course.