Load Balancing

Goals: To distribute load onto different servers, and to provide redundancy / failover for each other.

redundancy: if one server in a cluster of servers fail, the load balancer can temporarily remove that server from the cluster, and divide the load onto the functioning servers.
failover: When an error happens and the tasks is moved from the failing server to a functioning server

Common Load Balancing Schemes

An even task distribution scheme means that the tasks are distributed evenly between the servers in the cluster.
Suitable when the servers in the cluster all have the same capacity, and the incoming tasks statistically require the same amount of work.
Problems:
- Even task distribution ignores the difference in the work required to process each task.

Idea: configure your DNS to return different IP addresses to different computers when they request an IP address for your domain name.

Idea: distributes the incoming tasks onto the servers in the cluster using weights.
- That means that you can specify the weight (ratio based on each server's capacity) of tasks a server should receive relative to other servers.
Sutitable: when the servers in the cluster do not all have the same capacity.
Problem: does not take the work required to process the tasks into consideration.

The two previous load balancing schemes are based on the assumption that any incoming task can be processed independently of previously executed tasks. This may not always be the case though.
Idea: All tasks (e.g. HTTP requests) belonging to the same session (e.g the same user) are sent to the same server.
With sticky session load balancing it isn't the tasks that are distributed out to the servers, but rather the task sessions.
Problems: This will of course result in a somewhat more unpredictable distribution of work load, as some sessions will contain few tasks, and other sessions will contain many tasks.
Another solution is to avoid using session variables completely, or to store the session variables in a database or cache server, accessible to all servers in the cluster.

Idea: the load balancer keeps a task queue for each server.
- this load balancing scheme implicitly takes both the work required to process each task and the capacity of each server into consideration.

The load balancer will have to do a bit more accounting using this scheme. It has to keep track of task queues, and it has to keep track of when a task is completed, so it can be removed from the corresponding task queue.

Idea: all incoming tasks are stored in a task queue. The servers in the server cluster connects to this queue and takes the number of tasks they can process.

In this scheme there is no real load balancer. Each server takes the load it is able to handle. There is just the task queue and the server.