Skip to content

Setting up Load Balancing

Load balancing in Unikraft Cloud is very easy: as soon as you attach more than one instance to a service, UKC will automatically start balancing traffic between the multiple instances. The load balancing is done based on the number of connections (in TCP mode) or requests (in HTTP mode); more on that below. This diagram illustrates the overall mechanism:

Load Balancing in Unikraft Cloud Load Balancing in Unikraft Cloud

Because of load balancing, instances in a service must be of the same kind (e.g., expose the same port). You can remove instances from a service at any time, and, when you do, UKC will immediately take the instance out of the load balancing service.

To set load balancing up, first use kraft cloud deploy with the -p flag so that a service is created as part of the instance creation. For example, let’s use NGINX as the app:

Terminal window
git clone https://github.com/unikraft-cloud/examples
cd examples/nginx/
kraft cloud deploy -p 443:8080 .

This single command will (a) create a service via the -p flag and (b) start an NGINX instance:

Terminal window
[] Deployed successfully!
────────── name: nginx-8ujeu
────────── uuid: d6238ac6-27d2-47b3-8a45-c6cac99fb4ef
───────── state: running
─────────── url: https://wandering-shape-n6mhimgn.fra0.kraft.host
───────── image: nginx@sha256:269192f523dca7498423bc54676ab08e415e9c7442d1bd3d65f07ab5e50a43d
───── boot time: 20.18 ms
──────── memory: 128 Mi
─────── service: wandering-shape-n6mhimgn
── private fqdn: nginx-8ujeu.internal
──── private ip: 172.16.6.7
────────── args: /usr/bin/nginx -c /etc/nginx/nginx.conf

With this in place, it’s now time to start a second instance and attach it to the service that was just created (named, in this case, wandering-shape-n6mhimgn):

Terminal window
cd examples/nginx/
kraft cloud deploy --service wandering-shape-n6mhimgn .

The command’s output should be similar to this one:

Terminal window
[] Deployed successfully!
────────── name: nginx-djta3
────────── uuid: 06c972a6-a117-4b07-8eba-9389b4cccb42
───────── state: running
─────────── url: https://wandering-shape-n6mhimgn.fra0.kraft.host
───────── image: nginx@sha256:c00c11a5cbd6a3020dd4d9703fbeb2a2f2aab37f18f7a0ba9c66db5a71897c3a
───── boot time: 20.46 ms
──────── memory: 128 MiB
______─ service: wandering-shape-n6mhimgn
── private fqdn: nginx-djta3.internal
──── private ip: 172.16.6.3
────────── args: /usr/bin/nginx -c /etc/nginx/nginx.conf

Notice that both the url and service fields in the 2 instances are the same, as they should be. To check that it worked, run the following command:

Terminal window
kraft cloud service list -o list

You should see output such as:

uuid: 954b4a52-fc51-4eac-b5c9-1fc2a368a237
name: wandering-shape-n6mhimgn
fqdn: wandering-shape-n6mhimgn.fra0..kraft.host
services: 443:8080/tls+http 80:443/http+redirect
instances: 06c972a6-a117-4b07-8eba-9389b4cccb42 d6238ac6-27d2-47b3-8a45-c6cac99fb4ef
created at: 14 minutes ago
persistent: true

Note the two instances (their UUIDs) under the instances field. You’re now load balancing across 2 NGINX instances!

Load Balancing Algorithm

The load balancing algorithm is a variant of least_conn. For every instance, we track the number of current in-flight TCP connections (if in tcp mode) or requests (if in http mode).

To select an instance, we go over all instances in the service and find the set of instances that have the least amount of in-flight requests/connections, and pick randomly from that set.

To illustrate, imagine we had the following scenario:

Instance ID | # in-flight conns
i-0 | 4
i-1 | 1
i-2 | 2
i-3 | 1

In this case, the algorithm would first choose instances i-1 and i-3, since they both have the least number of connections at the moment (only 1 each). After that, the algorithm would choose randomly between these 2 instances and assign the new connection to it. For example, if it chose i-1, the next new connection would go to i-3 since it’d be now the only instance with only 1 connection (assuming none of the connections that the other instances are handling are closed).

Learn More

  • The kraft cloud CLI reference, and in particular the services sub-command
  • Unikraft Cloud’s REST API reference, and in particular the section on services