Setting up Load Balancing

Load balancing in Unikraft Cloud is very easy: as soon as you attach more than one instance to a service, UKC will automatically start balancing traffic between the multiple instances. The load balancing is done based on the number of connections (in TCP mode) or requests (in HTTP mode); more on that below. This diagram illustrates the overall mechanism:

Because of load balancing, instances in a service must be of the same kind (e.g., expose the same port). You can remove instances from a service at any time, and, when you do, UKC will immediately take the instance out of the load balancing service.

To set load balancing up, first use kraft cloud deploy with the -p flag so that a service is created as part of the instance creation. For example, let’s use NGINX as the app:

git clone https://github.com/unikraft-cloud/examples
cd examples/nginx/
kraft cloud deploy -p 443:8080 .

This single command will (a) create a service via the -p flag and (b) start an NGINX instance:

[●] Deployed successfully!
 │
 ├────────── name: nginx-8ujeu
 ├────────── uuid: d6238ac6-27d2-47b3-8a45-c6cac99fb4ef
 ├───────── state: running
 ├─────────── url: https://wandering-shape-n6mhimgn.fra0.kraft.host
 ├───────── image: nginx@sha256:269192f523dca7498423bc54676ab08e415e9c7442d1bd3d65f07ab5e50a43d
 ├───── boot time: 20.18 ms
 ├──────── memory: 128 Mi
 ├─────── service: wandering-shape-n6mhimgn
 ├── private fqdn: nginx-8ujeu.internal
 ├──── private ip: 172.16.6.7
 └────────── args: /usr/bin/nginx -c /etc/nginx/nginx.conf

With this in place, it’s now time to start a second instance and attach it to the service that was just created (named, in this case, wandering-shape-n6mhimgn):

cd examples/nginx/
kraft cloud deploy --service wandering-shape-n6mhimgn .

Tip

If you get a prompt saying “deployment already exists: what would you like to do with the 1 existing instance(s)?”, simply choose keep.

The command’s output should be similar to this one:

[●] Deployed successfully!
 │
 ├────────── name: nginx-djta3
 ├────────── uuid: 06c972a6-a117-4b07-8eba-9389b4cccb42
 ├───────── state: running
 ├─────────── url: https://wandering-shape-n6mhimgn.fra0.kraft.host
 ├───────── image: nginx@sha256:c00c11a5cbd6a3020dd4d9703fbeb2a2f2aab37f18f7a0ba9c66db5a71897c3a
 ├───── boot time: 20.46 ms
 ├──────── memory: 128 MiB
 ├______─ service: wandering-shape-n6mhimgn
 ├── private fqdn: nginx-djta3.internal
 ├──── private ip: 172.16.6.3
 └────────── args: /usr/bin/nginx -c /etc/nginx/nginx.conf

Notice that both the url and service fields in the 2 instances are the same, as they should be. To check that it worked, run the following command:

kraft cloud service list -o list

You should see output such as:

       uuid: 954b4a52-fc51-4eac-b5c9-1fc2a368a237
       name: wandering-shape-n6mhimgn
       fqdn: wandering-shape-n6mhimgn.fra0..kraft.host
   services: 443:8080/tls+http 80:443/http+redirect
  instances: 06c972a6-a117-4b07-8eba-9389b4cccb42 d6238ac6-27d2-47b3-8a45-c6cac99fb4ef
 created at: 14 minutes ago
 persistent: true

Note the two instances (their UUIDs) under the instances field. You’re now load balancing across 2 NGINX instances!

Tip

Currently it’s not possible to set the service’s name via kraft cloud deploy. If you’d like to set the service’s name, first use the kraft cloud service command to create the service, and then use the --service flag of kraft cloud deploy to attach the instance being created to the service, as outlined in this guide

Tip

If you want to make sure that no existing connections are dropped when stopping an instance, use UKC’s draining feature by stopping an instance through kraft cloud instance stop -w TIMEOUT_MS

Load Balancing Algorithm

The load balancing algorithm is a variant of least_conn. For every instance, we track the number of current in-flight TCP connections (if in tcp mode) or requests (if in http mode).

To select an instance, we go over all instances in the service and find the set of instances that have the least amount of in-flight requests/connections, and pick randomly from that set.

To illustrate, imagine we had the following scenario:

Instance ID | # in-flight conns
i-0         |         4
i-1         |         1
i-2         |         2
i-3         |         1

In this case, the algorithm would first choose instances i-1 and i-3, since they both have the least number of connections at the moment (only 1 each). After that, the algorithm would choose randomly between these 2 instances and assign the new connection to it. For example, if it chose i-1, the next new connection would go to i-3 since it’d be now the only instance with only 1 connection (assuming none of the connections that the other instances are handling are closed).

Learn More

• The kraft cloud CLI reference, and in particular the services sub-command
• Unikraft Cloud’s REST API reference, and in particular the section on services