The Kubernetes clusters deployed in cloud environments use the functionality from the provider to dynamically provision managed load balancers for the services of type LoadBalancer. MetalLB implements the load balancer functionality for local or bare metal Kubernetes clusters that are not deployed in cloud environments.
Kubernetes Service Types
Kubernetes has three types of services:
The cluster IP service assigns an internal IP address to the service, making it accessible only within the cluster.
The node port service assigns a random port number on the node to expose the service endpoint. You can reach the service from outside the cluster specifying the node IP address and the port number.
The load balancer service uses the cloud controller manager to request a managed load balancer from the cloud provider.
Bare metal and local clusters such as minikube do not have this functionality and load balancer services always get a
MetalLB provides a load balancer implementation that can be configured to use ARP (layer 2) or BGP (layer 3) to announce service IP addresses.
The service IP address is published by sending ARP (Address Resolution Protocol) responses in the local network. This method operates in the data link layer of the OSI model (layer 2).
The administrator only has to provide an IP address range to be used by MetalLB.
The service IP address is published with BGP requests to the router. This method operates in the network layer of the OSI model (layer 3).
The administrator has to provide the following items to configure MetalLB with BGP:
The IP address of the router.
The AS (Autonomous System) number of the router.
A private AS number to be used by MetalLB.
The IP address range CIDR prefix to be used by MetalLB.
The BGP configuration method is outside the scope of this material.
MetalLB has two components
It is the main MetalLB component that tracks the creation of the load balancer services and allocates the IP addresses. The controller pod is executed from a deployment.
Manages the advertisement of the IP address associated with them. The speaker daemon set runs a pod on every node
When configured in layer 2 mode, the speaker application sends Gratuitous ARP messages for IPv4 addresses and Unsolicited Neighbor Advertisement for IPv6. If the configuration is in layer 3, then speaker sends BGP advertisements instead.
MetalLB installation options
There are various methods to install MetalLB in a Kubernetes cluster:
Apply the YAML manifests from the MetalLB repository.
kustomizeand reference the
kustomization.yamlfile in the source repository
Add the chart repository and deploy the
Install the MetalLB Operator from OperatorHub.
metallbadd-on in minikube.
Installing MetalLB in Minikube
Minikube clusters have the option to install MetalLB as an add-on. To install MetalLB on minikube, verify the add-on is listed.
[user@host kbe]$ minikube addons list |--------------------------|----------|------------|-----------------------| | ADDON NAME | PROFILE | STATUS | MAINTAINER | |--------------------------|----------|------------|-----------------------| ...output omitted... | metallb | minikube | disabled | unknown (third-party) | ...output omitted... |--------------------------|----------|------------|-----------------------| ? To see addons list for other profiles use: `minikube addons -p name list`
Enable the MetalLB add-on to install the tool.
[user@host kbe]$ minikube addons enable metallb ▪ Using image metallb/controller:v0.9.6 ▪ Using image metallb/speaker:v0.9.6 ? The 'metallb' addon is enabled
Configure the add-on with the range of IP addresses that MetalLB assigns to the load balancer services. CD: Fixed future tense ("will") use
[user@host kbe]$ minikube addons configure metallb -- Enter Load Balancer Start IP: 192.168.59.20 -- Enter Load Balancer End IP: 192.168.59.30 ▪ Using image metallb/speaker:v0.9.6 ▪ Using image metallb/controller:v0.9.6 ✅ metallb was successfully configured
The load balancer IP address range for MetalLB must not overlap with the DHCP IP address range.
The configuration step created a configuration map named
config in the
metallb-system namespace with the MetalLB layer 2 settings. You can edit this configuration map to change the settings.
The IP address pool for MetalLB can be specified as
<start-address>-<end-address> or a CIDR block such as
--- apiVersion: v1 kind: ConfigMap metadata: namespace: metallb-system name: config data: config: | address-pools: - name: default protocol: layer2 # (1) addresses: - 192.168.59.20-192.168.59.30 # (2)
The configuration is set to layer 2 and MetalLB uses ARP.
IP address range for MetalLB
Inspect the components deployed in the
[user@host kbe]$ kubectl get daemonsets -n metallb-system NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE speaker 1 1 1 1 1 beta.kubernetes.io/os=linux 10m [user@host kbe]$ kubectl get deployments -n metallb-system NAME READY UP-TO-DATE AVAILABLE AGE controller 1/1 1 1 10m [user@host kbe]$ kubectl get pods -n metallb-system NAME READY STATUS RESTARTS AGE controller-66bc445b99-tsc68 1/1 Running 0 10m speaker-pcs65 1/1 Running 0 10m
MetalLB creates an endpoint resource with the same name as the load balancer service, verify that the endpoint has an IP address and port number and is not in the
[user@host kbe]$ kubectl get endpoints nginx NAME ENDPOINTS AGE nginx 172.17.0.19:80 27m
Under some circumstances the load balancer service IP address refuses connections, even if it has an external IP address provided by MetalLB.
[user@host kbe]$ kubectl get services -l app=nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx LoadBalancer 10.102.69.228 192.168.59.20 80:30574/TCP 60s
You can verify if the TCP connection to the service can be established by using
[user@host kbe]$ nc -vz 192.168.59.20 80 Ncat: Version 7.70 ( https://nmap.org/ncat ) Ncat: Connection refused.
You can also review if the IP address range is not set in the configuration map for MetalLB.
[user@host kbe]$ kubectl get configmap config -n metallb-system -o yaml --- apiVersion: v1 kind: ConfigMap metadata: namespace: metallb-system name: config data: config: | address-pools: - name: default protocol: layer2 addresses: - - # (1)
The IP address range is empty.
You can assign or change the IP address range for MetalLB by editing the configuration map.
[user@host kbe]$ kubectl edit configmap config -n metallb-system configmap/config edited
You can also extract the resource as YAML, edit the file and apply the resource again to commit the changes.
After the IP address range is configured, the components must be rolled out to apply the changes.
[user@host kbe]$ kubectl rollout restart deployment controller -n metallb-system deployment.apps/controller restarted [user@host kbe]$ kubectl rollout restart daemonset speaker -n metallb-system daemonset.apps/speaker restarted [user@host kbe]$ kubectl get pods -n metallb-system NAME READY STATUS RESTARTS AGE controller-6884978f-l5q4w 1/1 Running 0 20s speaker-qzxtr 1/1 Running 0 10s
You can verify if the service responds after all the components are restarted.
[user@host kbe]$ kubectl get services -l app=nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx LoadBalancer 10.102.69.228 192.168.59.20 80:30574/TCP 60s [user@host kbe]$ nc -vz 192.168.59.20 80 Ncat: Version 7.70 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.59.20 80. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds.
You can also verify if the IP address of the service is associated with the MAC address of the minikube VM.
[user@host kbe]$ minikube ssh ...output omitted... $ ip addr show dev eth1 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 08:00:27:56:de:f2 brd ff:ff:ff:ff:ff:ff # (1) inet 192.168.59.123/24 brd 192.168.59.255 scope global dynamic eth1 valid_lft 586sec preferred_lft 586sec $ exit logout
The MAC address is
Delete the entry for the service IP address from the ARP table on the client machine.
[root@host kbe]# arp -d 192.168.59.20 [root@host kbe]# arp -d 192.168.59.20 No ARP entry for 192.168.59.20
arping tool to discover which MAC address is associated with the IP address of the service.
[root@host kbe]# arping -c 4 -I vboxnet0 192.168.59.20 ARPING 192.168.59.20 from 192.168.59.1 vboxnet0 Unicast reply from 192.168.59.20 [08:00:27:56:DE:F2] 2.460ms Unicast reply from 192.168.59.20 [08:00:27:56:DE:F2] 3.484ms Unicast reply from 192.168.59.20 [08:00:27:56:DE:F2] 1.016ms Unicast reply from 192.168.59.20 [08:00:27:56:DE:F2] 1.061ms Sent 4 probes (1 broadcast(s)) Received 4 response(s)
Review the ARP table on the client machine and verify that the MAC address matches minikube VM.
[root@host kbe]# arp -an 192.168.59.20 ? (192.168.59.20) at 08:00:27:56:de:f2 [ether] on vboxnet0