Label nodes
To start the Bottlerocket updater operator agent on your nodes, you will need to add the bottlerocket.aws/updater-interface-version label. We recommend using update-interface-version 2.0.0 for Bottlerocket OS version >=v0.4.1 which uses Bottlerocket’s update API to dispatch updates.
With kubectl configured for the desired cluster, you can use the below command to get all nodes:
kubectl get nodes
Make a note of all the node names that you would like the Bottlerocket update operator to manage.
Next, add the updater-interface-version label to the nodes. For each node, use this command to add an updater-interface-version label. Make sure to change NODE_NAME with the name collected from the previous command:
kubectl label node NODE_NAME bottlerocket.aws/updater-interface-version=2.0.0
If all nodes in the cluster are running Bottlerocket and require the same updater-interface-version, you can label all at the same time by running this:
kubectl label node $(kubectl get nodes -o jsonpath=’{.items[*].metadata.name}’) bottlerocket.aws/updater-interface-version=2.0.0
Install
Now we can install the Bottlerocket update operator using the recommended configuration defined here:
kubectl apply -f ./update-operator.yaml
Coordination
The update operator controller and agent processes communicate by updating the node’s annotations as the node steps through an update. The node’s annotations are used to communicate an intent that acts as a goal or target that is set by the controller. The controller uses internal policy checks to manage which intent should be communicated to an agent. This allows the controller to fully own and coordinate each step taken by agents throughout its cluster. No agent process will otherwise take any disruptive or intrusive action without being directed by the controller to do so (in fact the agent is limited to periodic metadata updates only).
To handle and respond to intents, the agent and controller processes subscribe to Kubernetes’ node resource update events. These events are emitted whenever an update is made on the subscribed to resource, including heartbeats, other node status changes (pods, container image listing), and metadata changes (labels and annotations).
Observing State
The update operator’s state can be closely monitored through the labels and annotations on node resources. The state and pending activity are updated as progress is being made. The following command requires kubectl to be configured for the development cluster to be monitored and jq to be available on $PATH.
kubectl get nodes -o json \
| jq -C -S ‘.items | map(.metadata|{(.name): (.annotations*.labels|to_entries|map(select(.key|startswith(“bottlerocket.aws”)))|from_entries)}) | add’
There is a get-nodes-status Makefile target provided for monitoring nodes during development. Note: the same dependencies and assumptions for the above command apply here.
# get the current status:
make get-nodes-status
# or periodically (handy for watching closely):
watch -c — make get-nodes-status
Image Region
update-operator.yaml pulls operator images from Amazon ECR Public. You may also choose to pull from regional Amazon ECR repositories such as the following.
- 917644944286.dkr.ecr.af-south-1.amazonaws.com
- 375569722642.dkr.ecr.ap-east-1.amazonaws.com
- 328549459982.dkr.ecr.ap-northeast-1.amazonaws.com
- 328549459982.dkr.ecr.ap-northeast-2.amazonaws.com
- 328549459982.dkr.ecr.ap-south-1.amazonaws.com
- 328549459982.dkr.ecr.ap-southeast-1.amazonaws.com
- 328549459982.dkr.ecr.ap-southeast-2.amazonaws.com
- 328549459982.dkr.ecr.ca-central-1.amazonaws.com
- 328549459982.dkr.ecr.eu-central-1.amazonaws.com
- 328549459982.dkr.ecr.eu-north-1.amazonaws.com
- 586180183710.dkr.ecr.eu-south-1.amazonaws.com
- 328549459982.dkr.ecr.eu-west-1.amazonaws.com
- 328549459982.dkr.ecr.eu-west-2.amazonaws.com
- 328549459982.dkr.ecr.eu-west-3.amazonaws.com
- 509306038620.dkr.ecr.me-south-1.amazonaws.com
- 328549459982.dkr.ecr.sa-east-1.amazonaws.com
- 328549459982.dkr.ecr.us-east-1.amazonaws.com
- 328549459982.dkr.ecr.us-east-2.amazonaws.com
- 328549459982.dkr.ecr.us-west-1.amazonaws.com
- 328549459982.dkr.ecr.us-west-2.amazonaws.com
Current Limitations
- pod replication & healthy count is not taken into consideration (https://github.com/bottlerocket-os/bottlerocket/issues/502)
- nodes update without pause between each node (https://github.com/bottlerocket-os/bottlerocket/issues/503)
- single-node cluster degrades into unschedulable on update (https://github.com/bottlerocket-os/bottlerocket/issues/501)
- node labels are not automatically applied to allow scheduling (https://github.com/bottlerocket-os/bottlerocket/issues/504)
Updating Process
The Bottlerocket operating system includes primary and secondary partitions which are completely identical. When the OS is updated, only the inactive partition is updating simultaneously and when it’s finished the partition table is automatically updated to swap both partition sets.
Bottlerocket downloads a full file system image and reboots into it unlike other general-purpose Linux distributions based on package manager. In case of boot failures, it can automatically roll back and workload failures can stimulate manual rollbacks. Therefore, the updating process is faster, easier, and more secure.
How Much Does It Cost?
Bottlerocket is open-source and doesn’t require any extra payment, the only thing you have to pay for is AWS and Amazon EC2.
Final Note
Bottlerocket is the peak of AWS experience in running containers at scale. It helps enterprises run containers efficiently, ensure better security, and remove obstacles by contributing unique expertise and code. Its architecture is flexible enough to support a variety of container orchestrators and cloud environments. Though currently it can’t be called the number one OS for the most effective running of containers by the public, experts predict its rapid growth in the near future.
