Contact us
10 minutes

After being announced almost a year ago, AWS Bottlerocket is tremendously gaining popularity. While a rationalized operating system for the more effective running of containers isn’t a breakthrough, Bottlerocket’s focus on security and native integration with popular AWS services is giving it privilege over alternative solutions.

Here is a very simple, yet great example of Bottlerocket operated nodes(kudos to Andrey Devyatkin):

What is AWS Bottlerocket?

AWS Bottlerocket is a new open-source Linux-based OS that includes only essential software to run containers and comes with a transactional update mechanism. This feature allows users to utilize container orchestrators in order to run operating system updates with the slightest disruption, enabling reduced security attack surface and enhanced resource usage.

Here you can find a very simple example of AWS Bottlerocket based nodes.

What Are Containers?

The code and dependencies that are needed for running an app in the container’s environment are encapsulated by a unit called a container. All the containers are isolated from each other though share the same OS. Contrary to virtual machines, containers are speedy and boot up faster as they leverage the host OS.

Kubernetes, also known as K8s is a very popular open-source system for automating deployment, scaling, and managing containerized applications. Other popular container orchestration services include Amazon ECS, Red Hat, Helios, etc.

While running container-based environments brings a range of advantages, it also presents some challenges for users such as performance issues, difficulties with updating processes, and security issues.

What are the Main Components of Bottlerocket?

  • Bottlerocket contains a minimal operating system with Linux Kernel and system software incorporated.
  • It Includes an admin container that enables complex debugging and problem fixing.
  • By integration with container orchestrators such as the Amazon EKS platform, it manages and orchestrates updates.
  • A single-step atomic update mechanism allows applying and rolling back operating system updates simultaneously.

What is a Bottlerocket Update Operator?

A Kubernetes Operator is responsible for Bottlerocket updates on hosts in a cluster. As soon as you’ve installed it, the Bottlerocket update operator begins a controller deployment on one node and agent daemon det on every Bottlerocket node. Bottlerocket node, in turn, is responsible for draining the node, managing periodically querying updates, and performing them when it’s required by the controller.

What Are the Benefits of AWS Bottlerocket?

  • Bottlerocket improves uptime and significantly reduces operational costs, as thousands of updates to the OS can be applied simultaneously with minimal disruptions to the applications and rolled back if needed excluding the risk of errors.
  • Bottlerocket ensures better resource utilization and security because of including only essential components. The integrity of exchanged containers is checked by using a cryptographic digest and any corruption or anomaly can restart the whole process.
  • Compared to Red Hat, Ubuntu, or other Linux platforms, Bottlerocket provides a more concordant host deployment environment. Every deployed instance is the same as the last.
  • With the help of orchestration services like Amazon EKS, updates can be easily automated as soon as they are available. Contrary to traditional Linux-based systems that utilize package-by-package updates, Bottlerocket uses image-based updates. AWS claims that the updating process is as simple as updating your phone.
  • Bottlerocket is open-source and universally available so anyone interested can make code, design, and dataset changes to it. Users also get everything required due to excellent support by AWS such as Amazon EKR, Amazon EKS, Amazon EC2, etc.
benefits of Bottlerocket

How is Bottlerocket different from Amazon Linux?

Amazon Linux is a general-purpose OS to run a wide range of applications that are packaged with the RPM Package Manager or containers. Amazon Linux is optimized to provide the ability to configure each instance as necessary for its workload using traditional tools such as yum, ssh, tcpdump, netconf, etc. Bottlerocket, on the other hand, is purpose-built for running containers and allows you to manage a large number of container hosts identically with automation. Specifically, Bottlerocket differs from Amazon Linux in the following ways:

  • Bottlerocket does not have a package manager and software can only be run as containers. Updates to Bottlerocket are applied and can be rolled back in a single atomic step, which reduces update errors.
  • The primary mechanism to manage Bottlerocket hosts is with a container orchestrator like Amazon EKS. Unlike Amazon Linux, logging into individual Bottlerocket instances is intended to be an infrequent operation for advanced debugging and troubleshooting.

Bottlerocket Installation

The Bottlerocket update operator is a Kubernetes operator that coordinates Bottlerocket updates on hosts in a cluster. When installed, the Bottlerocket update operator starts a controller deployment on one node and agent daemon set on every Bottlerocket node, which takes care of periodically querying updates, draining the node, and performing an update when asked by the controller. Updates to Bottlerocket are rolled out in waves to reduce the impact of issues; the nodes in your cluster may not all see updates at the same time.

To install the Bottlerocket update operator in a Kubernetes cluster, the following are required resources and configuration (suggested deployment is defined in update-operator.yaml):

  • Update operator’s container image.
  • Holding the Operator’s binaries and supporting environment (CA certificates).
  • Controller deployment.
  • Schedules a stop-restart-tolerant controller process on available nodes.
  • Agent daemon set.
  • Schedules agent on Bottlerocket hosts.
  • Bottlerocket namespace.
  • Groups Bottlerocket related resources and roles.
  • Service account for the agent.
  • Used for authenticating the agent process on Kubernetes APIs.
  • Cluster privileged credentials with read-write access to nodes for the agent.
  • Grants the agent’s service account permissions to update annotations for its node.
  • Service account for the controller.
  • Used for authenticating the controller process on Kubernetes APIs.
  • Cluster privileged credentials with access to pods and nodes for controller.
  • Grants the controller’s service account permissions to update annotations and manage pods that are scheduled on nodes (to cordon & drain) before and after updating.

Once the deployment’s resources are in place, there is one more step needed to schedule and place the required pods on Bottlerocket nodes. By default — in the suggested deployment, each Workload resource constraints scheduling of the update operator by limiting pods to Bottlerocket nodes based on their labels. These labels are not applied on nodes automatically and will need to be set on each using kubectl. The agent relies on each node’s updater components and schedules its pods based on their interface supported. The node indicates its updater interface version in a label called Agent deployments, respective to the interface version, are scheduled using this label and target only a single version in each.

  • For Bottlerocket OS versions >= v0.4.1, we recommend using update-interface-version 2.0.0 to leverage Bottlerocket’s API to dispatch updates.
  • Bottlerocket OS versions < v0.4.1 are only compatible with update-interface-version 1.0.0.
  • With this version, the agent needs to run in a privileged container with access to the root filesystem.

For the 2.0.0 updater-interface-version, this label looks like:

Each workload resource may have additional constraints or scheduling affinities based on each node’s labels in addition to the label scheduling constraint.

Customized deployments may use the suggested deployment as a starting point, with customized container images specified if needed.

Scheduled Components

The update operator system is deployed as a replica set (for the controller) and a daemon set (for the agent). Each runs their respective process configured as either a -controller or an -agent:

  • bottlerocket-update-operator -controller
  • The coordinating process is responsible for the handling update of Bottlerocket nodes cooperatively with the cluster’s workloads.
  • bottlerocket-update-operator -agent
  • The on-host process is responsible for publishing update metadata and executing update activities.

Getting Started

Bottlerocket installation

Label nodes

To start the Bottlerocket updater operator agent on your nodes, you will need to add the label. We recommend using update-interface-version 2.0.0 for Bottlerocket OS version >=v0.4.1 which uses Bottlerocket’s update API to dispatch updates.

With kubectl configured for the desired cluster, you can use the below command to get all nodes:

kubectl get nodes

Make a note of all the node names that you would like the Bottlerocket update operator to manage.

Next, add the updater-interface-version label to the nodes. For each node, use this command to add an updater-interface-version label. Make sure to change NODE_NAME with the name collected from the previous command:

kubectl label node NODE_NAME

If all nodes in the cluster are running Bottlerocket and require the same updater-interface-version, you can label all at the same time by running this:

kubectl label node $(kubectl get nodes -o jsonpath=’{.items[*]}’)


Now we can install the Bottlerocket update operator using the recommended configuration defined here:

kubectl apply -f ./update-operator.yaml


The update operator controller and agent processes communicate by updating the node’s annotations as the node steps through an update. The node’s annotations are used to communicate an intent that acts as a goal or target that is set by the controller. The controller uses internal policy checks to manage which intent should be communicated to an agent. This allows the controller to fully own and coordinate each step taken by agents throughout its cluster. No agent process will otherwise take any disruptive or intrusive action without being directed by the controller to do so (in fact the agent is limited to periodic metadata updates only).

To handle and respond to intents, the agent and controller processes subscribe to Kubernetes’ node resource update events. These events are emitted whenever an update is made on the subscribed to resource, including heartbeats, other node status changes (pods, container image listing), and metadata changes (labels and annotations).

Observing State

The update operator’s state can be closely monitored through the labels and annotations on node resources. The state and pending activity are updated as progress is being made. The following command requires kubectl to be configured for the development cluster to be monitored and jq to be available on $PATH.

kubectl get nodes -o json \

| jq -C -S ‘.items | map(.metadata|{(.name): (.annotations*.labels|to_entries|map(select(.key|startswith(“”)))|from_entries)}) | add’

There is a get-nodes-status Makefile target provided for monitoring nodes during development. Note: the same dependencies and assumptions for the above command apply here.

# get the current status:

make get-nodes-status

# or periodically (handy for watching closely):

watch -c — make get-nodes-status

Image Region

update-operator.yaml pulls operator images from Amazon ECR Public. You may also choose to pull from regional Amazon ECR repositories such as the following.


Current Limitations

  1. pod replication & healthy count is not taken into consideration (
  2. nodes update without pause between each node (
  3. single-node cluster degrades into unschedulable on update (
  4. node labels are not automatically applied to allow scheduling (

Updating Process

The Bottlerocket operating system includes primary and secondary partitions which are completely identical. When the OS is updated, only the inactive partition is updating simultaneously and when it’s finished the partition table is automatically updated to swap both partition sets.

Bottlerocket downloads a full file system image and reboots into it unlike other general-purpose Linux distributions based on package manager. In case of boot failures, it can automatically roll back and workload failures can stimulate manual rollbacks. Therefore, the updating process is faster, easier, and more secure.

How Much Does It Cost?

Bottlerocket is open-source and doesn’t require any extra payment, the only thing you have to pay for is AWS and Amazon EC2.

Final Note

Bottlerocket is the peak of AWS experience in running containers at scale. It helps enterprises run containers efficiently, ensure better security, and remove obstacles by contributing unique expertise and code. Its architecture is flexible enough to support a variety of container orchestrators and cloud environments. Though currently it can’t be called the number one OS for the most effective running of containers by the public, experts predict its rapid growth in the near future.

0 people like this

This website uses cookies to ensure you get the best experience on our website.

Learn more
Thank you for getting in touch!
We'll get back to you soon.
Sending error!
Please try again later.
Thank you, your message has been sent.
Please try again later, or contact directly through email:
Format: doc, docx, rtf, txt, odt, pdf (5Mb max size)
Thank you, your message has been sent.
Please try again later, or contact directly through email: