Coffee Codex - Nitro Sandboxes
Introduction
I’m at Royal Bakehouse in Bellevue, WA, and today I’m reading about AWS Nitro Sandboxes.
Nitro
I’ve written about Nitro before and how it’s special hardware that AWS made to run workloads. The idea is that, before Nitro, workloads looked like this:
and after Nitro, they looked like this:
With Nitro cards, the server is almost 100% the user’s VM and hypervisor instead of allocating 30% to other tasks. Basically, these cards take infrastructure work off of the main CPU.
Note: This is not the same as the Nitro Hypervisor; we’ll get to that later.
Firecracker
I’ve also written about Firecracker/Lambda workers before! The flow for Lambda is generally:
- User requests a Lambda
- Lambda finds an available execution environment, or starts one up
- When a new microVM is needed, a Firecracker process uses Kernel-based Virtual Machine (KVM) to create and run it
- The function runs in the microVM
Note that this flow uses KVM and not the Nitro Hypervisor.
Nitro Hypervisor
Ok, we’re almost there, the last puzzle piece is the Nitro Hypervisor. The Nitro Hypervisor is based on KVM, but specialized for the aforementioned Nitro Cards. The hypervisor is responsible for allocating CPU and memory to VMs and keeping them isolated from each other. The hypervisor can stay lean because a lot of the responsibility has been offloaded to the Nitro cards.
The Nitro hypervisor must live on a machine that is equipped with Nitro cards.
Nitro Sandbox
So how do “traditional” sandbox approaches (like Lambda) compare with the new Nitro Sandboxes? Well, here’s the table:
| Setup | VM manager | Hypervisor |
|---|---|---|
| Traditional Firecracker | Firecracker | KVM |
| Nitro Sandboxes | ? | Nitro |
Hmm, there’s a cell missing. You’re not going to believe this, but it’s Firecracker 🤯.
| Setup | VM manager | Hypervisor |
|---|---|---|
| Traditional Firecracker | Firecracker | KVM |
| Nitro Sandboxes | Firecracker | Nitro |
So Nitro sandboxes have Firecracker communicating with the Nitro hypervisor instead of KVM.
Of course, it’s not that simple and there are a few differences.
KVM already lets us run isolated VMs, as we saw with Lambda. So what’s different here?
With the Nitro Sandboxes approach, the EC2 instance takes advantage of the Nitro System it already runs on. Instead of asking KVM in the instance’s Linux kernel to run the VMs, Firecracker asks the Nitro Hypervisor underneath the instance.
AWS calls the EC2 instance the “parent”, but the sandboxes run alongside it under Nitro. They use CPU and memory from the parent’s existing allocation. This uses AWS’s existing hypervisor to run the workloads instead of a separate KVM layer. EKS and ECS can use this to give each pod or task its own VM.
In direct mode, the parent can still read and write the sandbox’s memory, just like a KVM host can. The difference here is which hypervisor runs the VMs.
Confidential mode goes further. Even if someone gets root on the parent instance, they can’t read or change the sandbox’s private memory. The parent still handles disk and network traffic, so keeping that data private requires encryption too.