search

bottlerocket-os / bottlerocket

An operating system designed for hosting containers
https://bottlerocket.dev
Other
8.8k stars 520 forks source link

add support for Firecracker microVMs #812

Open bcressey opened 4 years ago

bcressey commented 4 years ago

We're excited about Firecracker and the ability to provide secure, multi-tenant, minimal-overhead execution of container workloads.

We need to investigate the best way to integrate it, and decide whether we can include it in the Kubernetes variant, or whether it needs its own variant.

Some areas for research:

We would want the microVM kernel and userspace to also be Bottlerocket, and may need a simplified boot path for this use case.

egernst commented 4 years ago

may be interesting to investigate Kata Containers as an integration point as well @bcressey

photoszzt commented 4 years ago

Can I run bottlerock built image on firecracker now? I assume it only needs to change the kernel config according to: https://github.com/firecracker-microvm/firecracker/blob/master/resources/microvm-kernel-config The other things are the same?

kennu commented 2 years ago

I'm also interested in a supported way to run Bottlerocket inside Firecracker VMs. I need to run some user containers in securely isolated VMs, on a shared (bare metal) EC2 instance, and it seems Firecracker + Bottlerocket would be the best combination.

stevehipwell commented 2 years ago

I'm also interested in a supported way to run Bottlerocket inside Firecracker VMs. I need to run some user containers in securely isolated VMs, on a shared (bare metal) EC2 instance, and it seems Firecracker + Bottlerocket would be the best combination.

@kennu do you mean running Firecracker VMs from a Bottlerocket instance?

kennu commented 2 years ago

@stevehipwell No, I mean running Bottlerocket OS inside Firecracker VMs. Firecracker VM needs to run some OS to run Docker containers and it seems Bottlerocket would be well suited.

stevehipwell commented 2 years ago

@kennu so in theory you could be running a Bottlerocket instance hosting a Firecracker VM running Bottlerocket?

I'm just observing this issue but I understood it to be for Bottlerocket to support running Firecracker VMs rather than as the VM OS? I may well be wrong, but that was what I'm here for; running an EKS cluster which supports Firecracker VMs for isolation of certain workloads.

kennu commented 2 years ago

I guess the wording of the original issue is a bit unclear on which way round it is :-)

I can see the need for both. In my current use case, I'm running an EC2 bare metal instance => Amazon Linux (ECS optimized) => Firecracker => customized Ubuntu. I would like to replace the customized Ubuntu with a more supported and lightweight OS (Bottlerocket), but secondarily also replace the Amazon Linux.

Ultimately, I could just launch ECS (or EKS) clusters that would be able to run containers inside Firecracker VMs when isolation is required. Basically same as Fargate, but with more options to optimize and customize the platform as needed.

photoszzt commented 2 years ago

I think the original issue mentioned both: using bottlerock as the os for the bare metal machine and bottlerock as the microVM OS("microVM kernel and userspace to also be Bottlerocket" in the original issue).

CarlosLaraFP commented 1 year ago

I have several ECS Fargate tasks and trying to specify Bottlerocket OS through CDK. Does this capability exist?

stmcginnis commented 1 year ago

Hi @CarlosLaraFP - it is not currently possible to use Bottlerocket for Fargate tasks. I expect that support to be available eventually, but it is not there today.

jpmcb commented 1 year ago

Hi @CarlosLaraFP - currently, through the CDK, you can only specify Amazon Linux, Amazon Linux 2, or Window machines. Here's a reference in Go for the CDK.

If you have a Bottlerocket specific question or problem not related to adding support for Firecracker microVMs, please feel free to open a new issue!

Edit: this is supported in ECS, those bindings are just missing in the Go library: https://github.com/aws/aws-cdk/issues/25598

jpculp commented 1 year ago

I believe the CDK does support Bottlerocket: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.BottleRocketImage.html

stevehipwell commented 6 months ago

Would it be possible for Bottlerocket to support direct configuration of Firecracker instead of the multiple manual steps described in the recent Enhancing Kubernetes workload isolation and security using Kata Containers blog post? It'd be great to just set a configuration flag and have the node configured for Kata Containers, with Firecracker and it's snapshotter enabled.