[EKS] [request]: CRI-O support

[brianmori]

Tell us about your request What do you want us to build?

Which service(s) is this request for? EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

CentOS and RHEL 8 have replaced Docker with Buildah and Podman mostly for better security reasons.

Here is a link: https://developers.redhat.com/blog/2019/02/21/podman-and-buildah-for-docker-users/

We would like to have the possibility to use Podman instead of Docker in EKS to not have the Docker daemon running as root for unnecessary reasons.

Are you currently working around this issue? No, we use Docker at this moment

Additional context None

Attachments None

[whereisaaron]

Hi @brianbordini, I think you are mixing up container image build tools with container runtimes. podman and buildah are not alternative runtimes for Kubernetes, but you can choose to use them for building container images and to run them (in the absence of Kubernetes).

Kubernetes switched from the docker runtime to containerd in 2017, and containerd went GA with Kubernetes 1.10 and containerd 1.1 in 2018. So running docker as the container runtime (and thus dockerd) has been optional since then. Inertia and better tooling means running dockerd is still common for Kubernetes installs. As even if you have have switched your clusters to containerd, you can still optionally run the dockerd daemon.

So if you don't want to run dockerd on EKS, you could instead be asking for a switch/option of the containerd runtime (https://github.com/aws/containers-roadmap/issues/313).

https://kubernetes.io/blog/2017/11/containerd-container-runtime-options-kubernetes/ https://kubernetes.io/blog/2018/05/24/kubernetes-containerd-integration-goes-ga/

https://developers.redhat.com/blog/2019/01/29/podman-kubernetes-yaml/ https://developer.ibm.com/articles/templating-and-podman-openshift/

[inductor]

Podman must support CRI first. It’s not a problem with EKS at least for now.

One more thing to note, you can also ask them to support CRI-O which is another Kubernetes compatible runtime mainly developed by RedHat. In fact, it runs as the default container runtime for OpenShift4 so if you want something thinner than Docker, maybe that’s the option.

[haircommander]

One more thing to note, you can also ask them to support CRI-O which is another Kubernetes compatible runtime mainly developed by RedHat...

Hey all, CRI-O and podman maintainer here. I agree with @inductor, what it seems you're looking for @brianbordini is for EKS to support CRI-O. podman is meant as a docker cli replacement, whereas CRI-O is for the kubernetes use-case.

I would love to see CRI-O supported in EKS, and am happy to assist in this endeavour

[brianmori]

Hello, you are totally correct, I did not deep dive in the matter. Feel free to close the issue as it is a duplicated for https://github.com/aws/containers-roadmap/issues/313

[haircommander]

Feel free to close the issue as it is a duplicated for #313

I disagree with you here, I'd rename this to [EKS][request] CRI-O support. CRI-O and containerd CRI are two different projects

[jrhoward]

I would love to see support for CRI-O

[nagarajui7]

I would appreciate if EKS starts supporting different run-times like containerd, cri-o

[robrotheram]

As now Kubernetes have announced their deprecation of docker (https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.20.md#deprecation) I just wondering what EKS roadmap on which container runtime that will be used

[claywd]

Or what can be done to contribute to the effort?

[pseudoers]

Support for containerd is about to land now wonder where this is going. If this support can land soon, that'd be nice so my team can experiment sysbox with it.

[kpanic9]

If possible it would be good to know where this is at the EKS roadmap specially since now we have to select either containerd or CRI-O. Is there any chance that this will be implemented before 1.22?

[hf-cjharries]

Currently there's parity by using the Docker engine locally and in the cluster. Since EKS is forcing all consumers to use containerd, what is the AWS-supported local toolchain? I need to work on converting my org to use the new tools AWS support will be telling us to use when tickets are submitted. Were multiple runtimes supported by EKS, this could easily be left up to the consumer and I wouldn't assume that AWS is attempting to shoehorn all consumers into their flow.

I'm assuming AWS will be releasing a production-ready local CLI? Last I checked, ctr was not a docker replacement and crictl does not have a stable release.

If EKS supported the status quo of multple engines, I'd say these questions can be answered by "use community standards." Until EKS decides to support community standards, these issues have to be addressed by AWS.

[McPatate]

Hi everyone, we've started looking into building an AMI with CRI-O for EKS.

I've managed to deploy an EKS with the terraform module.

The code is present here : https://github.com/huggingface/amazon-eks-ami.

I was wondering if this could be upstreamed ? Regardless, feedback is more than welcome, there's probably some things that aren't suitable for production in there.

---

You will probably need to set a bigger disk size :

eks-worker-al2.json

{
  "variables" : {
    "launch_block_device_mappings_volume_size": "30", # this might be excessive
  },
  "builders": [
    {
      "type": "amazon-ebs",
      "ami_block_device_mappings": [
        {
          "volume_size": 30, # this too might be excessive
        }
      ]
    }
  ]
}

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 18.0"

  cluster_name    = "test-crio"
  cluster_version = "1.22"

  eks_managed_node_group_defaults = {
    bootstrap_extra_args = "--container-runtime crio"

    pre_bootstrap_user_data    = <<-EOT
        export CONTAINER_RUNTIME="crio"
      EOT
    enable_bootstrap_user_data = true
  }

  eks_managed_node_groups = {
    some_node_group = {
      ami_id         = "ami-xxxxx"
      name           = "node_group"
      min_size       = 1
      max_size       = 1
      desired_size   = 1
      instance_types = ["t3.large"]
    }
  }

[inductor]

@McPatate It's nice to see something that actually works. Not sure if EKS team would adopt something like this because they decided to go with containerd as the alternative runtime for Docker, so I assume that only if this demand is huge for particular reasons, it will be in the list.

One question here. Do you really need to build conmon inside of the worker? Having build dependencies on nodes are not ideal for cluster admins.

[McPatate]

@McPatate It's nice to see something that actually works. Not sure if EKS team would adopt something like this because they decided to go with containerd as the alternative runtime for Docker, so I assume that only if this demand is huge for particular reasons, it will be in the list.

I felt the same thing, better ask and see !

One question here. Do you really need to build conmon inside of the worker? Having build dependencies on nodes are not ideal for cluster admins.

For now we're only testing out CRI-O, so I tried to get it working before thinking about doing it cleanly. To answer your question, no, there's certainly other ways to do so. I'll give all that a look if we confirm that we're going to go with CRI-O.

[haircommander]

Please let me know if you have any questions or issues setting CRI-O up! I'm excited you're trying it out :slightly_smiling_face:

[jrm16020]

Are there any updates on this endeavor? I'm looking forward to using the NRI support built into cri-o recently and, soon, containerd.