bottlerocket-os / bottlerocket

An operating system designed for hosting containers
https://bottlerocket.dev
Other
8.81k stars 520 forks source link

Not able to configure runtimeRequestTimeout in bottlerocket API #4259

Closed felipewnp closed 3 weeks ago

felipewnp commented 1 month ago

Image I'm using:

1.25.0

What I expected to happen:

Be able to configure the parameter runtimeRequestTimeout on the Kubernetes config file.

What actually happened:

According to the latest API documentation, a timeout setting is not available.

My question:

If there's a parameter that needs changing inside kubelet-config.json, how should it be done?

For reference, I'm running Karpenter on EKS with AL2 and trying to migrate to BottleRocket.

felipewnp commented 3 weeks ago

I've tried chatGPT, but I'm pretty sure this is wrong: .. .

Configuring runtimeRequestTimeout in Kubernetes on AWS Bottlerocket requires modifying the kubelet settings. Bottlerocket is a minimal, container-optimized OS with a specific configuration model, so these settings are managed through user data or configuration settings specific to Bottlerocket.

Here’s how to configure runtimeRequestTimeout:

  1. Understand the Parameter:

    • runtimeRequestTimeout specifies the timeout for calls made by the kubelet to the container runtime.
    • In Bottlerocket, kubelet parameters can be set in the kubernetes settings section.
  2. Edit Bottlerocket’s User Data:

    • When launching your Bottlerocket instance in AWS, you can pass configuration in user data to set custom kubelet parameters.
    • This can be done using the settings.kubernetes configuration, where kubelet flags are adjusted.
  3. Set runtimeRequestTimeout:

    • In your user data, specify the parameter as part of the settings.kubernetes.kubelet-runtime-request-timeout field.
    • For example, in your user data:

      [settings.kubernetes]
      kubelet-runtime-request-timeout = "10s" # Adjust timeout as needed
  4. Launch or Update Bottlerocket Configuration:

    • When creating a new Bottlerocket instance, apply the user data.
    • Alternatively, you can use the AWS Systems Manager (SSM) to apply this configuration on an existing Bottlerocket node using the apiclient tool.
  5. Apply Configuration Using apiclient:

    • For existing Bottlerocket instances, connect via SSM and use apiclient to set the parameter:

      apiclient set settings.kubernetes.kubelet-runtime-request-timeout=10s
  6. Restart Kubelet (if needed):

    • After applying the settings, you may need to restart the kubelet for the changes to take effect.

Following these steps should allow you to adjust the runtimeRequestTimeout for the kubelet in AWS Bottlerocket Kubernetes nodes.

ytsssun commented 3 weeks ago

Hi @felipewnp, thanks for opening up this issue. May I know your use case and why you need to change the runtimeRequestTimeout value? That being said, is this a blocking issue for your to migrate from AL2 to Bottlerocket?

As of now we hard coded the runtimeRequestTimeout to be 15min. To modify it, we would indeed need to expose an API for it to be adjustable, in which case this issue would be treated as a feature request. If you agree, I will adjust the issue title to reflect that.

It is worth noting that the API mentioned in the ChatGPT response does not exist today.

felipewnp commented 3 weeks ago

Hello @ytsssun Thanks for the reply!

My use case is that in AL2, I've had to change this parameter (runtimeRequestTimeout) because it was causing several problems with contextDeadlineExceeded.

On AL2, the default is only 1 or 2 minutes.

Since the default in Bottlerocket is 15 minutes, which is actually greater than the 10 minutes I've been using, I think it won't be a problem, so I'm closing the issue for now.

Thanks again for the explanation about the default value. I didn't find it before opening the issue.