search

aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.66k stars 3.92k forks source link

batch: support multi-node job on Amazon EKS #30906

Open mazyu36 opened 3 months ago

mazyu36 commented 3 months ago

Describe the feature

AWS batch now supports multi-node parallel jobs on Amazon EKS from Jul 11, 2024. But L2 Construct does not support.

MultiNodeJobDefinition is only for multi-node jobs on Amazon ECS.

Use Case

To use multi-node jobs on Amazon EKS.

Proposed Solution

Add new class like MultiNodeEksJobDefinition.

Other Information

When addressing this issue, is submitting an RFC mandatory?

Acknowledgements

CDK version used

2.147.0

Environment details (OS name and version, etc.)

all

pahud commented 3 months ago

Thank you mazyu. We are looking forward to your PRs as always :)

mazyu36 commented 3 months ago

I realized that this cannot be implemented at present. EKS configuration is not yet supported in CloudFormation's NodeProperties.

Node properties can't be specified for Amazon EKS based job definitions.

We need to wait for CloudFormation to support this feature.

delagoya commented 2 months ago

This is supported by CloudFormation.

EKS properties are under NodeRangeProperties

delagoya commented 2 months ago

A comment on the above - it made sense to fold the Batch API construct NodeProperties.NodeRangeProperties[].container to NodeProperties.containers[] in the CDK L2 construct when you could only define a single container per NodeRange, but now that you can define multiple containers per NodeRange, the L2 construct should reflect the API more closely.

If a new MultiNodeEksJobDefinition class is created as part of the pull request, it makes more sense for the class to follow the API model and define a readonly nodeRanges: EksNodeRange[] property.