aws / aws-parallelcluster

AWS ParallelCluster is an AWS supported Open Source cluster management tool to deploy and manage HPC clusters in the AWS cloud.
https://github.com/aws/aws-parallelcluster
Apache License 2.0
816 stars 309 forks source link

Feature Request: cdk construct for parallel cluster #5541

Open snemir2 opened 11 months ago

snemir2 commented 11 months ago

Now that parallel cluster supports CloudFormation custom resource, was wondering if you can officially support cdk deployment of parallel clusters. As a quick poc, deploying the custom resource and then paralell cluster works in principal (python cdk below)

class CustomResourceProvider(Stack):
    ServiceToken=None
    def __init__(self, scope: Construct, construct_id: str, env , params, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        #cloudfomation for custom resource provider 
        PC_CF_CRP="https://" + env.region +"-aws-parallelcluster.s3." + env.region + ".amazonaws.com/parallelcluster/" + params['PC_VERSION'] + "/templates/custom_resource/cluster.yaml"

        #deploy cfn_stack for custom resource provider
        cfn_stack=cdk.CfnStack(self, "ParallelClusterCustomResourceProvider",
                               template_url=PC_CF_CRP)

        output=cdk.CfnOutput( self, "ServiceToken", 
                   value=cfn_stack.get_att("Outputs.ServiceToken").to_string())

        self.ServiceToken=cfn_stack.get_att("Outputs.ServiceToken").to_string()

class ParallelCluster(Stack):

    def __init__(self, scope: Construct, construct_id: str, env , params, ServiceToken, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        # now lets provision a cluster thrugh the deployed custom resource provider

        ###this needs to be parameterized by environment
        CLUSTER_CONFIG = yaml.safe_load(open("deployment_v2/config-file.yaml", 'r'))

        print ("ServiceToken: " + ServiceToken)     
        CustomResource(self, "PclusterClusterSERGEY",
            resource_type="Custom::PclusterCluster",
            service_token=ServiceToken, 
            properties={
                'ClusterName': 'SERGEY',
                'ClusterConfiguration': CLUSTER_CONFIG

            }

        )
chenwany commented 11 months ago

Hello! Thank you for the feature request! Could you elaborate a bit more to help me to understand the request? Do you mean adding official support for ParallelCluster deployment with cdk by introducing ParallelCluster construct in aws-construct-library: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-construct-library.html

Thanks

snemir2 commented 10 months ago

The goal is to easily provision clusters with CDK with the rest of the infra.

chenwany commented 10 months ago

Thanks, I tracked this into our backlog

rvencu commented 9 months ago

Hi, I am also keen to get this feature so I can convert my current convoluted solution to a more elegant one

tcutts commented 5 months ago

I'd like to see ParallelCluster formally supported by CDK too - wondering whether the feature request really needs to be with CDK rather than ParallelCluster though?

pradoz commented 3 weeks ago

I would find this enhancement useful as well. Ideally, the configuration file would be abstracted into parameters to allow cluster management directly from CDK

@chenwany Where can one view where this landed in the backlog?