search

aws / aws-parallelcluster

AWS ParallelCluster is an AWS supported Open Source cluster management tool to deploy and manage HPC clusters in the AWS cloud.
https://github.com/aws/aws-parallelcluster
Apache License 2.0
828 stars 312 forks source link

ParallelCluster awsbatch mode multiple cluster queues and NICE DCV #3062

Open andrei-xdlab opened 3 years ago

andrei-xdlab commented 3 years ago

Required Info:

  1. Do you support NICE DCV on awsbatch head node? I am getting Unable to connect messages [dcv custom-dcv] enable = master port = 8443 access_from = 0.0.0.0/0

  2. Do you have the ability to submit batch jobs from awsbatch head node to a queue created outside of the awsbatch 3 create a spot awsbatch queue as part of parallel cluster build c) create multiple spot and ondemand awsbatch queues as part of parallel cluster build d) ability to submit jobs from awsbatch head node to a queue created in AWS Batch console?

demartinofra commented 3 years ago
  1. Do you support NICE DCV on awsbatch head node?

yes we do support this. Can you please share the full cluster config? Are you adding dcv_settings=custom-dcv in the cluster config section?

  1. Do you have the ability to submit batch jobs from awsbatch head node to a queue created outside of the awsbatch?

This is currently not supported and I can mark this as a feature request

enrico-usai commented 3 years ago

I think the possibility to submit batch jobs from ParallelCluster head node to an AWS Batch Queue created outside of ParallelCluster is out of scope for the product.

Instead I added the enhancement label to consider to add the possibility to create multiple queues for AWS Batch, as part of ParallelCluster cluster creation.

Thanks!

andrei-xdlab commented 3 years ago

thanks. ability to provision multiple batch queues for different type of workloads is very useful