Azure / az-hop

The Azure HPC On-Demand Platform provides an HPC Cluster Ready solution
https://azure.github.io/az-hop/
MIT License
64 stars 54 forks source link

Resource Provider requirements #1092

Open matt-chan opened 2 years ago

matt-chan commented 2 years ago

Hi Xavier,

We have run into a situation a few times now where we discover that certain resource providers are not enabled on a given subscription (e.g. compute, storage, network, etc). Usually we find this out after already specifying our requirements to the customer so we end up having to make a follow-up request to them and it can be a little embarrassing.

Do you know if there is a full list of resource providers which are needed for az-hop? I can submit a PR for the docs after that. Off the top of my head I can think of:

Thanks! Matt

xpillons commented 2 years ago

These are the azurerm resources used, I don't know how they map to a provider but it should be straighforward to find the providers from the documentation https://registry.terraform.io/providers/hashicorp/azurerm/3.22.0

matt-chan commented 2 years ago

Hi Xavier,

I got the list I think. build.sh -a plan outputs the API endpoints in stdout. All of the resources start with "Microsoft."

The following bash line will print all the API calls: ./build.sh -a plan | grep -o Microsoft.[^/]\* | sort -u

For our deployment, we got:

Would it be possible to check on one of your more complex deployments as well please? Lustre is not enabled on this deployment for example.

Also, I think there might still be a few calls from the image build step with Packer which are not included here. This might make it require Microsoft.VirtualMachineImages.

I'll PR the docs after we get an exhaustive list

Thanks! Matt

matt-chan commented 2 years ago

Oh also, one of our internal clients suggested we give them an azcli script to run as Contributor on the subscription to enable all the providers in a single shot. I can work on a PR for that as well.

xpillons commented 2 years ago

For the whole list, you should get it by enabling lustre and do a plan I guess. The image built is creating Image Definition and Version in addition to build a VM, so I guess this should be all covered by the one used by Terraform. Yes please go ahead with the PR.

matt-chan commented 2 years ago

Hi Xavier,

It looks like lustre won't pull in any extra resource providers? I'm surprised.

By the way, I see on the Cyclecloud documentation that they also use Microsoft.commerce (billing) and Microsoft.resources (not sure for what). If we enable these on our subscription, will Cyclecloud behave differently? I notice for example we don't have data on cost etc.

xpillons commented 1 year ago

Lustre is a marketplace image so no resource provider needed today. This will change with the managed service. It's true that data on cost is not displayed, need to dig more in to this.