microsoft / CromwellOnAzure

Microsoft Genomics implementation of the Broad Institute's Cromwell workflow engine on Azure
MIT License
134 stars 55 forks source link

Azure Resource naming convention improvements #518

Open jlester-msft opened 2 years ago

jlester-msft commented 2 years ago

Problem: Cromwell on Azure deployer is creating VM and Storage Account resources using a naming scheme of "prefix + random digits". While this helps to create unique entities, it becomes a maintenance hassle once the resources are created and being maintained over the lifetime of the Cromwell server. Try typing in your random storage account a few dozen times without carefully looking at it or copy/pasting it.

For example, the deployer takes a prefix and then adds a different random UUID suffix to that name. This means that the UUID used in the VM is different than the UUID used in the Storage Account. Which makes it basically impossible to ever memorize those resource names, so either you have to go to the Azure Portal to copy/paste the names or you have to create aliases you can use (i.e., environmental variables).

Solution: It would be better if the deployer created resources using either just the user provided prefix. Or that prefix plus something sensible or a dictionary style generator. For example, if the prefix was set to 'genomics-pipeline' you might generate something like: VM: genomics-pipeline-vm or genomics-pipeline-vm1 if that name wasn't available SA: genomicspipeline or genomicspipelinesa or genomicspipelinesabrowncat

Similarly for pool resources, the name should be more comprehensible. Pool names should be 'prefix + full vm family name + perhaps managed identity name, etc. + some small subset of unique digits'. So rather than: e575819cc52-D1_v2-UNZATQPZ44VEVK6QWJIDVROOMMSC26ZV-5CGYD5BFMSNKG You would ideally want something like: wdltest-standard_d1_v2-wdltest.azurecr.io-5CGYD5BFMSNKG or wdltest-standard_d1_v2-ConfigA-5CGYD5BFMSNKG wdltest-standard_d1_v2-ConfigB-QWJIDVROOMMS

BMurri commented 1 year ago

Consider: https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ready/azure-best-practices/resource-abbreviations

ngambani commented 8 months ago

@BMurri is this an active issue? or already addressed?

BMurri commented 8 months ago

This has not been addressed and it is a pain point.