Azure / az-hop

The Azure HPC On-Demand Platform provides an HPC Cluster Ready solution
https://azure.github.io/az-hop/
MIT License
63 stars 54 forks source link

config.yml: documentation of `images:` vs `image:` field in `queues` #1319

Open ltalirz opened 1 year ago

ltalirz commented 1 year ago

These are just some comments on the images: list in the configuration template

https://github.com/Azure/az-hop/blob/3b943a965e72d8757d1b1ec36bdc1e15e4074ebb/config.tpl.yml#L258-L339

  1. It would be great to document what the metadata in this list is actually used for. E.g. if an admin just builds the images via packer, are the publisher, offer, ... metadata fields actually used in any way (?)
  2. While there are prebuilt offers for some of the listed images on the marketplace, the publisher and offer names do not necessarily match the ones listed here. This can be confusing to a new user

Example 1

in the list

   - name: azhop-centos79-desktop3d 
     publisher: azhop 
     offer: CentOS 
     sku: 7.9-gen2-desktop3d 
     hyper_v: V2 
     os_type: Linux 
     version: 7.9 

on the marketplace

  - publisher: azhpc
    offer: azhop-desktop
    sku: centos-7_9
    version: latest

Example 2

in the list

  - name: azhop-almalinux85-v2-rdma-gpgpu
    publisher: azhop
    offer: almalinux
    sku: 8_5-hpc-gen2
    hyper_v: V2
    os_type: Linux
    version: 8.5

on the marketplace:

  - publisher: almalinux
    offer: almalinux-hpc
    sku: 8_5-hpc-gen2
    version: 8.5.20211210

cc @matt-chan

ltalirz commented 1 year ago

The packer JSON files are yet a third place with different information https://github.com/Azure/az-hop/blob/e706d1b78d08891b2366cf97292850c6b261e9cc/packer/azhop-centos79-desktop3d.json#L6-L9

xpillons commented 1 year ago

I will try to summarize these metadata. In the config file, the images definition the name map to a packer file name. The publisher, offer, sku and versions are the metadata used to create the image definition in the Shared Image Gallery. These are totally independent from any marketplace meanings. In the .json files used by packer, publisher, offer, sku and version are the one needed to retrieve the image from the azure marketplace used to build the VM on which scripts will be run before the image is captured. Once captured it will be pushed in the SIG using the metadata used in the images section of the config file.

ltalirz commented 1 year ago

In the config file, the images definition the name map to a packer file name. The publisher, offer, sku and versions are the metadata used to create the image definition in the Shared Image Gallery. These are totally independent from any marketplace meanings.

Could this not be placed in the packer JSON file then to keep it together at the same place?

I believe this information is not referenced directly from the config.yml - at the queue level you either specify a marketplace URN or a reference to the custom image

xpillons commented 1 year ago

can't be in packer directly as these values are only for packer, but it could make sense to have the images defined only in the packer directory.