search

singularityhub / singularity-hpc

Local filesystem registry for containers (intended for HPC) using Lmod or Environment Modules. Works for users and admins.
https://singularity-hpc.readthedocs.io
Mozilla Public License 2.0
111 stars 26 forks source link

Is the remote registry using a branch/tag? #625

Closed surak closed 1 year ago

surak commented 1 year ago

on my settings.yml, I have:

registry: [https://github.com/singularityhub/shpc-registry]
sync_registry: https://github.com/singularityhub/shpc-registry

Which I understand should be enough for it to take the most recent versions of everything.

But if I try, with 0.1.17, to do a

shpc install nvcr.io/nvidia/tensorflow
Module nvcr.io/nvidia/tensorflow:22.10-tf2-py3 was created.

However, if I look at the nvcr.io/nividia/tensorflow file here: https://github.com/singularityhub/shpc-registry/blob/main/nvcr.io/nvidia/tensorflow/container.yaml , I see that it has a more recent version:

https://github.com/singularityhub/shpc-registry/blob/fb42a39e7ab0c2451743f6d26f98cc46fe4079c9/nvcr.io/nvidia/tensorflow/container.yaml#L9

22.11-tf2-py3: sha256:9546a59d4eb6a4517f843bdf61a7e7279621ca4c2c74fb0da5f59c99dff3d259

I see this line on the "main" branch of shpc-registry, so it might be something per-version? Is something I am missing?

Thanks!

vsoch commented 1 year ago

Thanks for opening this issue @surak - this is indeed a bug with the automation! We update the recipes in main with several ways, but the actual update of the docs, in that it runs on push to main, could not actually be triggered by another bot. So the solution is to add a nightly run (which would happen right after the manual run) and I've just triggered the update now for you. Try again - I just updated the docs and did a run so it should be the correct version now.

The update was infrequent because it's fairly time intensive with over 8K containers. I've opened an issue to work on that https://github.com/singularityhub/shpc-registry/issues/81.

surak commented 1 year ago

Thank you, it's no problem. My question is to understand from where shpc fetches this info. My understanding was that it fetched it from github somehow, but now I'm not so sure. I will check the source :-)

vsoch commented 1 year ago

It's from GitHub pages, which is derived from main.

vsoch commented 1 year ago

@surak do you want to discuss anything further here?

surak commented 1 year ago

No, thanks! And thanks for the talk today, I’m in London!

vsoch commented 1 year ago

oh nice!!! That room was SO cool I got ALL the Harry Potter vibes! :mage: