sregistry rename exception

[tschoonj]

Describe the bug Trying to rename an image in the registry database using something like:

> sregistry rename library/name:latest library/name:some-tag

And getting:

Traceback (most recent call last):
  File "/dls_sw/apps/python/anaconda/4.6.14/64/envs/python3.7/bin/sregistry", line 10, in <module>
    sys.exit(main())
  File "/dls_sw/apps/python/anaconda/4.6.14/64/envs/python3.7/lib/python3.7/site-packages/sregistry/client/__init__.py", line 319, in main
    extra=extra)
  File "/dls_sw/apps/python/anaconda/4.6.14/64/envs/python3.7/lib/python3.7/site-packages/sregistry/client/rename.py", line 17, in main
    cli.announce(args.command, quiet=args.quiet)
TypeError: announce() got an unexpected keyword argument 'quiet'

To Reproduce Steps to reproduce the behavior: see description

Expected behavior

My image's name would have changed in the database, but not the actual image filename... not sure if this is what this command is supposed to do!

Version of Singularity and Singularity Registry Client

> sregistry version
0.2.21

[vsoch]

This is likely a result from when we re-organized the folders there to better reflect collections and versions.

I have never liked this function - it means that we lose the organization and naming schema of sregistry. What do you think about removing it? What use cases do you think exist for it?

[tschoonj]

My use case would be to use it for changing the name of an image that I have added to the database. The filename itself should not be changed obviously.

I am kind of confused about the difference between mv and rename, does the former actually move an image out of the database, while the latter keeps it there but under a different name?

Just wondering, because moving and renaming something is pretty much always the same thing.

[vsoch]

So you would pull something like vanessa/container and then rename it? Doesn't that lose reproducibility?

Move is to move it somewhere else on the host (outside of the database). Originally, I had a user that felt strongly about having both, so I provided them. I'm moderately okay with mv, less so with rename.

[tschoonj]

I am currently having a bash script that is always creating a new singularity image and then adds it to the registry using the name library/image, after first removing the old build in the registry if present.

However now and then I want to keep a particular build, and want to rename library/image to something like library/image:my-tag, as I do not want it to get deleted next time I recreate the image.

[vsoch]

Why don't you just pull without adding to your database?

sregistry pull --no-cache <blah> 

And then add it to what you want it to be?

sregistry add --name my/container path-to-file.sif

[vsoch]

okay, so you are saying that rename is something you'd like to keep for this purpose. I'll see if I can look into the issue, but overall we need more hardened tests for all these commands.

[tschoonj]

I am actually not pushing the images to my S3 bucket, and just keep them in my local registry/database. So pull would not work here?

[vsoch]

Oh I see, you are just adding them after build? Can you give me a complete example? Like:

sudo singularity build container.sif Singularity
sregistry add --name library/image container.sif

And then using rename?

[tschoonj]

Here's an extract from my bash script:

# images to get a new build -> they will get built in this order!
declare -a images=("SavuDeps" "SavuCore" "SavuAstra")

for image in "${images[@]}" ; do
        echo "##############################"
        echo "#"
        echo "#   Building $image"
        echo "#"
        echo "##############################"
        LOCAL_IMAGE=${image}.simg
        REGISTRY_IMAGE=${image:4}
        REGISTRY_IMAGE=savu/${REGISTRY_IMAGE,,}

        #
        # remove the image, if it has been built before, including from local registry
        #
        rm -f ${LOCAL_IMAGE}
        sregistry get $REGISTRY_IMAGE && sregistry rm $REGISTRY_IMAGE

        #
        #  build the image
        #
        sudo ${SUDO_OPTIONS} singularity build ${LOCAL_IMAGE} Singularity.${image}

        #
        #  add to local registry
        #
        sregistry add --copy --name ${REGISTRY_IMAGE} ${LOCAL_IMAGE}
done

I will execute this bash script many times, but only now and then will there be a build that I want to keep, so I thought of doing that with sregistry rename whenever necessary.

[vsoch]

Ok, I have something for you to test! Will link here in a second.

[vsoch]

https://github.com/singularityhub/sregistry-cli/pull/226

And holy cow, I really need to get on testing soon - it's just totally unacceptable for the package to miss these errors! Just need to make some time!

[tschoonj]

It works!

Not sure if it matters but the image file itself gets renamed as well (and loses the checksum in the process):

> sregistry images
Containers:   [date]   [client] [uri]
1  June 05, 2019       [s3] centos7/conda-build:latest@0a182a8156f8487bbbe27371dbe4d621
2  June 10, 2019       [s3] savu/deps:latest@525fba821308f393a1a2aaf952324b9a
3  June 11, 2019       [s3] savu/core:latest@fcf15eaa1388364fdc085da8a74069ef
4  June 11, 2019       [s3] savu/astra:latest@458a0bee0a1b0b9780b4b69137ecdec7
> sregistry get savu/astra
/dls/science/users/awf63395/singularity/shub/savu/astra/latest@458a0bee0a1b0b9780b4b69137ecdec7.sif
> sregistry rename savu/astra savu/astra:master
[client|s3] [database|sqlite:////dls/science/users/awf63395/singularity/sregistry.db]
[bucket:s3://s3.Bucket(name='singularity-savu')]
[rename] latest@458a0bee0a1b0b9780b4b69137ecdec7.sif => /dls/science/users/awf63395/singularity/shub/savu/astra/master.sif
> sregistry images
Containers:   [date]   [client] [uri]
1  June 05, 2019       [s3] centos7/conda-build:latest@0a182a8156f8487bbbe27371dbe4d621
2  June 10, 2019       [s3] savu/deps:latest@525fba821308f393a1a2aaf952324b9a
3  June 11, 2019       [s3] savu/core:latest@fcf15eaa1388364fdc085da8a74069ef
4  June 11, 2019       [s3] savu/astra:master@458a0bee0a1b0b9780b4b69137ecdec7
> sregistry get savu/astra
ERROR image savu/astra not found in database
> sregistry get savu/astra:master
/dls/science/users/awf63395/singularity/shub/savu/astra/master.sif

[vsoch]

We should keep the checksum. Let me take a look.

[vsoch]

Fixed with #226

[tschoonj]

many, many thanks!