singularityhub / sregistry

server for storage and management of singularity images
https://singularityhub.github.io/sregistry
Mozilla Public License 2.0
103 stars 42 forks source link

Failed upload creates "shadow" container image #388

Open hashkeks opened 2 years ago

hashkeks commented 2 years ago

Describe the bug When there is only slightly more space left on the registry host than a image upload would need, the upload fails with HTTP error 500 (internal server error). The problem is, that even though the upload fails, a "logical" / shadow image remains. This shadow image is then part of a collection and seems to reference an actual image file that is not present. Therefore:

To Reproduce For example: we have 212MB of storage left and upload a 161MB image that hasn't been previously uploaded. An image object is created in sregistry to represent the image file in collections, etc. The upload finishes with 100% done, but nonetheless an error message is appearing with HTTP 500. All the space is used up on the MinIO storage (guess due to MinIO data safety mechanisms needing more space than just the image size?). The image only appears in the collection overview, but not inside the collection. If we start the upload again, it fails instantly but still a shadow image is created and remains.

err1 err2

Expected behavior

1) The upload starts just as usual 2) While the upload is going on, the container image is created in sregistry and added to the collection. Also the "Builds" counter is increased by one. 3) Mid-way, when the storage is full, the upload is aborted. An error message follows and the shadow image is deleted, "Builds" counter decreased by one. Everything is like before the upload.

vsoch commented 2 years ago

I’m aware of this bug- the reason it happens is because we don’t know the name, tag etc until the last step and then we rename and show in the collection. It should not, however, interfere with deleting a collection, although it will show it to exist.

If you have a suggested fix that resolves the issue you are seeing I’m happy to review a PR!