search

coiled / feedback

A place to provide Coiled feedback
14 stars 3 forks source link

Non-private software environment is private on dockerhub #109

Closed rubenvdg closed 3 years ago

rubenvdg commented 3 years ago

If I set Docker Hub under Container Registry Options and make a public conda environment with coiled, it becomes a private repo on my Docker Hub. Is this expected behavior?

Problem is that I'm using a private free-tier account on Docker Hub, which only allows for 1 private repo. If you exceed this, the repos become unavailable. As a consequence, I catch the following when making a new cluster:

ValueError: Unable to create cluster: Getting image source signatures
error copying layers and metadata from "containers-storage:[overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev]localhost/6be24e83-8b3e-40ca-af71-15f67e53aee9:latest" to "docker://rvdgeer/ruben-van-de-geer9-dask-env-test:6be24e83-8b3e-40ca-af71-15f67e53aee9": Error trying to reuse blob sha256:f5600c6330da7bb112776ba067a32a9c20842d6ecc8ee3289f1a713b644092f8 at destination: Error checking whether a blob sha256:f5600c6330da7bb112776ba067a32a9c20842d6ecc8ee3289f1a713b644092f8 exists in docker.io/rvdgeer/ruben-van-de-geer9-dask-env-test: errors:
denied: **requested access to the resource is denied**
error parsing HTTP 401 response body: unexpected end of JSON input: ""
FabioRosado commented 3 years ago

Thank you for the questions, we have recently changed a few things about how the software environments are created and perhaps this is the reason why the uploaded image is being set as private. Let me test things out on my end.

FabioRosado commented 3 years ago

Ruben, could you give me more details on how you were trying to build your conda environment? I've tried to create a software environment with:

coiled.create_software_environment(name="soft-env", conda=["dask"])

This creates a public repository on docker hub. Even if passing private=True it creates a public one, so I am unable to reproduce this

rubenvdg commented 3 years ago

Strange. I did:

import coiled

coiled.create_software_environment(
    name="new-docker-test",
    container="rvdgeer/dask-base:latest",
    private=True
)
cluster = coiled.Cluster(
    name="new-cluster",
    software="new-docker-test",
)

and I end up with:

image

Not a big deal though, but unexpected nonetheless.

rubenvdg commented 3 years ago

@FabioRosado As you mentioned on Slack, this is due to my default privacy settings on Docker Hub being set to "private". Thanks for the feedback!