search

singularityhub / sregistry

server for storage and management of singularity images
https://singularityhub.github.io/sregistry
Mozilla Public License 2.0
103 stars 42 forks source link

HTTP 405 Method Not Found during singularity push #292

Closed lmcdasm closed 4 years ago

lmcdasm commented 4 years ago

Describe the bug When performing a singularity push to the registry, any image (small or large) is returning a 405.

In the uswgi logs (im running sregistry containerized) we see the "Not Found: /v1/collections/" being thrown

here is the full log from the post from server ( i removed the [pid]) stuff

Attempt 1 (using the "name" of the collection: singularity push -U ./hello.simg library://daniel_smith/collections/development/test-crg121:latest)

GET GetEntitiesView TOKEN IS VALID Not Found: /v1/collections/ Method Not Allowed: /v1/collections GET NamedEntityView <QueryDict: {}> daniel_smith

ATTEMPT 2: (just "collections" in URL path) - singularity push -U ./hello.simg library://daniel_smith/collections/test-crg121:latest

GET GetNamedCollectionView Method Not Allowed: /v1/collections GET NamedEntityView <QueryDict: {}> daniel_smith

ATTEMPT 3: using the Collection NUmber "3" in path

GET GetEntitiesView TOKEN IS VALID Not Found: /v1/collections/ Method Not Allowed: /v1/collections

To Reproduce COMMANDS BEING DONE: singularity remove login (API KEY)works: copy the token to $HOME/.sregistry do: 001 ~]$ singularity pull --name hello.simg shub://vsoch/hello-world to get an image do: 001 ~]$ singularity push -U ./hello.simg library://daniel_smith/collections/test-crg121:latest

Also tried: do: 001 ~]$ singularity push -U ./hello.simg library://daniel_smith/collections/3/test-crg121:latest

Where /3/ is the number of a "collection" that i had created. cn001 ~]$ singularity push -U ./hello.simg library://daniel_smith/collections/3/test-crg121:latest WARNING: Skipping container verifying FATAL: Unable to push image to library: request did not succeed: http status code: 405

Expected behavior Image is pushed to my users collectinos If applicable, add versions and screenshots to help explain your problem.

seems like im stuck on the v1 path.

lmcdasm commented 4 years ago

Here is the Collections layout image

vsoch commented 4 years ago

Why are you adding "collections" to the URL? The push should be to the collection name. You can follow the page exactly here: https://singularityhub.github.io/sregistry/docs/client

singularity push -U busybox_latest.sif library://vsoch/dinosaur-collection/another:latest

Based on your collection, you would push to:

singularity push -U busybox_latest.sif library://daniel_smith/test/container:latest

Where are you reading docs that say to put "collections" there?

lmcdasm commented 4 years ago

Hey there...

Im getting it from here (maybe it misread),

image

lmcdasm commented 4 years ago

but i see what you mean - sorry. brain dead :)

vsoch commented 4 years ago

Singularity != Singularity Registry Server, sorry for the misunderstanding! I have no knowledge about how Sylabs manages or deploys their registry, I've developed the registry here to work with the same API endpoints. That said, I think the usage above shows the same thing - they have a collection named "collection" (akin to your "test") above. Does that make sense?

vsoch commented 4 years ago

@lmcdasm you totally don't need to apologize - the fact that any of us are functioning at all is astounding. Take care of yourself, and hang in there... containers should be the last of your worries these days.

lmcdasm commented 4 years ago

you as well.. i was copy pasting the issue froma user and didnt think about it.

i do have some things to come back to you on this week (OCI and larger file sizes)..

thanks again.. we can close this

vsoch commented 4 years ago

I actually started working on OCI last week! It's going to be a bit of work because it requires python handles for OCI, and also (what I'm hoping will be) a Django module for deploying an OCI registry easily. It's a lot of work that isn't considered part of my main role, so I can only promise that I'm making time for it, and keeping it up, development wise.

lmcdasm commented 4 years ago

so.. a bit more help here.

i have a collection called development (again this might be out of your balliwick here).

Unable to push image to library: request did not succeed: http status code: 405 [stefan.lange@cn001 ~]$ singularity push -U ./hello.simg library://daniel_smith/development/crg121:latest

lmcdasm commented 4 years ago

as for OCI.. no worries at all. .we have a dockerV2 one for our "first outing" of release, however, it puts me in a bit of a weird spot since i have a HPC setup that produces everything in singularity going foward and then i have a group (eng) that consume things - but since its K8s based, they want docker - i havent see too much movement on the K8s Compliant Singularity front (but if it was OCI compliant then it would be a path forward for sure).

vsoch commented 4 years ago

I think Singularity is oci compliant, check out 

 singularity oci --help                                                                                                                     
Manage OCI containers

Usage:
  singularity oci

Description:
  Allow you to manage containers from OCI bundle directories.

  NOTE: all oci commands requires to run as root

Options:
  -h, --help   help for oci

Available Commands:
  attach      Attach console to a running container process (root user only)
  create      Create a container from a bundle directory (root user only)
  delete      Delete container (root user only)
  exec        Execute a command within container (root user only)
  kill        Kill a container (root user only)
  mount       Mount create an OCI bundle from SIF image (root user only)
  pause       Suspends all processes inside the container (root user only)
  resume      Resumes all processes previously paused inside the container (root user only)
  run         Create/start/attach/delete a container from a bundle directory (root user only)
  start       Start container process (root user only)
  state       Query state of a container (root user only)
  umount      Umount delete bundle (root user only)
  update      Update container cgroups resources (root user only)

Examples:
  All group commands have their own help output:

  $ singularity oci create -b ~/bundle mycontainer
  $ singularity oci start mycontainer

For additional help or support, please visit https://www.sylabs.io/docs/
lmcdasm commented 4 years ago

indeed. singularity is by virtue of the fact they are using dockerV2 image format (so im pretty sure the singularity part is) - i think (for the K8s hooks) there is a need to simple switch and replace docker "run" with singularity execute - i do think that all the montioring that K8s has on the underside that binds into the docker daemon would need to be retools to work with singularity to allow a "distributed" sense (much like querying docker.sock on the system)

vsoch commented 4 years ago

Have you seen https://github.com/sylabs/singularity-cri?

lmcdasm commented 4 years ago

hey there.

indeed, i had stumbled across that but hadnt had a chance to dig into it - on my todo list.

as for my original issue, im stumped now - i had this working to push to this sregistry (yours ) with singularity working before and now its not :(

all fun and games..

Cheers, D

On Mon, 30 Mar 2020 at 13:47, Vanessasaurus notifications@github.com wrote:

Have you seen https://github.com/sylabs/singularity-cri?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/singularityhub/sregistry/issues/292#issuecomment-606145125, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACHMXRAFVHPIFZMYCV5XLNLRKDLMDANCNFSM4LWW77OQ .

asherRTK commented 1 year ago

Hi @vsoch , I have almost the same problem when I use the latest version, and I have no idea how to fix it. Here is my git status (and I just change these three files):

$ git status On branch master Your branch is up to date with 'origin/master'.

Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git restore ..." to discard changes in working directory) modified: .minio-env modified: docker-compose.yml modified: shub/settings.py

1. .minio-env diff --git a/.minio-env b/.minio-env index 95ab4d1..a2fee8a 100644 --- a/.minio-env +++ b/.minio-env @@ -1,2 +1,5 @@ MINIO_ROOT_USER=minioadmin MINIO_ROOT_PASSWORD=minioadmin + +MINIO_BROWSER=on >> only add this line

2. docker-compose.yml diff --git a/docker-compose.yml b/docker-compose.yml index 04197fd..c7430c0 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,8 +1,12 @@ +version: '2' + services: db: image: postgres environment:

3. shub/settings.py diff --git a/shub/settings.py b/shub/settings.py index 6cc7b2a..bec27c7 100644 --- a/shub/settings.py +++ b/shub/settings.py @@ -12,6 +12,8 @@ from importlib import import_module import yaml import os import sys import ldap >> try to use my LDAP from django_auth_ldap.config import LDAPSearch, PosixGroupType BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(file))) @@ -26,12 +28,12 @@ SETTINGS_FILE = os.environ.get("SREGISTRY_SETTINGS_FILE") or os.path.join( BOOLEAN_DEFAULTS = { "DEBUG": True, "API_REQUIRE_AUTH": False, "ENABLE_GOOGLE_AUTH": False, "ENABLE_TWITTER_AUTH": False, "ENABLE_GITHUB_AUTH": False, "ENABLE_GITLAB_AUTH": False, "ENABLE_BITBUCKET_AUTH": False, "ENABLE_GITHUB_ENTERPRISE_AUTH": False, @@ -75,8 +77,8 @@ STRING_DEFAULTS = { "DOMAIN_NAME": "http://my_ip", "DOMAIN_NAME_HTTP": "http://my_ip", "HELP_CONTACT_EMAIL": "vsoch@users.noreply.github.com", "HELP_INSTITUTION_SITE": "https://srcc.stanford.edu", @@ -95,7 +97,7 @@ STRING_DEFAULTS = { "MINIO_ROOT_USER": None, "MINIO_ROOT_PASSWORD": None, "MINIO_SERVER": "minio:9000", # Internal to sregistry "MINIO_EXTERNAL_SERVER": "my_ip:9000", # minio server for Singularity to interact with "MINIO_BUCKET": "sregistry", "MINIO_REGION": "us-east-1", @@ -149,13 +151,13 @@ STRING_DEFAULTS = { "AUTH_LDAP_SERVER_URI": "ldap://my_ldap_ip", # "ldaps://ldap.example.com "AUTH_LDAP_BIND_DN": "", "AUTH_LDAP_BIND_PASSWORD": "", "AUTH_LDAP_USER_SEARCH": LDAPSearch("O=XXXXXXXX",ldap.SCOPE_SUBTREE, "(uid=%(user)s)"), # "ou=users,dc=example,dc=com" "AUTH_LDAP_GROUP_SEARCH": LDAPSearch("O=XXXXXXXX",ldap.SCOPE_SUBTREE, "(objectClass=posixGroup)"), # "ou=groups,dc=example,dc=com"

Anyone in this group can get a token to manage images, not superuser

 "AUTH_LDAP_STAFF_GROUP_FLAGS": None,  # "cn=staff,ou=django,ou=groups,dc=example,dc=com",
 # Anyone in this group is a superuser for the app

@@ -178,7 +180,7 @@ INTEGER_DEFAULTS = {

Uploads > 2.5GB will not use memory, but the filesystem

 "DATA_UPLOAD_MAX_MEMORY_SIZE": None,
 # Limit users to N collections (None is unlimited)
"USER_COLLECTION_LIMIT": 30,
 # The number of collections to show on the /<domain>/collections page
 "COLLECTIONS_VIEW_PAGE_COUNT": 250,
 # The maximum number of downloads allowed per container/collection, per week

@@ -439,7 +441,7 @@ MINIO_ROOT_PASSWORD = os.environ.get("MINIO_ROOT_PASSWORD") or cfg.MINIO_ROOT_PA

PLUGINS_ENABLED = [ 'ldap_auth' ] AUTH_LDAP_USER_ATTR_MAP = { } AUTH_LDAP_USER_FLAGS_BY_GROUP = { }

My Client singularity Info:

$singularity remote list NAME URI ACTIVE GLOBAL EXCLUSIVE INSECURE SylabsCloud cloud.sylabs.io NO YES NO NO myserver YES NO NO YES >> try to use HTTP protocol

$ singularity push -U test.sif library://asherRTK/test/test.sif:latest WARNING: Skipping container verification FATAL: Unable to push image to library: request did not succeed: http status code: 405

docker-compose logs: uwsgi_1 | No changes detected in app 'users' uwsgi_1 | No changes detected in app 'api' uwsgi_1 | No changes detected in app 'logs' uwsgi_1 | No changes detected in app 'main' uwsgi_1 | No changes detected uwsgi_1 | Operations to perform: uwsgi_1 | Apply all migrations: users uwsgi_1 | Running migrations: uwsgi_1 | No migrations to apply. uwsgi_1 | Operations to perform: uwsgi_1 | Apply all migrations: auth uwsgi_1 | Running migrations: uwsgi_1 | No migrations to apply. uwsgi_1 | Operations to perform: uwsgi_1 | Apply all migrations: admin, api, auth, authtoken, contenttypes, django_rq, logs, main, sessions, social_django, taggit, users uwsgi_1 | Running migrations: uwsgi_1 | No migrations to apply. uwsgi_1 | uwsgi_1 | 164 static files copied to '/var/www/static', 178 unmodified. uwsgi_1 | Starting periodic command scheduler: cron. uwsgi_1 | [uWSGI] getting INI configuration from uwsgi.ini uwsgi_1 | Tue Nov 15 04:43:27 2022 - Starting uWSGI 2.0.21 (64bit) on [Tue Nov 15 04:43:27 2022] uwsgi_1 | Tue Nov 15 04:43:27 2022 - compiled with version: 10.2.1 20210110 on 15 November 2022 04:39:24 uwsgi_1 | Tue Nov 15 04:43:27 2022 - os: Linux-5.13.0-52-generic #59~20.04.1-Ubuntu SMP Thu Jun 16 21:21:28 UTC 2022 uwsgi_1 | Tue Nov 15 04:43:27 2022 - nodename: 35ccbe0b03c9 uwsgi_1 | Tue Nov 15 04:43:27 2022 - machine: x86_64 uwsgi_1 | Tue Nov 15 04:43:27 2022 - clock source: unix uwsgi_1 | Tue Nov 15 04:43:27 2022 - pcre jit disabled uwsgi_1 | Tue Nov 15 04:43:27 2022 - detected number of CPU cores: 48 uwsgi_1 | Tue Nov 15 04:43:27 2022 - current working directory: /code uwsgi_1 | Tue Nov 15 04:43:27 2022 - detected binary path: /usr/local/bin/uwsgi uwsgi_1 | Tue Nov 15 04:43:27 2022 - uWSGI running as root, you can use --uid/--gid/--chroot options uwsgi_1 | Tue Nov 15 04:43:27 2022 - WARNING: you are running uWSGI as root !!! (use the --uid flag) uwsgi_1 | Tue Nov 15 04:43:27 2022 - chdir() to /code/ uwsgi_1 | Tue Nov 15 04:43:27 2022 - your memory page size is 4096 bytes uwsgi_1 | Tue Nov 15 04:43:27 2022 - detected max file descriptor number: 1048576 uwsgi_1 | Tue Nov 15 04:43:27 2022 - lock engine: pthread robust mutexes uwsgi_1 | Tue Nov 15 04:43:27 2022 - thunder lock: disabled (you can enable it with --thunder-lock) uwsgi_1 | Tue Nov 15 04:43:27 2022 - uwsgi socket 0 bound to TCP address :3031 fd 3 uwsgi_1 | Tue Nov 15 04:43:27 2022 - uWSGI running as root, you can use --uid/--gid/--chroot options uwsgi_1 | Tue Nov 15 04:43:27 2022 - WARNING: you are running uWSGI as root !!! (use the --uid flag) uwsgi_1 | Tue Nov 15 04:43:27 2022 - Python version: 3.9.15 (main, Oct 26 2022, 03:47:43) [GCC 10.2.1 20210110] uwsgi_1 | Tue Nov 15 04:43:27 2022 - Python main interpreter initialized at 0x5609b5dd75c0 uwsgi_1 | Tue Nov 15 04:43:27 2022 - uWSGI running as root, you can use --uid/--gid/--chroot options uwsgi_1 | Tue Nov 15 04:43:27 2022 - WARNING: you are running uWSGI as root !!! (use the --uid flag) uwsgi_1 | Tue Nov 15 04:43:27 2022 - python threads support enabled uwsgi_1 | Tue Nov 15 04:43:27 2022 - your server socket listen backlog is limited to 100 connections uwsgi_1 | Tue Nov 15 04:43:27 2022 - your mercy for graceful operations on workers is 60 seconds uwsgi_1 | Tue Nov 15 04:43:27 2022 - mapped 521440 bytes (509 KB) for 16 cores uwsgi_1 | Tue Nov 15 04:43:27 2022 - Operational MODE: preforking+threaded uwsgi_1 | Mon Nov 14 22:43:27 2022 - WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x5609b5dd75c0 pid: 27 (default app) uwsgi_1 | Mon Nov 14 22:43:27 2022 - uWSGI running as root, you can use --uid/--gid/--chroot options uwsgi_1 | Mon Nov 14 22:43:27 2022 - WARNING: you are running uWSGI as root !!! (use the --uid flag) uwsgi_1 | Mon Nov 14 22:43:27 2022 - uWSGI is running in multiple interpreter mode uwsgi_1 | Mon Nov 14 22:43:27 2022 - spawned uWSGI master process (pid: 27) uwsgi_1 | Mon Nov 14 22:43:27 2022 - spawned uWSGI worker 1 (pid: 28, cores: 4) uwsgi_1 | Mon Nov 14 22:43:27 2022 - spawned uWSGI worker 2 (pid: 29, cores: 4) uwsgi_1 | Mon Nov 14 22:43:27 2022 - spawned uWSGI worker 3 (pid: 33, cores: 4) uwsgi_1 | Mon Nov 14 22:43:27 2022 - spawned uWSGI worker 4 (pid: 37, cores: 4) uwsgi_1 | [pid: 37|app: 0|req: 1/1] 10.22.238.89 () {40 vars in 782 bytes} [Mon Nov 14 22:43:52 2022] GET / => generated 8260 bytes in 703 msecs (HTTP/1.1 200) 4 headers in 124 bytes (1 switches on core 0) uwsgi_1 | [pid: 37|app: 0|req: 2/2] 10.22.238.89 () {42 vars in 831 bytes} [Mon Nov 14 22:43:54 2022] GET /login/ => generated 5728 bytes in 18 msecs (HTTP/1.1 200) 4 headers in 124 bytes (1 switches on core 1) uwsgi_1 | [pid: 37|app: 0|req: 3/3] 10.22.238.89 () {42 vars in 922 bytes} [Mon Nov 14 22:43:55 2022] GET /ldap_auth/login/?next=http://my_ip/login/ => generated 5832 bytes in 23 msecs (HTTP/1.1 200) 7 headers in 395 bytes (1 switches on core 2) uwsgi_1 | Binding as uwsgi_1 | Invoking search_s('O=XXXXXXXX', 2, '(uid=asherRTK)') uwsgi_1 | search_s('O=XXXXXXXX', 2, '(uid=%(user)s)') returned 1 objects: cn=asherRTK,o=xxxxxxxx uwsgi_1 | Binding as cn=asherRTK,o=xxxxxxxx uwsgi_1 | Populating Django user asherRTK uwsgi_1 | [pid: 37|app: 0|req: 4/4] 10.22.238.89 () {50 vars in 1148 bytes} [Mon Nov 14 22:44:02 2022] POST /ldap_auth/login/?next=http://my_ip/login/ => generated 0 bytes in 100 msecs (HTTP/1.1 302) 9 headers in 579 bytes (1 switches on core 3) uwsgi_1 | [pid: 37|app: 0|req: 5/5] 10.22.238.89 () {44 vars in 955 bytes} [Mon Nov 14 22:44:02 2022] GET /login/ => generated 6200 bytes in 34 msecs (HTTP/1.1 200) 4 headers in 124 bytes (1 switches on core 0) uwsgi_1 | [pid: 37|app: 0|req: 6/6] 10.22.238.89 () {42 vars in 891 bytes} [Mon Nov 14 22:44:04 2022] GET /collections => generated 8930 bytes in 70 msecs (HTTP/1.1 200) 4 headers in 124 bytes (1 switches on core 1) uwsgi_1 | [pid: 37|app: 0|req: 7/7] 10.22.238.89 () {42 vars in 900 bytes} [Mon Nov 14 22:44:06 2022] GET /collections/1 => generated 12571 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 293 bytes (1 switches on core 2) uwsgi_1 | [pid: 37|app: 0|req: 8/8] () {30 vars in 426 bytes} [Mon Nov 14 22:44:13 2022] GET /assets/config/config.prod.json => generated 481 bytes in 4 msecs (HTTP/1.1 200) 3 headers in 109 bytes (1 switches on core 3) uwsgi_1 | GET NamedEntityView uwsgi_1 | <QueryDict: {}> uwsgi_1 | asherRTK uwsgi_1 | Not Found: /v1/entities/asherRTK uwsgi_1 | [pid: 37|app: 0|req: 9/9] () {32 vars in 481 bytes} [Mon Nov 14 22:44:29 2022] GET /v1/entities/asherRTK => generated 0 bytes in 14 msecs (HTTP/1.1 404) 4 headers in 115 bytes (1 switches on core 0) uwsgi_1 | Method Not Allowed: /v1/entities uwsgi_1 | [pid: 29|app: 0|req: 1/10] () {34 vars in 487 bytes} [Mon Nov 14 22:44:29 2022] POST /v1/entities => generated 41 bytes in 449 msecs (HTTP/1.1 405) 5 headers in 157 bytes (1 switches on core 0)

Do you have any clue about this issue? thanks.

vsoch commented 1 year ago

Can you please format code in code blocks so I can read it?

And try a 3.6x or even a little earlier version of singularity so we can rule out changes to the client.

its 1am here and this is a personal project so I can’t work on during the work day but we can go back and forth in evenings. Thanks!

asherRTK commented 1 year ago

Can you please format code in code blocks so I can read it? -> sorry for the terrible format before, I have corrected it.

And try a 3.6x or even a little earlier version of singularity so we can rule out changes to the client. -> ok, I would try it, but I have to use 3.9.x eventually.

its 1am here and this is a personal project so I can’t work on during the work day but we can go back and forth in evenings. Thanks! -> it's ok, thank you for your reply.

vsoch commented 1 year ago

ok, I would try it, but I have to use 3.9.x eventually.

yes totally OK, this is to help debug the underlying issue.