nginx: [emerg] chown("/var/www/images/_upload", 100) failed (1: Operation not permitted)

[mikygit]

Hello, I'm having trouble to launch the registry server via docker-compose (see error in tilte). Nginx seems to require more permissions. I'm on linux, using docker 18.09.7 and launching docker compose under my account.

Any ideas?

[vsoch]

Where is this chown coming from? Those folders should exist in the Nginx container and Nginx should be able to write to them.

[mikygit]

Exactly, that's what I thought too. it's the folders mounted by uwsgi (/var/www/images for instance) I 777 them all but in vain :-(

(base) -bash-4.2$ ll images/ total 2 drwxrwxrwx. 12 nfsnobody nfsnobody 12 May 27 17:17 _upload (base) -bash-4.2$ ll images/_upload/ total 5 drwxrwxrwx. 2 nfsnobody nfsnobody 2 May 27 17:17 0 drwxrwxrwx. 2 nfsnobody nfsnobody 2 May 27 17:17 1 drwxrwxrwx. 2 nfsnobody nfsnobody 2 May 27 17:17 2 drwxrwxrwx. 2 nfsnobody nfsnobody 2 May 27 17:17 3 drwxrwxrwx. 2 nfsnobody nfsnobody 2 May 27 17:17 4 drwxrwxrwx. 2 nfsnobody nfsnobody 2 May 27 17:17 5 drwxrwxrwx. 2 nfsnobody nfsnobody 2 May 27 17:17 6 drwxrwxrwx. 2 nfsnobody nfsnobody 2 May 27 17:17 7 drwxrwxrwx. 2 nfsnobody nfsnobody 2 May 27 17:17 8 drwxrwxrwx. 2 nfsnobody nfsnobody 2 May 27 17:17 9

[vsoch]

Any change of permissions would need to be done in the container (where you are not your regular user). This is hard to help you with because I don't know exactly what you've done.

[mikygit]

I just ran docker-compose up -d actually :-(

[mikygit]

Initially, i even got this one: ERROR: for sregistry_minio_1 Cannot start service minio: b"error while creating mount source path '/home/xxx/projects/mr/sregistry/minio-images': chown /home/xxx/projects/mr/sregistry/minio-images: operation not permitted"

ERROR: for minio Cannot start service minio: b"error while creating mount source path '/home/xxx/projects/mr/sregistry/minio-images': chown /home/xxx/projects/mr/sregistry/minio-images: operation not permitted" ERROR: Encountered errors while bringing up the project.

[vsoch]

@mikygit I just cloned the repository fresh, added the SECRET_KEY to settings, and brought up the containers without a hitch, so I can't reproduce your issue so I'm not sure how to help. From your error messages, it looks like you don't have permissions to create directories across containers, the error is related to not being able to mount. You might need to chat with whomever manages your account, both on the system and docker.

[mikygit]

Ok. Thank for your help. I've been running and deploying docker containers for years on the same servers so i'm quite puzzled. It's definitely to do with nginx user not having the permissions on the host. Will continue digging and keep you informed in case it might be helpull to others. Thanx again.

[vsoch]

I'm sorry I can't help! If I could reproduce the issue it would be fairly easy to look into. After you posted the minio error I don't think the issue is specific to nginx - it looks like the containers aren't able to create volumes. Please do keep digging and update here if you find anything! If it helps, here are my versions:

$ docker --version
Docker version 20.10.3, build 48d30b5
$ docker-compose --version
docker-compose version 1.26.2, build unknown

And I'm on ubuntu 20.04.

[mikygit]

Don't be sorry, no worries ;-) I actually managed to make it work. Well, to the next problem, which is good ;-) I followed those recommandations: https://www.rockyourcode.com/run-docker-nginx-as-non-root-user/ Modified some files here and there.

Now, it fails on the login step. My first question is: could we remove it? I mean, could it be freely open to anyone in the cie? I don't care about security. If not, could a local github be used?

[mikygit]

Hummm looks like it's not possible to use entreprise github :-( Do you confirm?

[mikygit]

Actually I'm saying this because it tried to login on the sregistry web page but actually, if i can push and pull from the registry that would be fine ...

[vsoch]

haha yes I don't give just anyone access to push to the repository :)

If you need to login with GitHub you need to follow the instructions to create developer tokens, it won't work out of the box. And you do need to have a user account to push (not pull). Are your reading the docs?

[mikygit]

Hummm there might have been a bit of misunderstanding from me then. I thought I was installing a local singularity registry. Is it not?

[vsoch]

Yes you are, I misread "push to sregistry" as trying to make changes to the GitHub repo and pushing them here.

[mikygit]

;-) So any chance i could either deactivate the authentification mechanisms or plugin in an entreprise github?

[vsoch]

Is the issue that the enterprise Github has a different callback url?

[mikygit]

No, the issue is that it goes to github since i did not find where to provide our entreprise github url. Here's the http call when clicking on 'login': https://github.com/login/oauth/authorize?client_id=c9f41fc9bsdfhtrsfe02f&response_type=code&state=DxvMAxqJcFVue723834oeN8DIhJwlKK1Th&redirect_uri=http://xxxx:9876/complete/github/

[vsoch]

Ah! I think we can fix that. Follow the instructions here to set the GitHub Enterprise URLs. Make sure to remove the other original one for non enterprise. https://python-social-auth.readthedocs.io/en/latest/backends/github_enterprise.html

[mikygit]

Should I set ENABLE_GITHUB_AUTH to False and set ENABLE_GITHUB_ENTREPRISE_AUTH to True? 'cos it's still heading to normal github. I did not find where to remove the 'original one for non entreprise'. I modified secret.py with the followings: SOCIAL_AUTH_GITHUB_ENTERPRISE_KEY = 'xxx' SOCIAL_AUTH_GITHUB_ENTERPRISE_SECRET = 'xxx' SOCIAL_AUTH_GITHUB_ENTERPRISE_URL = "https://xxx.com/" SOCIAL_AUTH_GITHUB_ENTERPRISE_API_URL = "https://xxx/api/v3/"

[mikygit]

Ok apparently, ENABLE_GITHUB_ENTREPRISE_AUTH won't work since it does not seem to be handled. Any thoughts?

[vsoch]

We are close I think! I can have a PR for you later today or this weekend that adds this variable to the places it needs to be.

[mikygit]

ok thanx!

[mikygit]

could you keep me informed when completed?

[vsoch]

Here is a PR to test! https://github.com/singularityhub/sregistry/pull/365. I would strongly suggest doing a new clone, and making sure you aren't defining other flavors of GitHub auth. I can't test this locally (I don't have enterprise) so it will be up for us to work together talking on here to get it working right.

[mikygit]

Excellent thanx! I'll test it in the week-end, thank you again

[mikygit]

Hello @vsoch, It looks like something is still missing in the credential steps. Now I see the 'enterprise' button on the login page it still redirect to a github authentification although the enterprise settings: https://github.com/login/oauth/authorize?client_id=None&redirect_uri=http://xxx:9876/complete/github/&state=HcBPDFwQuxSScKs5R13QXxJPql3aFBWl&response_type=code

[vsoch]

The docs said that the same GitHub backend is used, but the code suggests otherwise! https://github.com/python-social-auth/social-core/blob/eeee9763862d768627e134a42e4664456b6eabec/social_core/backends/github_enterprise.py.

Let's try with these changes:

https://github.com/singularityhub/sregistry/pull/365/commits/113f11d77b0122276e444b1f76d3d2cbe066a71c.

You will need to re-pull and restart containers. Make sure there isn't an error in the logs before trying again!

[mikygit]

@vsoch hurayyyyyyy, it works! Had the change the DOMAINs config to point to my local server instead on localhost and it seems to be working now. Do i also have to update this one: SOCIAL_AUTH_LOGIN_REDIRECT_URL = "http://127.0.0.1" from auth.py. I guess no.

Thanx a lot!!!

[vsoch]

Woohoo! If you want to give the PR one final review (for docs clarity Etc) I’ll put together a new release tomorrow. Until then, g’nite!

[mikygit]

I will. I need to figure out how to push on my server now. Thought it would be straight forward as for docker but it does not seem to be ... nb: goot night, it's the morning here ;-)

[vsoch]

I would just take the time to setup https for the server. You could also try mkcert : https://singularityhub.github.io/sregistry/docs/install/https#faux-https

[vsoch]

Closed with #365. Happy sleeping (it's morning here now!) :laughing: