Closed lgh2 closed 4 years ago
@lgh2 what does ls -lah warehouse/static
give you when run from the same location?
This is the output:
$ ls -lah
total 596K
drwxr-xr-x 14 user user 4.0K Feb 14 16:28 .
drwxr-xr-x 6 user user 4.0K Feb 12 16:34 ..
-rw-r--r-- 1 user user 25 Jan 8 11:18 .babelrc
drwxr-xr-x 2 user user 4.0K Jan 15 17:45 bin
-rw-r--r-- 1 user user 118 Nov 5 15:08 codecov.yml
-rw-r--r-- 1 user user 406 Nov 5 15:08 CONTRIBUTING.rst
-rw-r--r-- 1 user user 891 Nov 5 15:08 .coveragerc
drwxr-xr-x 2 user user 4.0K Feb 14 12:59 dev
-rw-r--r-- 1 user user 2.4K Jan 15 17:45 docker-compose.yml
-rw-r--r-- 1 user user 6.6K Jan 15 17:45 Dockerfile
-rw-r--r-- 1 user user 954 Jan 15 17:45 Dockerfile.static
-rw-r--r-- 1 user user 65 Jan 8 11:18 .dockerignore
drwxr-xr-x 5 user user 4.0K Feb 14 16:28 docs
drwxr-xr-x 8 user user 4.0K Feb 14 16:34 .git
-rw-r--r-- 1 user user 271 Dec 19 15:21 .gitignore
-rw-r--r-- 1 user user 8.2K Feb 14 16:28 Gulpfile.babel.js
drwxr-xr-x 2 root root 12K Feb 9 15:48 htmlcov
-rw-r--r-- 1 user user 9.5K Nov 5 15:08 LICENSE
-rw-r--r-- 1 user user 5.6K Feb 14 16:28 Makefile
drwxr-xr-x 147 user user 4.0K Nov 28 18:03 node_modules
-rw-r--r-- 1 user user 1.7K Jan 15 17:45 package.json
-rw-r--r-- 1 user user 426K Jan 15 17:45 package-lock.json
drwxr-xr-x 2 user user 4.0K Nov 5 15:08 policies
-rw-r--r-- 1 user user 208 Dec 19 15:21 Procfile
-rw-r--r-- 1 user user 1.5K Jan 29 13:26 README.rst
drwxr-xr-x 2 user user 4.0K Feb 14 16:28 requirements
-rw-r--r-- 1 user user 52 Nov 5 15:08 requirements.txt
-rw-r--r-- 1 user user 13 Dec 19 15:21 runtime.txt
-rw-r--r-- 1 user user 1.2K Feb 9 15:41 .sass-lint.yml
-rw-r--r-- 1 user user 332 Nov 5 15:08 setup.cfg
drwxr-xr-x 3 user user 4.0K Feb 9 15:52 .state
drwxr-xr-x 6 user user 4.0K Feb 9 15:41 tests
-rw-r--r-- 1 user user 1.9K Jan 8 11:18 .travis.yml
drwxr-xr-x 2 user user 4.0K Jan 23 13:27 vcl
drwxr-xr-x 21 user user 4.0K Feb 14 16:28 warehouse
@lgh2 That's the output of ls -lah
, can you try ls -lah warehouse/static
?
I apologize! Here it is:
$ ls -lah warehouse/static
total 24K
drwxr-xr-x 6 user user 4.0K Feb 14 13:21 .
drwxr-xr-x 21 user user 4.0K Feb 14 16:28 ..
drwxr-xr-x 6 root root 4.0K Feb 14 13:21 dist
drwxr-xr-x 2 user user 4.0K Dec 19 15:21 images
drwxr-xr-x 3 user user 4.0K Feb 14 16:28 js
drwxr-xr-x 8 user user 4.0K Feb 12 14:14 sass
@lgh2 It's a little odd that a) you have a dist directory here and b) that it's owned by root
. Did you sudo gulp dist
or something similar at one point? (Shouldn't be necessary, with or without sudo
).
You should be able to sudo rm -rf warehouse/static/dist
to remove that directory, and then make clean
and make purge
should work as expected.
No, I don't think I've ever run sudo gulp dist
.
Thank you for the fix -- it worked!
Hmm, maybe a sudo make serve
or a sudo docker-compose build static
or something could have done it?
At any rate, developers shouldn't be running any of those commands as root, so I don't see this being a reoccurring issue.
I'm agreeing with @di and closing this issue.
None of our documentation tells people to run those commands with sudo
. If one of us runs into the make clean
issue again then we'll be in a better position to look at our bash histories and figure out why we inappropriately used sudo
or what other command got us into this situation.
Thanks all.
We ran into this problem again during a sprint night last week but I didn't think to capture people's bash histories. I do think it would be good to include some troubleshooting help in the Getting Started docs, perhaps along the lines of https://github.com/brainwane/warehouse/tree/make-clean-issue .
I think this could be resolved by fixing https://github.com/pypa/warehouse/issues/1536, which would remove the need to ever write or remove static files to/from the user's filesystem.
I was hoping that #5084 fixed this, but just now I ran into the issue again. I got Permission denied
errors when running make clean
. I ran ls -lah warehouse/static
and the dist
line said:
drwxr-xr-x 6 root root 4.0K Jun 3 14:28 dist
After I ran sudo rm -rf warehouse/static/dist
I was able to make clean
and make purge
without difficulty. And I know -- because this is a fresh machine and I looked through my bash history -- that I
never ran sudo make serve
or sudo docker-compose build static
, nor ran sudo
with any Make commands or with any Warehouse-specific docker commands (I did run sudo docker run hello-world
when checking that Docker had installed correctly).
Just ran into this again.
The issue here is that we mount the warehouse
directory into the container: https://github.com/pypa/warehouse/blob/6721a8aba73e13129f4400a64ff273099ce52850/docker-compose.yml#L65
When our static assets are compiled, this produces the warehouse/static/dist
directory, which is then written onto the host file system.
Originally make clean
was trying to remove this directory. This is unnecessary now as everything happens within the container. I removed this in #8063.
I tried to exclude these directories from being synced back to the host as well, but the only way to do this in docker-compose causes other permissions issues when attempting to modify these directories within the container.
I suspect there may still be issues on certain platforms when gulp
attempts to remove a directory that was created in the container and was synced to the host, but for now since this issue is about make clean
, I'm going to consider this resolved.
It appears that
make clean
fails when run on the Warehouse repository on my machine.Here is the error message I got: