Closed jmatsushita closed 7 years ago
The demo course doesn't affect migrations. Migrations should have been run earlier, but may have failed. We've seen that in the past. Please post the complete output of running make devstack.provision
.
Thanks for your help. Here it is https://gist.github.com/jmatsushita/d0b0876e48e7a45f0f39d3d7863e5369
---> pavelib.servers.update_db
---> pavelib.prereqs.install_prereqs
---> pavelib.prereqs.install_node_prereqs
Node prereqs unchanged, skipping...
---> pavelib.prereqs.install_python_prereqs
---> pavelib.prereqs.uninstall_python_packages
pip freeze
pip uninstall --disable-pip-version-check -y edxval
Uninstalling edxval-0.0.13:
Exception:
Traceback (most recent call last):
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/pip/commands/uninstall.py", line 76, in run
requirement_set.uninstall(auto_confirm=options.yes)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/pip/req/req_set.py", line 346, in uninstall
req.uninstall(auto_confirm=auto_confirm)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/pip/req/req_install.py", line 754, in uninstall
paths_to_remove.remove(auto_confirm)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/pip/utils/__init__.py", line 267, in renames
shutil.move(old, new)
File "/usr/lib/python2.7/shutil.py", line 300, in move
rmtree(src)
File "/usr/lib/python2.7/shutil.py", line 256, in rmtree
onerror(os.rmdir, path, sys.exc_info())
File "/usr/lib/python2.7/shutil.py", line 254, in rmtree
os.rmdir(path)
OSError: [Errno 39] Directory not empty: '/edx/app/edxapp/venvs/edxapp/lib/python2.7/site-packages/edxval-0.0.13-py2.7.egg-info'
Pre-reqs are failing to install, preventing the migrations from being run. @nedbat do we still need to uninstall these packages? Can we take steps to get rid of that script?
Why is it failing to uninstall edxval? That seems odd.
We almost certainly don't need to still uninstall edxval, but this code was intended to be lightweight and idempotent, as a way to prevent infuriating devstack problems. We occasionally need to add new packages to the uninstall list, so I'd be very hesitant to remove the uninstall code completely. Do we think this is something particular to edxval?
Thanks, splitting the file did make the error feedback come at the right time indeed. https://gist.github.com/jmatsushita/bfcea18fb5fe9b94a9b8ce47b4ac6d22
I'll try to investigate why edxval uninstall fails.
There's something quite strange going on with the file system in the docker image:
ls -la /edx/app/edxapp/venvs/edxapp/lib/python2.7/site-packages/
# chown -R edxapp:edxapp /edx/app/edxapp/venvs/edxapp/lib/python2.7/site-packages
chown: cannot access `/edx/app/edxapp/venvs/edxapp/lib/python2.7/site-packages/edx_enterprise-0.27.0.dist-info': No such file or directory
chown: cannot access `/edx/app/edxapp/venvs/edxapp/lib/python2.7/site-packages/edxval-0.0.12-py2.7.egg-info': No such file or directory
chown: cannot access `/edx/app/edxapp/venvs/edxapp/lib/python2.7/site-packages/pa11ycrawler-1.6.1.dist-info': No such file or directory
chown: cannot access `/edx/app/edxapp/venvs/edxapp/lib/python2.7/site-packages/xblock_poll-1.2.5-py2.7.egg-info': No such file or directory
chown: cannot access `/edx/app/edxapp/venvs/edxapp/lib/python2.7/site-packages/edxval-0.0.13-py2.7.egg-info/PKG-INFO': No such file or directory
chown: cannot access `/edx/app/edxapp/venvs/edxapp/lib/python2.7/site-packages/edxval-0.0.13-py2.7.egg-info/SOURCES.txt': No such file or directory
chown: cannot access `/edx/app/edxapp/venvs/edxapp/lib/python2.7/site-packages/edxval-0.0.13-py2.7.egg-info/dependency_links.txt': No such file or directory
chown: cannot access `/edx/app/edxapp/venvs/edxapp/lib/python2.7/site-packages/edxval-0.0.13-py2.7.egg-info/installed-files.txt': No such file or directory
chown: cannot access `/edx/app/edxapp/venvs/edxapp/lib/python2.7/site-packages/edxval-0.0.13-py2.7.egg-info/requires.txt': No such file or directory
chown: cannot access `/edx/app/edxapp/venvs/edxapp/lib/python2.7/site-packages/edxval-0.0.13-py2.7.egg-info/top_level.txt': No such file or directory
I'm checking if it could be a file system issue on the machine I'm using. And I'll take a look at the docker build files.
The edxops/edxapp:devstack
container on the docker hub has this problem. Rebuilding a fresh edxapp container from edx/configuration
(i.e. make docker.build.edxapp
) doesn't result in the same unaccessible files in the site-packages repo.
Not exactly sure how this all came to be (the unaccessbile files are old versions of the packages, so maybe a race condition with the build and the pip package registry being updated?).
It might be a good idea to:
Thanks for bringing this to my attention. I have been building the images locally from a branch of edx/configuration. The problem is most likely due to the fact that, since edxapp takes so long to build, I have been using previous builds as a base image (e.g. FROM edxops/edxapp:devstack
). I will build one from scratch, and push it.
Excellent! It makes sense. I did bump into another obstacle after rebuilding from edx/configuration
master branch (a missing devstack.sh
entrypoint) so thanks for pointing me to the branch you've been using!
A new image has been pushed to Docker Hub. Let me know if you are still having issues.
Well unfortunately with the locally rebuilt image from the branch, I do get rid of the unaccessible files but the make devstack.provision
still fails with the same error message.
OSError: [Errno 39] Directory not empty: '/edx/app/edxapp/venvs/edxapp/lib/python2.7/site-packages/edxval-0.0.13-py2.7.egg-info'
I'm trying with a fresh pull but I think it's something else.
This issue seems to point at a Kernel and Docker overlay file system interaction. I'm looking into it further.
I confirm that this issue came from a problem with the docker storage backend and the kernel! Sorry: it had nothing to do with EdX.
No worries. This is still good information to know for future issues. Thanks!
Hi there,
Awesome to see that there's docker based devstack!
After running
make devstack.provision
there are a few errors in the run (details below) but the script completes with a greenProvisioning complete!
.When connecting to port 18000 the following error shows up in the browser.
It seems that the problem is the import of the demo course: