search

overhangio / tutor

The Docker-based Open edX distribution designed for peace of mind
https://docs.tutor.overhang.io/
GNU Affero General Public License v3.0
922 stars 440 forks source link

Debug edx-platform env setup error #232

Closed carlos-wong closed 5 years ago

carlos-wong commented 5 years ago

Bug description

I am followed the https://docs.tutor.overhang.io/dev.html#prepare-the-edx-platform-repo. Try to setup the debug env.

my tutor ver: 3.5.3 use the openedx-platform: https://github.com/edx/edx-platform
git describe: release-2017-01-25-8422-g441d6384ee taged: open-release/ironwood.2

WARNING:py.warnings:/openedx/edx-platform/lms/djangoapps/courseware/__init__.py:5: DeprecationWarning: Importing 'lms.djangoapps.courseware' as 'courseware' is no longer supported
  warnings.warn("Importing 'lms.djangoapps.courseware' as 'courseware' is no longer supported", DeprecationWarning)

Traceback (most recent call last):
  File "./manage.py", line 120, in <module>
    startup.run()
  File "/openedx/edx-platform/lms/startup.py", line 19, in run
    django.setup()
  File "/openedx/venv/local/lib/python2.7/site-packages/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/openedx/venv/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models()
  File "/openedx/venv/local/lib/python2.7/site-packages/django/apps/config.py", line 202, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/openedx/edx-platform/lms/djangoapps/bulk_email/models.py", line 16, in <module>
    from openedx.core.djangoapps.course_groups.cohorts import get_cohort_by_name
  File "/openedx/edx-platform/openedx/core/djangoapps/course_groups/cohorts.py", line 9, in <module>
    from courseware import courses
  File "/openedx/edx-platform/lms/djangoapps/courseware/courses.py", line 25, in <module>
    from courseware.module_render import get_module
  File "/openedx/edx-platform/lms/djangoapps/courseware/module_render.py", line 60, in <module>
    from openedx.core.djangoapps.bookmarks.services import BookmarksService
  File "/openedx/edx-platform/openedx/core/djangoapps/bookmarks/services.py", line 12, in <module>
    from . import DEFAULT_FIELDS, api
  File "/openedx/edx-platform/openedx/core/djangoapps/bookmarks/api.py", line 11, in <module>
    from .models import Bookmark
  File "/openedx/edx-platform/openedx/core/djangoapps/bookmarks/models.py", line 41, in <module>
    class Bookmark(TimeStampedModel):
  File "/openedx/venv/local/lib/python2.7/site-packages/django/db/models/base.py", line 118, in __new__
    "INSTALLED_APPS." % (module, name)
RuntimeError: Model class openedx.core.djangoapps.bookmarks.models.Bookmark doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.

I think it is the env setup bug

How to reproduce

export TUTOR_EDX_PLATFORM_PATH=/path/to/edx-platform
tutor dev run lms pip install --requirement requirements/edx/development.txt
tutor dev run lms python setup.py install
tutor dev run lms paver update_assets --settings=tutor.development
tutor dev runserver lms --edx-platform-path=/path/to/edx-platform

I setup a total clean project

Environment

tutor, version 3.5.3 Ubuntu 18.04.2 LTS

Additional context

regisb commented 5 years ago

Hi again @carlos-wong! Thanks for the thorough bug report.

export TUTOR_EDX_PLATFORM_PATH=/path/to/edx-platform tutor dev run lms pip install --requirement requirements/edx/development.txt tutor dev run lms python setup.py install tutor dev run lms paver update_assets --settings=tutor.development tutor dev runserver lms --edx-platform-path=/path/to/edx-platform

Just to be clear: there is no need to pass --edx-platform-path in that last command if you already defined the TUTOR_EDX_PLATFORM_PATH earlier.

Are your development settings correct? Can you please post the content of $TUTOR_EDX_PLATFORM_PATH/lms/envs/tutor/development? (don't forget to obfuscate the secret parameter values)

carlos-wong commented 5 years ago

@regisb Hi agian :). thanks for you reply. I am removed the --edx-platform-path the error is remove. 

System check identified no issues (0 silenced).
July 06, 2019 - 08:24:30
Django version 1.11.21, using settings 'lms.envs.tutor.development'
Starting development server at http://0.0.0.0:8000/
Quit the server with CONTROL-C.

But I can't connect to the ip:8000. I am debug the server code on a remote server and want test on my laptop web browser. I just saw the ip is open for 0.0.0.0:8000. Should I config the domain name?

regisb commented 5 years ago

I am removed the --edx-platform-path the error is remove. Ok so I'll close this issue.

But I can't connect to the ip:8000. I am debug the server code on a remote server and want test on my laptop web browser. I just saw the ip is open for 0.0.0.0:8000. Should I config the domain name?

No, pointing to the domain name would not work. Instead, you should connect by ssh and setup a tunnel so that http://localhost:8000 will point to your server instead of your local laptop. I suggest you run:

ssh -D 5555 myusername@myserver

And then in your browser install an extension like FoxyProxy (https://getfoxyproxy.org/) such that you tunnel through localhost:5555. This is out of the scope of tutor issues, though.

carlos-wong commented 5 years ago

@regisb Got it. Thanks for you patience.

carlos-wong commented 5 years ago

@regisb Hi, sorry for the https://github.com/overhangio/tutor/issues/232#issuecomment-508907847. it is my mistake. I forgot open the port on the network rule setting page. after I am enable the 8000 port, I can browse the website on my laptop but the server run on the azure server.

carlos-wong commented 5 years ago

@regisb can you reopen this one? I am checked the code. When start

tutor dev runserver lms

event I am defined export TUTOR_EDX_PLATFORM_PATH

after start I am checked the container the path of my local repo is not mount to replace the image's code path.

carlos-wong commented 5 years ago

docker-compose -f /home/carlos/.local/share/tutor/env/local/docker-compose.yml --project-name tutor_dev run --rm -e SETTINGS=tutor.development --volume=/home/carlos/.local/s hare/tutor/env/build/openedx/themes:/openedx/themes -p 8000:8000 lms ./manage.py lms runserver 0.0.0.0:8000

if I am start the lms without pass the path the docker-compose command is not mount the code repo.

RuntimeError: Model class openedx.core.djangoapps.bookmarks.models.Bookmark doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.
Error: Command failed with status 1: docker-compose -f /home/carlos/.local/share/tutor/env/local/docker-compose.yml --project-name tutor_dev run --rm -e SETTINGS=tutor.development --volume=/home/carlos/.local/share/tutor/env/build/openedx/themes:/openedx/themes --volume=/home/carlos/openedx/edx-platform:/openedx/edx-platform -e USERID=0 -p 8000:8000 lms ./manage.py lms runserver 0.0.0.0:8000RuntimeError: Model class openedx.core.djangoapps.bookmarks.models.Bookmark doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.
Error: Command failed with status 1: docker-compose -f /home/carlos/.local/share/tutor/env/local/docker-compose.yml --project-name tutor_dev run --rm -e SETTINGS=tutor.development --volume=/home/carlos/.local/share/tutor/env/build/openedx/themes:/openedx/themes --volume=/home/carlos/openedx/edx-platform:/openedx/edx-platform -e USERID=0 -p 8000:8000 lms ./manage.py lms runserver 0.0.0.0:8000

if I am start the server with the path the lms will start failed. But I am done

tutor dev run lms pip install --requirement requirements/edx/development.txt
tutor dev run lms python setup.py install
tutor dev run lms paver update_assets --settings=tutor.development

this install command.

regisb commented 5 years ago

This is strange.

  1. Are you quite sure that you defined the TUTOR_EDX_PLATFORM_PATH environment variable in the same terminal as the other dev run commands?
  2. What is the complete output of tutor dev run lms python setup.py install?
carlos-wong commented 5 years ago

yes. I am add the env value to .bashrc. And I am checked by echo $TUTOR_EDX_PLATFORM_PATH before I am run the command .

this is the log

docker-compose -f /home/carlos/.local/share/tutor/env/local/docker-compose.yml --project-name tutor_dev run --rm -e SETTINGS=tutor.development --volume=/home/carlos/.local/share/tutor/env/build/openedx/themes:/openedx/themes -p 8000:8000 lms python setup.py install
running install
running bdist_egg
running egg_info
writing requirements to Open_edX.egg-info/requires.txt
writing Open_edX.egg-info/PKG-INFO
writing top-level names to Open_edX.egg-info/top_level.txt
writing dependency_links to Open_edX.egg-info/dependency_links.txt
writing entry points to Open_edX.egg-info/entry_points.txt
reading manifest file 'Open_edX.egg-info/SOURCES.txt'
writing manifest file 'Open_edX.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/cms
copying cms/wsgi.py -> build/lib.linux-x86_64-2.7/cms
copying cms/urls.py -> build/lib.linux-x86_64-2.7/cms
copying cms/__init__.py -> build/lib.linux-x86_64-2.7/cms
copying cms/urls_dev.py -> build/lib.linux-x86_64-2.7/cms
copying cms/celery.py -> build/lib.linux-x86_64-2.7/cms
copying cms/startup.py -> build/lib.linux-x86_64-2.7/cms
copying cms/conftest.py -> build/lib.linux-x86_64-2.7/cms
creating build/lib.linux-x86_64-2.7/lms
copying lms/wsgi.py -> build/lib.linux-x86_64-2.7/lms
copying lms/urls.py -> build/lib.linux-x86_64-2.7/lms
copying lms/__init__.py -> build/lib.linux-x86_64-2.7/lms
copying lms/celery.py -> build/lib.linux-x86_64-2.7/lms
copying lms/startup.py -> build/lib.linux-x86_64-2.7/lms
copying lms/wsgi_apache_lms.py -> build/lib.linux-x86_64-2.7/lms
copying lms/tests.py -> build/lib.linux-x86_64-2.7/lms
creating build/lib.linux-x86_64-2.7/openedx
copying openedx/__init__.py -> build/lib.linux-x86_64-2.7/openedx
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/lms
copying build/lib.linux-x86_64-2.7/lms/wsgi.py -> build/bdist.linux-x86_64/egg/lms
copying build/lib.linux-x86_64-2.7/lms/urls.py -> build/bdist.linux-x86_64/egg/lms
copying build/lib.linux-x86_64-2.7/lms/__init__.py -> build/bdist.linux-x86_64/egg/lms
copying build/lib.linux-x86_64-2.7/lms/celery.py -> build/bdist.linux-x86_64/egg/lms
copying build/lib.linux-x86_64-2.7/lms/startup.py -> build/bdist.linux-x86_64/egg/lms
copying build/lib.linux-x86_64-2.7/lms/wsgi_apache_lms.py -> build/bdist.linux-x86_64/egg/lms
copying build/lib.linux-x86_64-2.7/lms/tests.py -> build/bdist.linux-x86_64/egg/lms
creating build/bdist.linux-x86_64/egg/cms
copying build/lib.linux-x86_64-2.7/cms/wsgi.py -> build/bdist.linux-x86_64/egg/cms
copying build/lib.linux-x86_64-2.7/cms/urls.py -> build/bdist.linux-x86_64/egg/cms
copying build/lib.linux-x86_64-2.7/cms/__init__.py -> build/bdist.linux-x86_64/egg/cms
copying build/lib.linux-x86_64-2.7/cms/urls_dev.py -> build/bdist.linux-x86_64/egg/cms
copying build/lib.linux-x86_64-2.7/cms/celery.py -> build/bdist.linux-x86_64/egg/cms
copying build/lib.linux-x86_64-2.7/cms/startup.py -> build/bdist.linux-x86_64/egg/cms
copying build/lib.linux-x86_64-2.7/cms/conftest.py -> build/bdist.linux-x86_64/egg/cms
creating build/bdist.linux-x86_64/egg/openedx
copying build/lib.linux-x86_64-2.7/openedx/__init__.py -> build/bdist.linux-x86_64/egg/openedx
byte-compiling build/bdist.linux-x86_64/egg/lms/wsgi.py to wsgi.pyc
byte-compiling build/bdist.linux-x86_64/egg/lms/urls.py to urls.pyc
byte-compiling build/bdist.linux-x86_64/egg/lms/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/lms/celery.py to celery.pyc
byte-compiling build/bdist.linux-x86_64/egg/lms/startup.py to startup.pyc
byte-compiling build/bdist.linux-x86_64/egg/lms/wsgi_apache_lms.py to wsgi_apache_lms.pyc
byte-compiling build/bdist.linux-x86_64/egg/lms/tests.py to tests.pyc
byte-compiling build/bdist.linux-x86_64/egg/cms/wsgi.py to wsgi.pyc
byte-compiling build/bdist.linux-x86_64/egg/cms/urls.py to urls.pyc
byte-compiling build/bdist.linux-x86_64/egg/cms/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/cms/urls_dev.py to urls_dev.pyc
byte-compiling build/bdist.linux-x86_64/egg/cms/celery.py to celery.pyc
byte-compiling build/bdist.linux-x86_64/egg/cms/startup.py to startup.pyc
byte-compiling build/bdist.linux-x86_64/egg/cms/conftest.py to conftest.pyc
byte-compiling build/bdist.linux-x86_64/egg/openedx/__init__.py to __init__.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying Open_edX.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying Open_edX.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying Open_edX.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying Open_edX.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying Open_edX.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying Open_edX.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/Open_edX-0.11-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing Open_edX-0.11-py2.7.egg
Copying Open_edX-0.11-py2.7.egg to /openedx/venv/lib/python2.7/site-packages
Removing Open-edX 0.11 from easy-install.pth file
Adding Open-edX 0.11 to easy-install.pth file

Installed /openedx/venv/lib/python2.7/site-packages/Open_edX-0.11-py2.7.egg
Processing dependencies for Open-edX==0.11
Searching for setuptools==39.0.1
Best match: setuptools 39.0.1
Adding setuptools 39.0.1 to easy-install.pth file
Installing easy_install script to /openedx/venv/bin
Installing easy_install-3.6 script to /openedx/venv/bin

Using /openedx/venv/lib/python2.7/site-packages
Finished processing dependencies for Open-edX==0.11

I think the docker-compose is not mount the code repo too. Does it install to the container not the local reop?

carlos-wong commented 5 years ago

this is my echo command

carlos@open-edx-dev:~$ echo $TUTOR_EDX_PLATFORM_PATH
/home/carlos/openedx/edx-platform
carlos-wong commented 5 years ago

Updated

if I am install the lib by pass --edx-platform-path=/path/to/edx-platform

the command work.

append to 

tutor dev run lms pip install --requirement requirements/edx/development.txt
tutor dev run lms python setup.py install
tutor dev run lms paver update_assets --settings=tutor.development

this three command

regisb commented 5 years ago

This is really strange...

What is the complete output of echo $TUTOR_EDX_PLATFORM_PATH && tutor dev run lms python setup.py install?

carlos-wong commented 5 years ago 
/home/carlos/openedx/edx-platform
docker-compose -f /home/carlos/.local/share/tutor/env/local/docker-compose.yml --project-name tutor_dev run --rm -e SETTINGS=tutor.development --volume=/home/carlos/.local/share/tutor/env/build/openedx/themes:/openedx/themes -p 8000:8000 lms python setup.py install

I think the env is not work with the tutor command. 

carlos@open-edx-dev:~$ sudo tutor --version
tutor, version 3.5.3

this is my version

carlos-wong commented 5 years ago

@regisb Hi, Sorry to bother your. How's going on with this issue?

regisb commented 5 years ago

Wait, why do you run tutor as sudo? I believe tutor does not pick up environment variables defined by the original user when you run it as sudo.

carlos-wong commented 5 years ago

@regisb oh, I think this is the problem. Because the docker on my system need root right to run. I should change docker file mode.

regisb commented 5 years ago

This is the problem, as we can see with the following piece of code:

➭ export TUTOR_EDX_PLATFORM_PATH=/home/regis/edx-platform
➭ sudo python3 -c "import os; print(os.environ.get('TUTOR_EDX_PLATFORM_PATH'))"
None

Next time, please type the exact requested commands! it makes it much easier for me to debug.

carlos-wong commented 5 years ago

@regisb ok, sorry for that.