Short: removes the need for `pip install -e .' or DJANGO_SETTINGS_MODULE in environment, and adds defaults for the res, log, and pickle directories.
Long: I was also wondering about Issue #203, so I looked into it and found why pip install -e . is necessary: Django projects (i.e. langdeath) and apps (i.e. dld) are not intended to be installed. You are supposed to run them (with WSGI or manage.py runserver) from within the project directory. However, the parser scripts (extern/ld/parser_aggregator.py & extern/ld/load_country_data.py) attempt to import dld.models in order to use the database abstracted objects. For this type of import to work, dld has to be installed in the system---BUT, it is not designed to be installed, and so it uses relative paths to interact with the rest of Django. Installing it directly, using the standard pip install -U ., thus breaks the database.
There is a conflict: dld needs to be installed for extern/ld/parser_aggregator.py to use it, and it cannot be installed, because doing so violates Django expectations and breaks the sqlite database.
So, Judit used a clever solution: pip install -e . which installs packages to the system in "develop mode". Develop mode doesn't copy the files to the system modules directory, rather, it creates a symbolic reference in the system modules directory which points to the place in the filesystem where the modules were installed from. This allows you to edit files in the project directory, and not have to worry about running pip install -U . to update the location that python imports look at. It is a sort of half-installed state; python imports work naturally, but the imported files live in your local project directory, instead of in the system modules directory. All you have to do is remember to run pip install -e . after any time you run pip install -U., and everything works naturally.
The problem is that this behavior subverts the whole concept of "installation" to system directories. It also induces non-obvious side effects. These commits remove the need for pip install -e . by moving directories and files around so that python relative imports work correctly. The only change is that it is now required that the scripts be run from the project directory.
Also, I copied a line from manage.py to set the default for DJANGO_SETTINGS_MODULE. So, you don't have to `export DJANGO_SETTINGS_MODULE=langdeath.settings' before doing anything; the scripts can be run directly.
Finally, I changed the arguments to parser_aggregator.py, and updated doc/read.me to reflect the changed operation instructions.
Short: removes the need for `pip install -e .' or DJANGO_SETTINGS_MODULE in environment, and adds defaults for the res, log, and pickle directories.
Long: I was also wondering about Issue #203, so I looked into it and found why
pip install -e .is necessary: Django projects (i.e. langdeath) and apps (i.e. dld) are not intended to be installed. You are supposed to run them (with WSGI or manage.py runserver) from within the project directory. However, the parser scripts (extern/ld/parser_aggregator.py & extern/ld/load_country_data.py) attempt to import dld.models in order to use the database abstracted objects. For this type of import to work, dld has to be installed in the system---BUT, it is not designed to be installed, and so it uses relative paths to interact with the rest of Django. Installing it directly, using the standardpip install -U ., thus breaks the database.There is a conflict: dld needs to be installed for extern/ld/parser_aggregator.py to use it, and it cannot be installed, because doing so violates Django expectations and breaks the sqlite database.
So, Judit used a clever solution:
pip install -e .which installs packages to the system in "develop mode". Develop mode doesn't copy the files to the system modules directory, rather, it creates a symbolic reference in the system modules directory which points to the place in the filesystem where the modules were installed from. This allows you to edit files in the project directory, and not have to worry about runningpip install -U .to update the location that python imports look at. It is a sort of half-installed state; python imports work naturally, but the imported files live in your local project directory, instead of in the system modules directory. All you have to do is remember to runpip install -e .after any time you runpip install -U., and everything works naturally.The problem is that this behavior subverts the whole concept of "installation" to system directories. It also induces non-obvious side effects. These commits remove the need for
pip install -e .by moving directories and files around so that python relative imports work correctly. The only change is that it is now required that the scripts be run from the project directory.Also, I copied a line from manage.py to set the default for DJANGO_SETTINGS_MODULE. So, you don't have to `export DJANGO_SETTINGS_MODULE=langdeath.settings' before doing anything; the scripts can be run directly.
Finally, I changed the arguments to parser_aggregator.py, and updated doc/read.me to reflect the changed operation instructions.