cristi.crossculturalconsult.com yeilds Internal Server Error

[cccs-ip]

Cristi is requesting help to figure out what he did to produce 'Internal Server Error' on http://cristi.crossculturalconsult.com

[pwhipp]

500 server errors can be a pain because they can come from any one of many sources. Investigation should begin with a check of the logs. The first log to check is the one I generate from the spawned processes (typically in ~/gunicorn_logs//error.log)

In this case we see:

    respiter = self.wsgi(environ, resp.start_response)
  File "/home/cristi/.virtualenvs/cccs/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 187, in __call__
    self.load_middleware()
  File "/home/cristi/.virtualenvs/cccs/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 47, in load_middleware
    mw_instance = mw_class()
  File "/home/cristi/.virtualenvs/cccs/local/lib/python2.7/site-packages/django/middleware/locale.py", line 24, in __init__
    for url_pattern in get_resolver(None).url_patterns:
  File "/home/cristi/.virtualenvs/cccs/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 369, in url_patterns
    raise ImproperlyConfigured("The included urlconf %s doesn't have any patterns in it" % self.urlconf_name)
ImproperlyConfigured: The included urlconf core.urls doesn't have any patterns in it

The easiest way to then work with something like this is to debug it locally (using django runserver which displays errors like this in the console while you are working.

If it has to be debugged it on the server for some reason, you can use django shell to good effect.

For this error, I can't see anything relevant changes to the core/urls.py file (git log core/urls.py) and the error looks very strange because I can see the patterns when I poke at them using django shell.

So I did a guick google and found this

Cristi, you need to add:

DEBUG_TOOLBAR_PATCH_SETTINGS = False

to your personal settings. Then restart your service.

The debug toolbar is not generally used in production environments and 99% of developers are therefore only using it under django runserver where the problem does not occur which is why this is such an obscure issue.

When you are working on the server with the likes of the debug toolbar, you might find it more convenient to stop the service (sudo service cristi_cccs stop) and then run django runserver in a convenient terminal - you'll be able to see feedback on access constantly and errors like this in the terminal appearing while you work. Just remember to restart the service when done.

[cccs-ip]

Just to be clear, would this change be made in the ~/core/settings/production.py file (or dev.py or something similar)? I am still not entirely clear how one should go about making changes to these files that would not make subsequent changes in a pull or push.

[pwhipp]

It is confusing until you get used to deployments/virtualenvs/git/django. Divide and conquer:

Consider just a folder with a Django project in it. It is served using a virtualenv which holds all the installed modules and which, for convenience, identifies the settings file being used.

For example, staging has a virtualenv called staging which uses DJANGO_SETTINGS_MODULE=core.settings.staging. Alternatively, I have a virtualenv called 'cccs' which serves my local copy with DJANGO_SETTINGS_MODULE=core.settings.dev.

The folder with the Django project in it is looked after by git to manage changes. This has no direct impact on the above other than by helping to make certain versions available and to synchronize the folder with our Github repository. To deploy code, you git pull it from Github to the relevant working copy.

Most of the working copies for web services use deploy keys. You can make local changes to the files and even commit them locally but you will not be able to push those changes to the Github repository because deploy keys only have read access.

Thus Cristi just changes his own settings file (cristi.py) for this because the debug toolbar is not run by default (he'd put the change in settings/includes/common.py if that were the case so that all of the different settings would make use of the change).

A glitch in all this is that we currently only use a single requirements.txt. On complex projects I generally create a requirements folder that sorta mirrors the structure of the settings folder so it can cater for requirements that are not needed on all deployments (things like the debug toolbar which is just for development or gunicorn which is typically just for production services). We can do this but then users need to remember to e.g. pip install -r requirements/production.txt or pip install -r requirements/dev.txt - with just one dev package, this does not seem worth the bother. If we move to TDD or add substantial code analysis and testing utilities to our dev and testing environments it would be worth doing at that stage.

That is just python at the end of the day - each settings file in core/settings just imports the common settings as the first thing it does and then makes its own specific additions or changes.

[cccs-ip]

Thanks, Paul, though I remain a bit confused in the sense that all files are shared via Git, are they not? So if we create an aaron.py and an cristi.py, they would all be uploaded into branch master. Then with each push and pull, we would not be toggling between the aaron.py or cristi.py, as it is referenced according to how we set up our virtualenv.... for example, via echo "export DJANGO_SETTINGS_MODULE=abadi.settings.dev" >> ~/.virtualenvs/abadi/bin/postactivate

...if cristi did not re-create the production.py as cristi.py and is running his virtualenv as production.py, then his local settings would change via production and his changes would be pushed to master and pulled to the rest of the sites?

This requires vigilance and we need to ensure that all developers are creating their own settings.py files. This is not strongly stated in the current docs; we should revise accordingly.

[cccs-ip]

by the way: is it correct that the answer is "yes, this is an additional config to add to ~/core/settings/XXwhatever.py ?"

[pwhipp]

Your aaron.py and cristi.py are shared so anyone could duplicate your settings by using your module. The only settings file that is not shared is secrets.py.

There is no toggling though or need to recreate any production.py settings because the virtualenv setup determines which settings module is used and the virtualenv is not part of the repository. Thus your local system always uses aaron.py, mine uses dev.py and cristi's uses cristi.py. Git does not affect that. This makes it easy for me to 'check out' Cristi's settings locally (e.g. django runserver --settings core.settings.cristi) or for another developer to copy one of our settings files to work from.

You are right - If someone modifies production.py then they are modifying the production settings and there is nothing to stop someone from using the production settings locally (nor should there be). This is normal Python/Django though - the names are explicit. Most developers would just use core.settings.dev. Do you want me to update the docs?

[cccs-ip]

Thanks, Paul. And yes, I think we should make it more explicit in that section of the docs that the common developer settings file should be dev.py, but that developers are encouraged to make their own settings.py file (e.g. cristi.py) in the case that they need to make setting changes (as the one identified here). That way, other developers also using the dev.py file (like Ben.. or me) won't be surprised by a if a pull changes the dev.py settings file.

[cccs-ip]

Closing this issue as Cristi's site appears to be back online.