How to make django-environ work with uwsgi emperor?

[tiany]

I use this project with my django projection, deployed by uwsgi(emperor mode) + nginx, it worked well, thanks for your working, but when i touch the uwsgi config file, the env vars will not reload, for now, i can send signal 3 to the process which served my django app,, is there any good ways to make this done?

(sorry for my poor english...

[joke2k]

Hi @tiany don't worry about your english, mine seems worse than your ;)

About your question, i'm using uwsgi too and when i touch con uwsgi file the environment variables is correctly reloaded. Can you post more information like where do you use Env.read_env() and which variables do you want to load?

[gopar]

Had something similar @tiany

What was happening to me was that environ was looking for the file in a different directory other than the project. This is how I fixed it.

import os                                                                                                
environ.Env.read_env(os.path.join(os.getcwd(), ".env"))  

[tiany]

@gopar so your problem is when you start your project, no variable is loaded? I think mine is a little different, it can load all variables, but when i changed the env file and touch uwsgi.ini, uwsgi worker process restarted but all env variables is still the old ones.

@joke2k i wrote the Env.read_env in my project settings.py, env file was putted in the project root directory.

the process is like this:

1. all project goes well, ps -ef | grep uwsgi shows this:

   root     11631     1  0  2015 ?        00:02:12 /usr/local/bin/uwsgi --master --die-on-term --emperor /etc/uwsgi/apps-enabled
   root     14113 11631  0 Jan21 ?        00:01:19 /usr/local/bin/uwsgi --master --die-on-term --emperor /etc/uwsgi/apps-enabled
   www-data 15151 14113  0 Feb25 ?        00:00:38 /usr/local/bin/uwsgi --ini myapp_uwsgi.ini
   www-data 21437 15151  0 16:48 ?        00:00:00 /usr/local/bin/uwsgi --ini myapp_uwsgi.ini
   tiany    21568 15889  0 16:50 pts/4    00:00:00 grep --color=auto uwsgi

2. I change the env var in myapp.env file, and touch myapp_uwsgi.ini, after this, ps -ef | grep uwsgi shows this:

   root     11631     1  0  2015 ?        00:02:12 /usr/local/bin/uwsgi --master --die-on-term --emperor /etc/uwsgi/apps-enabled
   root     14113 11631  0 Jan21 ?        00:01:19 /usr/local/bin/uwsgi --master --die-on-term --emperor /etc/uwsgi/apps-enabled
   www-data 15151 14113  0 Feb25 ?        00:00:39 /usr/local/bin/uwsgi --ini myapp_uwsgi.ini
   www-data 21996 15151  0 16:54 ?        00:00:00 /usr/local/bin/uwsgi --ini myapp_uwsgi.ini
   tiany    22005 15889  0 16:54 pts/4    00:00:00 grep --color=auto uwsgi

3. after this, if i add some new env vars, that can be accessed in my code, but the exist env var will still be with the old value, not updated.

from my research, I can got the env from process 15151, and after touch that process not restarted so the env is not reloaded, either. if i killed pid 15151 all worked.

for now, I write a script to kill this pid, I'd want to find one grace way to do this.

Thanks a lot.

[QasimK]

@joke2k I have encountered the same issue as @tiany: touching the .ini file will not load the new values if the variables have been altered in my .env file

[max-arnold]

@joke2k I have the same issue with v0.4.0. It looks like logical consequence of the following design decisions:

1. Environment is kept in os.environ (and not in Env instance state)
2. There are multiple levels of inheritance: https://github.com/joke2k/django-environ/issues/88
3. Inheritance is handled via https://docs.python.org/3/library/stdtypes.html#dict.setdefault, which won't touch any value if it is already exists

uWSGI reload-os-env = true does not help.

[max-arnold]

Also this http://stackoverflow.com/questions/15619953/django-uwsgi-in-emperor-mode-how-to-reload-wsgi-py-of-a-project

[max-arnold]

@tiany @QasimK Could you please test this patch? https://github.com/joke2k/django-environ/pull/105

[QasimK]

@max-arnold I will test it soon. Although I am not running in uwsgi-emperor mode, from the looks of it this should resolve the issue that I was having.

[QasimK]

@max-arnold I can confirm that #105 resolves this issue for me :smile:

[max-arnold]

@joke2k Daniele, any chances for this PR (https://github.com/joke2k/django-environ/pull/105) to be merged? The problem seems to be fixed, unit tests pass.

[joke2k]

merged! thank you for debugging

[matteosimone]

I am seeing this issue on 0.4.4. I looked into the changes that resulted from this pr and these fixes were reverted. When I do a uwsgi reload after changing a .env file, the new settings are not pulled in from the file. Is anyone else having this issue?

[matteosimone]

Hi , for anyone else who runs into this problem:

If you do not care about environment variables taking priority over variables in .env files, then you can fix this problem by overriding read_env() and have it always overwrite existing keys instead of using cls.ENVIRON.setdefault. This seems to work correctly for my use case, please let me know if you know of any flaws with this approach (aside from changing the expected priority behavior).

[QasimK]

@matteosimone How do you override read_env()? Do you use your own fork of the package?

[matteosimone]

@QasimK make a subclass of Env and use it instead of importing from the django-environ package

It might look something like

import environ

class Env(environ.Env):
    @classmethod
    def read_env(cls, env_file=None, **overrides):
        # Make your own implementation based off the original ..

[khink]

I am still experiencing a problem that looks similar to this one.

We are using django-environ 0.4.5, which should have the #105 fix.

The problem is: touching the uwsgi-reload file does reload production.py (our Django settings file used on this environment), but the .env variables are not updated.

What i mean by this is, when is put MY_SETTING = 2 # or any number in the production.pysettings file, uwsgi-reload updates the value.

But if i use MY_SETTING = env.int('MY_SETTING'), where my .env contains MY_SETTING=2, the reload doesn't change the value if i change it to another number in .env. I see the same thing for an env.bool() setting.

The .env file has been loaded correctly at some point, probably when the uwsgi processes were killed and restarted (instead of reloaded).

Not sure is it matters, but we have master = true in the uwsgi.ini file.

To make sure the path to .env was correct, i also tried putting the full path to the env file in read_env: environ.Env.read_env('/opt/apps/my_django_project/.../settings/.env')

I'm not sure if i'm doing something wrong, or the fix (#105) doesn't cover our case. How could i debug this further?

[max-arnold]

@khink AFAIK, #105 was rolled back as it caused other problems

[khink]

@max-arnold Thanks for your speedy reply. You seem to be right, there's a revert commit: 04829316a106583ae559604247a68425c4c19045

@joke2k Maybe this ticket could be re-opened, if the fix was reverted?

[mattaw]

Please take a look at #225 as it should fix this properly, making django-environ use a copy of the environment instead of the os.environ mapper class.

[duwensi-bgi]

Please pay attention to here: environ.Env.read_env('/path/to/.env')。 When reloading uwsgi process, the read_env method read the env file again indeed. But all existing variables in env file will not be updated because the argument overwrite of read_env method is False by default. That is the reason.