What is the minimal nginx requirement? (Djano, uWSGI and uWSGI params)

Hello,

I have a Ubuntu 14.04 server with Nginx in version 1.4.6 recompiled with the nginx-http-shibboleth module in version 2.0.0. I made a shibboleth setup which is working if I call for shibbresponder summary.

Miscellaneous
Session Expiration (barring inactivity): 468 minute(s)
Client Address: ***.***.***.***
SSO Protocol: urn:oasis:names:tc:SAML:2.0:protocol
Identity Provider: https://*****/idp/shibboleth
Authentication Time: 2017-05-01T14:30:36.793Z
Authentication Context Class: urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport
Authentication Context Decl: (none)

Attributes
mail: 1 value(s)

At the end, I have use a Django Framework and try to get shibboleth data through a dedicated middleware found here https://github.com/Brown-University-Library/django-shibboleth-remoteuser. But impossible to get data at this point.

Here is a summary of my nginx site config

server {
        listen 443 default_server;
        ssl on;
        ssl_certificate      /etc/ssl/mysite.pem;
        ssl_certificate_key  /etc/ssl/mysite.key;

        index index.html index.htm;
        server_name mysite;

        location / {
                allow all;
                include shib_clear_headers;
                shib_request /shibauthorizer;

                more_clear_input_headers Variable-mail mail;

                shib_request_set $shib_mail $upstream_http_variable_mail;
                uwsgi_param REMOTE_MAIL $shib_mail;
                uwsgi_param REMOTE_TEST "TEST";

                shib_request_set $shib_auth_type $upstream_http_variable_auth_type;
                uwsgi_param Auth-Type $shib_auth_type;

                shib_request_set $shib_shib_application_id $upstream_http_variable_shib_application_id;
                uwsgi_param Shib-Application-ID $shib_shib_application_id;

                shib_request_set $shib_shib_authentication_instant $upstream_http_variable_shib_authentication_instant;
                uwsgi_param Shib-Authentication-Instant $shib_shib_authentication_instant;

                shib_request_set $shib_shib_authentication_method $upstream_http_variable_shib_authentication_method;
                uwsgi_param Shib-Authentication-Method $shib_shib_authentication_method;

                shib_request_set $shib_shib_authncontext_class $upstream_http_variable_shib_authncontext_class;
                uwsgi_param Shib-AuthnContext-Class $shib_shib_authncontext_class;

                shib_request_set $shib_shib_authncontext_decl $upstream_http_variable_shib_authncontext_decl;
                uwsgi_param Shib-AuthnContext-Decl $shib_shib_authncontext_decl;

                shib_request_set $shib_shib_identity_provider $upstream_http_variable_shib_identity_provider;
                uwsgi_param Shib-Identity-Provider $shib_shib_identity_provider;

                shib_request_set $shib_shib_session_id $upstream_http_variable_shib_session_id;
                uwsgi_param Shib-Session-ID $shib_shib_session_id;

                shib_request_set $shib_shib_session_index $upstream_http_variable_shib_session_index;
                uwsgi_param Shib-Session-Index $shib_shib_session_index;

                shib_request_set $shib_remote_user $upstream_http_variable_remote_user;
                uwsgi_param Remote-User $shib_remote_user if_not_empty;

                include /srv/www/mysite/uwsgi_params;
                uwsgi_pass unix:///srv/www/mysite/uwsgi.sock;

        }

        # Shibboleth configurations

        location = /shibauthorizer {
                internal;
                include fastcgi_params;
                fastcgi_pass unix:/var/run/shibboleth/shibauthorizer.sock;
        }

        location /Shibboleth.sso {
                include fastcgi_params;
                fastcgi_pass unix:/var/run/shibboleth/shibresponder.sock;
        }

        location /shibboleth-sp {
                alias /usr/share/shibboleth/;
        }
}

In this case my middleware found "TEST" value for REMOTE_TEST param, nothing for all other specified in this configuration instead of Remote-User which is not setup because detected as empty.

Is there something wrong, compatibility issues on services / modules I want to use together ? Or is it more something like a configuration issue ?

Thanks in advance for your answers.

Firstly, if_not_empty was only added to nginx as of 1.1.11 (docs) -- so that may be a source of problems on nginx 1.4.6.

As for Django, I'd suggest checking if it renames or normalises WSGI environment parameters (eg whether it converts your Remote-User to REMOTE_USER), but a brief look at the WSGI module seems to indicate that at least that module doesn't. Thus, it appears like you're sending Remote-User to Django when it needs REMOTE_USER (as per the middleware). Django's docs on request.META aren't very complete; they state META is for headers, but assuming you're using WSGI, then it's actually the WSGI environ.

In short, you'll need to debug further, perhaps by printing and inspecting request.META and request.environ, and seeing where your attributes are ending up. You could also try hard-coding the REMOTE_USER into your nginx config first to make sure it gets to your application.

Finally, if there's any chance that Django might be mixing HTTP headers and environment variables, make sure you drop them in nginx to avoid the security problem that is spoofing.

Also, we haven't tested nginx-http-shibboleth with nginx 1.4.6 -- our automated testing goes back to 1.8.1. If you're thinking there's a problem with the module after you've gone through the , you can run the tests tests for your nginx version and see what happens. PR with fixes are always welcome.

Firstly, if_not_empty was only added to nginx as of 1.1.11 (docs) -- so that may be a source of problems on nginx 1.4.6.

I think you made a mistake by reading version number 1.11 instead of 1.1.11, isn't it ?

As for Django, I'd suggest checking if it renames or normalises WSGI environment parameters (eg whether it converts your Remote-User to REMOTE_USER), but a brief look at the WSGI module seems to indicate that at least that module doesn't. Thus, it appears like you're sending Remote-User to Django when it needs REMOTE_USER (as per the middleware). Django's docs on request.META aren't very complete; they state META is for headers, but assuming you're using WSGI, then it's actually the WSGI environ.

This is right, and I logged data before this formatting condition. So even if REMOTE_USER is correctly formatted or not, I don't have any data from this variables.

I have this kind of code in python / Django for variables analysis:

for (header, value) in request.META.items():
    logger.warning(header + ': ' + str(value))

That returned this

[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: wsgi.multiprocess: True
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: HTTP_REFERER: https://***.***.***.***:443/
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: uwsgi.version: 2.0.11.2
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: SCRIPT_NAME:
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: REQUEST_METHOD: GET
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: PATH_INFO: /mypath/
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: SERVER_PROTOCOL: HTTP/1.1
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: QUERY_STRING:
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: CONTENT_LENGTH:
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: HTTP_USER_AGENT: Go-http-client/1.1
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: HTTP_CONNECTION: close
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: SERVER_NAME: mysite.com
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: REMOTE_ADDR: ***.***.***.***
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: Shib-AuthnContext-Class:
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: REMOTE_TEST: TEST
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: wsgi.url_scheme: https
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: Shib-Authentication-Method:
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: SERVER_PORT: 443
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: REMOTE_MAIL:
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: uwsgi.node: ip-***-***-***-***
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: SERVER_ADDR: ***.***.***.***
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: DOCUMENT_ROOT: /usr/share/nginx/www
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: Shib-Authentication-Instant:
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: wsgi.input: <uwsgi._Input object at 0x7f9319ea47b0>
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: HTTP_HOST: ***.***.***.***
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: HTTPS: on
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: wsgi.multithread: False
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: Shib-Session-Index:
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: REQUEST_URI: /watch/webcasts/
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: wsgi.version: (1, 0)
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: Shib-Application-ID:
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: wsgi.run_once: False
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: wsgi.errors: <open file 'wsgi_errors', mode 'w' at 0x7f93181b3b70>
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: REMOTE_PORT: 38042
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: REQUEST_SCHEME: https
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: Shib-Identity-Provider:
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: Auth-Type:
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: CONTENT_TYPE:
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: Shib-AuthnContext-Decl:
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: Shib-Session-ID:
[2017/05/02 08:55:53] WARNING middleware->middleware.py:43: wsgi.file_wrapper: <built-in function uwsgi_sendfile>

As you can see, the Shib- and REMOTE_ variables are registred but no value as been passed else for REMOTE_TEST which has been setup as TEST in nginx config.

Finally, if there's any chance that Django might be mixing HTTP headers and environment variables, make sure you drop them in nginx to avoid the security problem that is spoofing.

I tried to get the META en environ variables where I found the Shib- and REMOTE_ variables in the two cases as you mention. I will later remove all unnecessary variables thanks for your advice.

Also, we haven't tested nginx-http-shibboleth with nginx 1.4.6 -- our automated testing goes back to 1.8.1. If you're thinking there's a problem with the module after you've gone through the , you can run the tests tests for your nginx version and see what happens. PR with fixes are always welcome.

Here is my results, but apparently tests where done successfully :/

$ sudo PERL5LIB=$HOME/perl5/lib/perl5 prove
t/shibboleth.t .. ok     
All tests successful.
Files=1, Tests=50,  3 wallclock secs ( 0.04 usr  0.02 sys +  0.37 cusr  0.75 csys =  1.18 CPU)
Result: PASS

Ah yes, too many 1's in that version number; ignore me! Also, thanks for the test results for nginx 1.4.6. I'm not aware of any core nginx changes that might break this module, but there might have been or another module you're also installing might have conflicted.

So, as best I can tell now with your debugging, the issue doesn't appear to be Django -- as you've found, the uwsgi_param variables are shown there. The next thing you should check is your Shibboleth configuration and confirm shibboleth2.xml is set up correctly. Take a look through https://github.com/nginx-shib/nginx-http-shibboleth/blob/master/CONFIG.rst#configuring-shibboleths-shibboleth2xml-to-recognise-secured-paths as well as the 'gotchas' down the bottom of that page -- that's generally the most common issue where shibauthorizer isn't configured with the right domain/path/host/etc in shibboleth2.xml.

If that still doesn't resolve the issue, then you'll need to build nginx with debugging symbols and delve deeper. I've just added debugging steps at https://github.com/nginx-shib/nginx-http-shibboleth/blob/master/README.rst#debugging. If you've got any further suggestions on expanding those debugging instructions, PR are always welcome.

The next thing you should check is your Shibboleth configuration and confirm shibboleth2.xml is set up correctly.

So, basically SWITCH network gave me the shibboleth2.xml file generated from their platform with my provided paths. For now, I tried to install nginx with debug mode and configure it. Here is what I got during login process.

proxied.error.log:2017/06/01 16:33:58 [debug] 15296#0: *1 shib request handler
proxied.error.log:2017/06/01 16:33:58 [debug] 15296#0: *1 shib request done s:200
proxied.error.log:2017/06/01 16:33:58 [debug] 15296#0: *1 shib request handler
proxied.error.log:2017/06/01 16:33:58 [debug] 15296#0: *1 shib request set variables
proxied.error.log:2017/06/01 16:33:58 [debug] 15296#0: *1 shib request authorizer handler
proxied.error.log:2017/06/01 16:33:58 [debug] 15296#0: *1 shib request authorizer allows access

So, as you mentioned in Debugging chapter, the first lines seams to be correct but I didn't got any "authorizer copied header" line. If I understood at this point, I have to check my shibboleth2.xml file or my nginx configuration.

Damn! It was the RequestMapper missing from shibboleth2.xml ...

So, now I got data from shibboleth \o/ thank you very much @davidjb !

No problem, glad you got it sorted. If there's any improvements to our document you think might have helped you, suggestions and pull requests are always welcome.

nginx-shib / nginx-http-shibboleth

What is the minimal nginx requirement? (Djano, uWSGI and uWSGI params) #21