Issue with SSO authentication setup

[Hydrozyk]

Wondering if anyone else had similar issue and to hopefully debug why it is happening.

I followed Docker-compose installation from here: https://github.com/usnistgov/NEMO/wiki/Docker-Compose-installation Plus nanofab/nginx-shibboleth Docker image: https://hub.docker.com/r/nanofab/nginx-shibboleth for SSO authentication.

Shibboleth is working just fine with our institution IdP and I verified with https://myhost.com/Shibboleth.sso/Login https://myhost.com/Shibboleth.sso/Session login I get the session information so that is not the issue.

Running docker logs -f nemo reveals:

[17/Sep/2024 11:52:51] NEMO.middleware DEBUG Header: HTTP_REMOTE_USER not present or invalid
[17/Sep/2024 11:52:51] NEMO.middleware DEBUG Header: HTTP_REMOTE_USER not present or invalid
[17/Sep/2024 11:52:51] django.request WARNING Bad Request: /login/
Bad Request: /login/

Problem seems to be with nginx passing SHIB_REMOTE_USER header to NEMO app.

Nginx logs show:

[17/Sep/2024:15:54:46 +0000] "POST / HTTP/1.1" 403 2870 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:83.0) Gecko/20100101 Firefox/83.0" REMOTE_USER=- SHIB_REMOTE_USER=
[17/Sep/2024:16:03:41 +0000] "GET /favicon.ico HTTP/1.1" 403 146 "https://myhost.com/login/?next=/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:130.0) Gecko/20100101 Firefox/130.0" REMOTE_USER=- SHIB_REMOTE_USER=-

I followed https://hub.docker.com/r/nanofab/nginx-shibboleth nginx configuration but it still produces above NEMO error.

Nginx config snippet:

# Location secured by Shibboleth
        location / {
            include shib_fastcgi_params;
            include fastcgi_params;
            more_clear_input_headers 'Variable-*' 'Shib-*' 'Remote-User' 'REMOTE_USER' 'Auth-Type' 'AUTH_TYPE';
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            shib_request /shibauthorizer;
            shib_request_set $shib_remote_user $upstream_http_variable_remote_user;
            proxy_http_version 1.1;
            proxy_set_header REMOTE_USER $shib_remote_user;
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-Host $server_name;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header Connection "";
            proxy_pass http://172.18.0.3:8000;
        }
location = /shibauthorizer {
            internal;
            include fastcgi_params;
            include shibboleth_params;
            fastcgi_pass unix:/var/run/shibboleth/shibauthorizer.sock;
        }

settings.py I snippet:

MIDDLEWARE = [
    "django.middleware.security.SecurityMiddleware",
    "django.middleware.common.CommonMiddleware",
    "django.contrib.sessions.middleware.SessionMiddleware",
    "NEMO.middleware.SessionTimeout",  # Add a middleware to identify when the user's session has expired. This works in conjunction with LDAPS authentication on the public facing web server. Kerberos does not need this middleware on the internal facing web server.
    "django.middleware.csrf.CsrfViewMiddleware",
    "django.contrib.auth.middleware.AuthenticationMiddleware",
    "django.contrib.messages.middleware.MessageMiddleware",
    "django.middleware.clickjacking.XFrameOptionsMiddleware",
    "django.middleware.common.BrokenLinkEmailsMiddleware",
    'django.contrib.auth.middleware.RemoteUserMiddleware',
    "NEMO.middleware.DeviceDetectionMiddleware",
    'NEMO.middleware.HTTPHeaderAuthenticationMiddleware',
    "NEMO.middleware.ImpersonateMiddleware",
    "NEMO.middleware.NEMOAuditlogMiddleware",
]
AUTHENTICATION_HEADER = 'REMOTE_USER'
AUTHENTICATION_BACKENDS = ['NEMO.views.authentication.RemoteUserAuthenticationBackend']

shibboleth2.xml is set to:

<ApplicationDefaults entityID="https://myhost.com/shibboleth"
        REMOTE_USER="uid eppn"
        cipherSuites="DEFAULT:!EXP:!LOW:!aNULL:!eNULL:!DES:!IDEA:!SEED:!RC4:!3DES:!kRSA:!SSLv2:!SSLv3:!TLSv1:!TLSv1.1">

As well as attribute-map.xml: <Attribute name="username" id="uid"/>

Any help getting this solved is much appreciated!

[rptmat57]

just to test, you can try using proxy_set_header REMOTE_USER "<your username>"; to make sure the rest works fine in NEMO.

this would confirm that the issue is with the shibboleth configuration/attribute mapping.

[Hydrozyk]

just to test, you can try using proxy_set_header REMOTE_USER "<your username>"; to make sure the rest works fine in NEMO.

this would confirm that the issue is with the shibboleth configuration/attribute mapping.

Just to confirm, to test nginx property will be: proxy_set_header REMOTE_USER "hydrozyk"; in the / correct? Thanks for your feedback!

[rptmat57]

yes, wherever you already have it, you can change the $shib_remote_user to "hydrozyk". this would confirm that nginx does in fact send the correct header and NEMO can grab it and log you in.

my guess here is that the issue comes from the mapping of the attribute, where you shibboleth provider is sending the username as some other variable and that the current shibboleth configuration doesn't map it correctly to the uid or whatever it is using as the $shib_remote_user.

I have to admin I don't know much about that part of the authentication/configuration

[Hydrozyk]

Yes that actually worked! Looks like no CSS styling but it did worked.

[17/Sep/2024 12:23:07] NEMO.views.authentication DEBUG User hydrozyk exists in the database and is active.
[17/Sep/2024 12:23:07] NEMO.views.authentication DEBUG User hydrozyk successfully authenticated with RemoteUserAuthenticationBackend and was granted access.

[rptmat57]

for the static files you can use in nginx:

location /static { alias /etc/nginx/nemo; }

and in docker-compose.yml (assuming your nemo static files are in the ./nemo/static folder from where your docker-compose.yml file is:

services:
  nginx:
    volumes:
      ...
      - ./nemo/static:/etc/nginx/nemo

you'll need to run docker compose up -d nginx for it to take effect

[Hydrozyk]

Great that seems to work! But now when I logout it try to modify users it erros out with: Forbidden (403)

CSRF verification failed. Request aborted.
Help

Reason given for failure:

    Origin checking failed - https://myhost.com does not match any trusted origins.

In general, this can occur when there is a genuine Cross Site Request Forgery, or when [Django’s CSRF mechanism](https://docs.djangoproject.com/en/4.2/ref/csrf/) has not been used correctly. For POST forms, you need to ensure:

    Your browser is accepting cookies.
    The view function passes a request to the template’s [render](https://docs.djangoproject.com/en/dev/topics/templates/#django.template.backends.base.Template.render) method.
    In the template, there is a {% csrf_token %} template tag inside each POST form that targets an internal URL.
    If you are not using CsrfViewMiddleware, then you must use csrf_protect on any views that use the csrf_token template tag, as well as those that accept the POST data.
    The form has a valid CSRF token. After logging in in another browser tab or hitting the back button after a login, you may need to reload the page with the form, because the token is rotated after a login.

You’re seeing the help section of this page because you have DEBUG = True in your Django settings file. Change that to False, and only the initial error message will be displayed.

You can customize this page using the CSRF_FAILURE_VIEW setting.

Nemo log:

Forbidden (Origin checking failed - https://myhost.com does not match any trusted origins.): /deactivate_user/1/
Forbidden (Origin checking failed - https://myhost.com does not match any trusted origins.): /deactivate_user/1/

[rptmat57]

in settings.py, you should have: CSRF_TRUSTED_ORIGINS = ["https://{}".format(ALLOWED_HOSTS[0])]

check that the first of the ALLOWED_HOSTS is "myhost.com", otherwise you can set it directly: CSRF_TRUSTED_ORIGINS = ["https://myhost.com"]

[Hydrozyk]

That solved it!

Going back to SSO auth.

So if shibboleth works and I can login without any issues in to the /Shibboleth.sso/Login test page it could that something with the nemo app handling this remote_user string because that is where the problem is not Shibboleth but I might be missing something here.

[rptmat57]

It looks like NEMO is handling it correctly when hardcoded in nginx. So I think it's the shibboleth config not mapping the username from your identity provider to the correct field, which can be solved by either changing what the SAML provider is sending you or by changing the mapping in the xml files to assign the attribute (that seems to be different than "username") to the $shib_remote_user

It works this way:

1. Nginx redirects the user to IDP (identity provider) for authentication (we know that part works)
2. IDP authenticates the user and sends back user info to Nginx/Shibboleth (we know that part works)
3. Shibboleth maps the "username" attribute that is sent back to the $shib_remote_user variable
4. Nginx sets the REMOTE_USER http header to the value of that $shib_remote_user variable
5. NEMO grabs the REMOTE_USER header and logs the user in (we know that part works)

I think the issue is #3 where we are assuming the username is sent in the username attribute, but it could be sent back as user_name, sAMAccountName, user_id or something else.

Your IT, who provided all the xml files, should be able to identify where the discrepancy is and either change it on their end or tell you where to change it in your xml files.

[Hydrozyk]

Just got a word from them, they are mapping usernames fully-scoped (i.e. somename@myinstitution.edu) and that is their security policy I am told.

I wonder if it can be remapped in shibboleth/attribute-map.xml to pass to nginx without the @myinstitution.edu?

[rptmat57]

NEMO takes care of removing that if it is present, so I don't think that's the issue. Do you know which field they are sending that in?

[Hydrozyk]

NEMO takes care of removing that if it is present, so I don't think that's the issue. Do you know which field they are sending that in?

Sorry could you elaborate more on this? Is your question about what Shibboleth module puts into the REMOTE_USER variable or name of the field?

This is so that I can ask the IdP team the very specific question on this.

Thanks

[rptmat57]

When the header gets to NEMO, NEMO will strip out the domain if it's there. You can test that by using proxy_set_header REMOTE_USER "hydrozyk@example.com"; and it should work just fine

My question is what fields are being sent back after a user authenticates and what is the name of the field where the username is? Is it called "username" or "email" or anything else

[Hydrozyk]

When I go to /Shibboleth.sso/Session I see 3 user attributes:

Attributes
givenName: 1 value(s)
mail: 1 value(s)
sn: 1 value(s)

Is that what is needed? The field names? Is it possible to include test script in NEMO to print out the request headers?

Something like that:

from django.http import HttpResponse

def header_debug_view(request):
    headers = '\n'.join([f'{k}: {v}' for k, v in request.META.items()])
    return HttpResponse(f'<pre>{headers}</pre>')

[Hydrozyk]

I also see the following attributes in attribute-map.xml

<Attribute name="urn:oasis:names:tc:SAML:attribute:subject-id" id="subject-id">
        <AttributeDecoder xsi:type="ScopedAttributeDecoder" caseSensitive="false"/>
    </Attribute>

    <Attribute name="urn:oasis:names:tc:SAML:attribute:pairwise-id" id="pairwise-id">
        <AttributeDecoder xsi:type="ScopedAttributeDecoder" caseSensitive="false"/>
    </Attribute>

    <!-- The most typical eduPerson attributes. -->

    <Attribute name="urn:oid:1.3.6.1.4.1.5923.1.1.1.6" id="eppn">
        <AttributeDecoder xsi:type="ScopedAttributeDecoder" caseSensitive="false"/>
    </Attribute>
    <Attribute name="urn:mace:dir:attribute-def:eduPersonPrincipalName" id="eppn">
        <AttributeDecoder xsi:type="ScopedAttributeDecoder" caseSensitive="false"/>
    </Attribute>

    <Attribute name="urn:oid:1.3.6.1.4.1.5923.1.1.1.9" id="affiliation">
        <AttributeDecoder xsi:type="ScopedAttributeDecoder" caseSensitive="false"/>
    </Attribute>
    <Attribute name="urn:mace:dir:attribute-def:eduPersonScopedAffiliation" id="affiliation">
        <AttributeDecoder xsi:type="ScopedAttributeDecoder" caseSensitive="false"/>
    </Attribute>

    <Attribute name="urn:oid:1.3.6.1.4.1.5923.1.1.1.7" id="entitlement"/>
    <Attribute name="urn:mace:dir:attribute-def:eduPersonEntitlement" id="entitlement"/

[rptmat57]

in shibboleth2.xml, try using:

<ApplicationDefaults entityID="https://myhost.com/shibboleth"
        REMOTE_USER="eppn"
        cipherSuites="DEFAULT:!EXP:!LOW:!aNULL:!eNULL:!DES:!IDEA:!SEED:!RC4:!3DES:!kRSA:!SSLv2:!SSLv3:!TLSv1:!TLSv1.1">

[Hydrozyk]

Thanks for the update! I am getting this error now in NEMO image and wont log me in. django.request WARNING Bad Request: /login/

[rptmat57]

anything in nginx's logs?

[rptmat57]

also do you have a uid attribute in your attribute-map.xml

[Hydrozyk]

In nginx I see:

[19/Sep/2024:15:13:38 +0000] "GET /login/?next=/ HTTP/1.1" 400 46 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:130.0) Gecko/20100101 Firefox/130.0" REMOTE_USER=- SHIB_REMOTE_USER=

attribute-map.xml has this:
<Attribute name="urn:oasis:names:tc:SAML:attribute:subject-id" id="subject-id">
        <AttributeDecoder xsi:type="ScopedAttributeDecoder" caseSensitive="false"/>
</Attribute>
<Attribute name="username" id="uid"/>
<Attribute name="urn:oid:2.5.4.3" id="cn"/>
<Attribute name="urn:oid:2.5.4.4" id="sn"/>
<Attribute name="urn:oid:2.5.4.42" id="givenName"/>
<Attribute name="urn:oid:2.16.840.1.113730.3.1.241" id="displayName"/>
 <Attribute name="urn:oid:0.9.2342.19200300.100.1.1" id="uid"/>
<Attribute name="urn:oid:0.9.2342.19200300.100.1.3" id="mail"/>
.........

Which Shibb attribute exactly NEMO is looking for?

[Hydrozyk]

In nginx I see:

[19/Sep/2024:15:13:38 +0000] "GET /login/?next=/ HTTP/1.1" 400 46 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:130.0) Gecko/20100101 Firefox/130.0" REMOTE_USER=- SHIB_REMOTE_USER=

attribute-map.xml has this:

<Attribute name="urn:oasis:names:tc:SAML:attribute:subject-id" id="subject-id">
        <AttributeDecoder xsi:type="ScopedAttributeDecoder" caseSensitive="false"/>
</Attribute>
<Attribute name="username" id="uid"/>
<Attribute name="urn:oid:2.5.4.3" id="cn"/>
<Attribute name="urn:oid:2.5.4.4" id="sn"/>
<Attribute name="urn:oid:2.5.4.42" id="givenName"/>
<Attribute name="urn:oid:2.16.840.1.113730.3.1.241" id="displayName"/>
 <Attribute name="urn:oid:0.9.2342.19200300.100.1.1" id="uid"/>
<Attribute name="urn:oid:0.9.2342.19200300.100.1.3" id="mail"/>
.........

Which Shibb attribute exactly NEMO is looking for?

[Hydrozyk]

Got a word from IdP team that Okta sends the data in an XML document in the request body, and the user identifier is contained in an XML "Subject" element, in a sub-element called "NameID"

<saml2:Subject xmlns:saml2="urn:oasis:names:tc:SAML:2.0:assertion">
<saml2:NameID Format="urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress">[anyuser@idphost.com</saml2:NameID](mailto:anyuser@idphost.com%3c/saml2:NameID)>
<saml2:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:bearer">
<saml2:SubjectConfirmationData NotOnOrAfter="2024-09-18T13:43:25.725Z"
Recipient="https://myhost.com/Shibboleth.sso/SAML2/POST"/>
</saml2:SubjectConfirmation>
</saml2:Subject>

[rptmat57]

remove <Attribute name="username" id="uid"/> from attribute-map.xml and try again

[Hydrozyk]

I removed it and left just the <Attribute name="urn:oid:0.9.2342.19200300.100.1.1" id="uid"/> which was default there already.

NEMO log:

[19/Sep/2024 12:21:46] NEMO.middleware DEBUG Header: HTTP_REMOTE_USER not present or invalid
[19/Sep/2024 12:21:46] django.request WARNING Bad Request: /login/
Bad Request: /login/

I think the answer lays there somewhere, how to properly pass it to NEMO. What attribute is it looking exactly? In shibboleth2.xml

<ApplicationDefaults entityID="https://myhost.com/shibboleth"
        REMOTE_USER="uid eppn subject-id pairwise-id persistent-id"
        cipherSuites="DEFAULT:!EXP:!LOW:!aNULL:!eNULL:!DES:!IDEA:!SEED:!RC4:!3DES:!kRSA:!SSLv2:!SSLv3:!TLSv1:!TLSv1.1">

[rptmat57]

this means it will map the REMOTE_USER to "uid eppn subject-id pairwise-id persistent-id" in that order or preference.

which I think means if uid isn't present it uses eppn etc.

can you try putting mail at the front of that list?

[Hydrozyk]

Was there a way to see logs for shibboleth in nginx image or application? docker logs nginx just shows nginx logs.

[rptmat57]

yes, docker exec -it nginx tail /var/log/shibboleth/shibd.log and docker exec -it nginx tail /var/log/shibboleth/shibd_warn.log

[Hydrozyk]

All looks good here when I use /Shibboleth.sso/Login which means my IdP supplied configs are all working.

2024-09-20 12:24:51 INFO Shibboleth.Application : multiple CredentialResolver plugins, wrapping in a chain
2024-09-20 12:24:51 INFO Shibboleth.Application : building CredentialResolver of type Chaining...
2024-09-20 12:24:51 INFO XMLTooling.CredentialResolver.Chaining : building CredentialResolver of type File
2024-09-20 12:24:51 INFO XMLTooling.SecurityHelper : loading private key from file (/etc/shibboleth/sp-signing-key.pem)
2024-09-20 12:24:51 INFO XMLTooling.SecurityHelper : loading certificate(s) from file (/etc/shibboleth/sp-signing-cert.pem)
2024-09-20 12:24:51 INFO XMLTooling.CredentialResolver.Chaining : building CredentialResolver of type File
2024-09-20 12:24:51 INFO XMLTooling.SecurityHelper : loading private key from file (/etc/shibboleth/sp-encrypt-key.pem)
2024-09-20 12:24:51 INFO XMLTooling.SecurityHelper : loading certificate(s) from file (/etc/shibboleth/sp-encrypt-cert.pem)
2024-09-20 12:24:51 INFO Shibboleth.Listener : listener service starting
2024-09-20 12:27:47 INFO Shibboleth.SessionCache [1] [default]: new session created: ID (_bfcd47) (http://www.okta.com)

I feel that Nginx / location might be something to do with it. Was NEMO nginx/shibboleth tested with SSO? Any working configuration available for review?

Because other than that I am not sure what it could be, would suspect a BUG since nothing else worked.

Here is my nginx / location which does not seem to call SSO when go to https://myhost.com and it goes directly to NEMO.

# Location secured by Shibboleth
        location /{
            include shib_fastcgi_params;
            include fastcgi_params;
            include shib_clear_headers;
            more_clear_input_headers 'Variable-*' 'Shib-*' 'Remote-User' 'REMOTE_USER' 'Auth-Type' 'AUTH_TYPE';
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            shib_request /shibauthorizer;
            shib_request_set $shib_remote_user $upstream_http_variable_remote_user;
            proxy_http_version 1.1;
            proxy_set_header REMOTE_USER $shib_remote_user;
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-Host $server_name;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header Connection "";
            shib_request_use_headers on;
            proxy_pass http://172.18.0.3:8000;

        }

[20/Sep/2024 08:46:11] NEMO.middleware DEBUG Header: HTTP_REMOTE_USER not present or invalid
[20/Sep/2024 08:46:11] NEMO.middleware DEBUG Header: HTTP_REMOTE_USER not present or invalid
[20/Sep/2024 08:46:11] django.request WARNING Bad Request: /login/

[rptmat57]

A few facilities are using this successfully, including with OKTA. Again my guess is that the mapping/shibboleth2.xml config with REMOTE_USER is not working properly.

You can ask your IT what config you need to have the Shibboleth REMOTE_USER variable contain the username.

Out of curiosity, what do you have in shibboleth2.xml in the RequestMapper element?

[Hydrozyk]

Yes I can ask this but I need to know what exactly NEMO is looking for from shibboleth? Can you post a reference to that section of code or something? Or details code sample/example?

I searched RequestMapper in shibboleth2.xml and do not see any entries for it or any references.

eppn is what contains username myemail@myhost.com in shibboleth/attribute-map.xml

<Attribute name="urn:oid:1.3.6.1.4.1.5923.1.1.1.6" id="eppn">
        <AttributeDecoder xsi:type="ScopedAttributeDecoder" caseSensitive="false"/>
    </Attribute>

Because if I navigate to https://myhost.com/Shibboleth.sso/Session I see entry for it:

 Attributes
eppn: myemail@myhost.com
givenName: someentry
sn: someentry

It is set as first option in shibboleth2.xml

<!-- The ApplicationDefaults element is where most of Shibboleth's SAML bits are defined. -->
    <ApplicationDefaults entityID="https://tjr-lms.mit.edu/shibboleth"
        REMOTE_USER="eppn uid subject-id pairwise-id persistent-id"
        cipherSuites="DEFAULT:!EXP:!LOW:!aNULL:!eNULL:!DES:!IDEA:!SEED:!RC4:!3DES:!kRSA:!SSLv2:!SSLv3:!TLSv1:!TLSv1.1">

[Hydrozyk]

Since I verified that Shibboleth works as expected(info above) I have a feeling that Nginx variable shib_request_set might be a culprit here as outlined in https://hub.docker.com/r/nanofab/nginx-shibboleth

For "/" location /shibauthorizer is not being called and it goes directly to NEMO hence the error. Only if I go to https://myhost.com/Shibboleth.sso/Login I get proper SSO prompt hence the REMOTE_USER variable is empty.

Question remain on how to properly call this process in Nginx?

[rptmat57]

I have this before the ApplicationDefaults section, maybe give it a try:

    <RequestMapper type="XML">
        <RequestMap>
            <Host name="myhost.com"
                  authType="shibboleth"
                  requireSession="true"
                  redirectToSSL="443">
                <Path name="/" />
            </Host>
        </RequestMap>
    </RequestMapper>

[Hydrozyk]

I suppose I could try but I am reading that "/" location is ignored.

https://shibboleth.atlassian.net/wiki/spaces/SP3/pages/2065335006/HowToRequestMap

` <!-- Do NOT do this, it will be ignored!

`

[Hydrozyk]

Tried the request mapper option but it crashes nginx-shib image.

2024-09-24 02:58:02 ERROR XMLTooling.ParserPool : error on line 172, column 34, message: no declaration found for element 'RequestMapper'
2024-09-24 02:58:02 ERROR XMLTooling.ParserPool : error on line 172, column 34, message: attribute 'type' is not declared for element 'RequestMapper'

[rptmat57]

this is the start of my shibboleth2.xml file:

<SPConfig
        xmlns="urn:mace:shibboleth:2.0:native:sp:config"
        xmlns:conf="urn:mace:shibboleth:2.0:native:sp:config"
        xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion"
        xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol"
        xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata"
        clockSkew="180">

    <RequestMapper type="XML">
        <RequestMap>
            <Host name="nemo.stanford.edu"
                  authType="shibboleth"
                  requireSession="true"
                  redirectToSSL="443">
                <Path name="/" />
            </Host>
        </RequestMap>
    </RequestMapper>

[Hydrozyk]

Yeah, this is where I have it too but still crashes. I note that you have shibboleth:2.0 and the nginx-shib image is 3.0

[rptmat57]

ah that might be why it's not working. I haven't updated the config files to shibboleth 3 so I don't know the difference.

I am sorry but I don't think I can be much more helpful here. maybe someone in your IT department can take some time to look at all the configuration files and figure out a way to get it to work.

Please keep us posted if you figure it out!

[Hydrozyk]

Oh, OK you provide the image of 3.0 but have the docs for 2.0... that is something.... Ok.

[Hydrozyk]

Also, there is problem in NEMO image when using PostgreSQL DB, looks like it can't load needed module.

File "/usr/local/lib/python3.11/site-packages/django/db/backends/postgresql/base.py", line 29, in <module>
    raise ImproperlyConfigured("Error loading psycopg2 or psycopg module")
django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 or psycopg module
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/postgresql/base.py", line 25, in <module>
    import psycopg as Database
ModuleNotFoundError: No module named 'psycopg'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/postgresql/base.py", line 27, in <module>
    import psycopg2 as Database
ModuleNotFoundError: No module named 'psycopg2'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/django-admin", line 8, in <module>
    sys.exit(execute_from_command_line())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 416, in execute
    django.setup()
  File "/usr/local/lib/python3.11/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/local/lib/python3.11/site-packages/django/apps/registry.py", line 116, in populate
    app_config.import_models()
  File "/usr/local/lib/python3.11/site-packages/django/apps/config.py", line 269, in import_models
    self.models_module = import_module(models_module_name)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/local/lib/python3.11/site-packages/django/contrib/auth/models.py", line 3, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/usr/local/lib/python3.11/site-packages/django/contrib/auth/base_user.py", line 57, in <module>
    class AbstractBaseUser(models.Model):
  File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 143, in __new__
    new_class.add_to_class("_meta", Options(meta, app_label))
  File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 371, in add_to_class
    value.contribute_to_class(cls, name)
  File "/usr/local/lib/python3.11/site-packages/django/db/models/options.py", line 243, in contribute_to_class
    self.db_table, connection.ops.max_name_length()
                   ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/utils/connection.py", line 15, in __getattr__
    return getattr(self._connections[self._alias], item)
                   ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/utils/connection.py", line 62, in __getitem__
    conn = self.create_connection(alias)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 193, in create_connection
    backend = load_backend(db["ENGINE"])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 113, in load_backend
    return import_module("%s.base" % backend_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/postgresql/base.py", line 29, in <module>
    raise ImproperlyConfigured("Error loading psycopg2 or psycopg module")
django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 or psycopg module

[rptmat57]

you need to install psycopg2 python -m pip install psycopg2-binary an example on how to do that with docker can be found here: https://github.com/usnistgov/NEMO/wiki/Plugin-list#how-to-install-plugins-using-docker

[Hydrozyk]

I finally gave up on Nginx and set up Apache/Shibb docker image and that works just fine.

Also, how do I set up email function in NEMO? Server is using postfix via relayhost relay.myhost.com, no authentication needed.

[rptmat57]

You can just uncomment the lines in settings.py and replace the information: https://github.com/usnistgov/NEMO/blob/master/resources/settings.py#L225

[Hydrozyk]

Thanks. Facing issue with emails for approvals, all clickable URLs within emails arriving as internal docker address of http://172.20.0.2:8000 instead of https://myapp.myhost.com

I have the following in settings.py for ALLOWED_HOSTS, with .01 being Webserver and DB server

ALLOWED_HOSTS = ['myapp.myhost.com', 'localhost', '127.0.0.1', '172.20.0.1', '172.20.0.2', 172.20.0.3']
CSRF_TRUSTED_ORIGINS = ["https://{}".format(ALLOWED_HOSTS[0])]
SERVER_DOMAIN = "https://{}".format(ALLOWED_HOSTS[0])

[rptmat57]

this is likely your proxy configuration.

you can use something like this for apache:

Header set Host "myapp.myhost.com"
Header set X-Forwarded-For "myapp.myhost.com"

[Hydrozyk]

Any idea what could be happening after switching to NEMO-ce version with URLS and isntalling plugins with docker-compose file and Docker image: "registry.gitlab.com/nemo-community/nemo-ce:6.0.24"

[29/Oct/2024 10:13:49] NEMO.urls WARNING no urls found for NEMO plugin: NEMO.apps.nemo_ce [29/Oct/2024 10:13:49] NEMO.urls DEBUG automatically including urls for plugin: NEMO_reports [29/Oct/2024 10:13:49] NEMO.urls DEBUG automatically including urls for plugin: NEMO_billing [29/Oct/2024 10:13:49] NEMO.urls DEBUG automatically including urls for plugin: NEMO_billing.rates [29/Oct/2024 10:13:49] NEMO.urls DEBUG automatically including urls for plugin: NEMO_billing.invoices [29/Oct/2024 10:13:49] NEMO.urls DEBUG automatically including urls for plugin: NEMO_publications [29/Oct/2024 10:13:49] NEMO.urls DEBUG automatically including urls for plugin: NEMO_stockroom [29/Oct/2024 10:13:49] NEMO.urls DEBUG automatically including urls for plugin: NEMO_user_details [29/Oct/2024 10:13:49] NEMO.urls DEBUG automatically including urls for plugin: NEMO.apps.sensors [29/Oct/2024 10:13:49] NEMO.urls DEBUG automatically including urls for plugin: NEMO.apps.kiosk [29/Oct/2024 10:13:49] NEMO.urls DEBUG automatically including urls for plugin: NEMO.apps.contracts

INSTALLED_APPS = [
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.admin',
    'django.contrib.humanize',
    'NEMO_reports', # This needs to be before NEMO_billing (if installed) and NEMO
    'NEMO_billing',  # Add before NEMO to have new navbar menu items show up
    'NEMO_billing.rates',
    'NEMO_billing.invoices',
    'NEMO_publications',
    'NEMO_stockroom', # Add before NEMO to have new navbar menu items show up
    'NEMO_user_details',
    'NEMO.apps.sensors',
    'NEMO.apps.kiosk',
    'NEMO.apps.contracts',
    'NEMO.apps.nemo_ce', 
    'NEMO',
    "rest_framework",
    "rest_framework.authtoken",
    "django_filters",
    "mptt",
    "auditlog",
]

[rptmat57]

this looks all normal, there are no urls per se in NEMO-CE, the urls are directly included in NEMO. the nemo-ce app is just there to keep track of migrations that are not present in NEMO