RemoteUser / LDAP

[alpapan]

Hello

I was looking for any documentation for LDAP (RemoteUsers). I couldn't find anything for Apollo 2 (except an open ticket #115). The current documentation only states that we can add this:

            "name":"Remote User Authenticator",
            "className":"remoteUserAuthenticatorService",
            "active":true,

But sorry, I don't know how to configure it...

[alpapan]

Any chance it would be similar to...

https://galaxyproject.org/admin/config/apache-external-user-auth/

Before the virtualhost

<AuthnProviderAlias ldap whatevername-ldap>
  AuthLDAPURL "ldap://HOST:PORT/dc=DCNAME1,dc=DCNAME2"
  AuthLDAPBindDN "cn=ADMINUSER,dc=DCNAME1,dc=DCNAME2"
  AuthLDAPBindPassword "PASSWORD"
</AuthnProviderAlias>

Within the virtualhost:

   <Location "/apollo">
      AuthName "Genome Curation"
      AuthType Basic
      AuthBasicProvider whatevername-ldap
      Require valid-user
      RequestHeader set REMOTE_USER %{AUTHENTICATE_uid}e
   </Location>

It's not working yet:

but I will let update this issue if i resolve it (and hopefully I can document it...)

[alpapan]

(NB for others, these apache modules are needed) ldap.load ldap.conf authnz_ldap.load
proxy_connect.load

[alpapan]

I think the correct AuthLDAPURL ought to have this :

  AuthLDAPURL "ldap://HOST:PORT/dc=DCNAME1,dc=DCNAME2**?mail?sub?(objectClass=person)**"

I can also setup a password file for non-ldap users:

    <Location "/apollo">
       AuthName "Genome Curation"
       AuthType Basic
       AuthBasicProvider file ntg-ldap
       AuthUserFile "/etc/apache2/passes/passfile"
       Require valid-user
       RequestHeader set REMOTE_USER %{AUTHENTICATE_mail}e
    </Location>

(with use of classic htpasswd to create the password file)

That way I can have both types of users (LDAP and hard-coded)... however the initial config best practice still escapes:

These users are created automatically when they first login but they have no permissions set (obviously). I also realised I should setup an admin user (who has entry in the above password file):

    admin{
        username = "unique@email.com"
// where is the system variable defined?
        password = System.getenv("APOLLO_ADMIN_PASSWORD")?:"somepassword"
        firstName = "Dr"
        lastName = "Administrator"
    },

but I also note that if I create a user before the login the first time, then login will not pair them correctly. The only way is for them to login, then the administrator gives them the permissions, then they can work.

[alpapan]

When I add the above administrator and try to login with either

• this new adminisrator
• or my original admin user (role set as admin)

I successfully login but I get the Apollo acccess no organism message

When I look at the database (postgres; grails_user_roles and grails_user), the entry for the admin user is there and the role is set as 'ADMIN'.

When I try to login as any other user with ADMIN, it fails with the same message as above.

If I disable the Apache LDAP authentication it all works... Not sure how to debug this. I tried a couple of browsers but I may need to shutdown my browsers completely.

[alpapan]

(note no errors in the apache log with LogLevel debug)

[alpapan]

Peculiarly, normal users (ROLE = USERS) have no issues. It's only admins.

[alpapan]

Tried on on Firefox, Chrome, IE11 (same result)

On Firefox and Chrome my View Public Genomes doesn't work due to the proxy (separate issue for another day!)

[alpapan]

I converted one of my users (created via LDAP REMOTE_USER) that worked before to admin and log as admin in now works.

I'd love to debug this but i don't really know. I think the only thing I can think of is that only REMOTE_USER created seem to work but from looking at the database, the only differenece is that the metadata column of grails_user is empty for non-REMOTE users created but filled for REMOTE_USERS:

# select id,inactive,metadata,username from grails_user;
  id   | inactive |                       metadata                       |              username
-------+----------+------------------------------------------------------+-------------------------------------
    22 | f        | {"creator":"21"}                                     | test.test@daf.qld.gov.au
    21 | f        | {}                                                   | test@gmail.com
 39950 | f        |                                                      | info@test.science
 39953 | f        | {"INTERNAL_PASSWORD":"173428158887967486536605018"}  | (null)
 39945 | f        | {"INTERNAL_PASSWORD":"1134787337887466918647341396"} | test@test.edu.au

The first row was a user created by a non-remote admin (that's why metadata are not empty). I'll stop now. Anything else I can look for?

[alpapan]

Also: How am I supposed to create users if I don't know their LDAP passwords? Do they have to make an access to create the user and then I have to manually assign them to groups? (I can imagine people/wet-lab scientists getting pissed off with this)

[nathandunn]

I'm going to be ping @abretaud and @erasche as they were instrumental with both the REMOTE_USER and using LDAP.

So, it looks like you figured out most of it. One of the things that @erasche had me do, was add this:

https://genomearchitect.readthedocs.io/en/latest/Configure.html?highlight=register#register-admin-in-configuration

such that it would create the non-LDAP user.

What should happen with this configuration (but let me know if it doesn't) is that if a REMOTE_USER is authorized via LDAP per what you did above, then it should automatically create a new user with that username and a made-up internal password.

You will still need to authorize that user for organisms. Not sure how that part of your workflow would work.

In addition to this, I know there are a few other strategies to authorize clients.

1) use the web-services: http://demo.genomearchitect.org/Apollo2/WebServices/

2) use the web-services with the apollo python client (https://pypi.org/project/apollo/)

I know that @abretaud uses LDAP as well, so would be curious how this work.

[alpapan]

Hello,

Thanks! I already figured out the admin user but it didn't create the internal password:

  id   | inactive |                       metadata                       |              username
-------+----------+------------------------------------------------------+-------------------------------------
 39950 | f        |                                                      | info@test.science

Therefore it's sadly broken. What I ended up doing is

0. turn on ldap authentication
1. register a normal user with ldap,
2. turnoff ldap authentication
3. make said ldap user an admin
4. turn on ldap authentication

i'm using the git head version

Also I still need to figure out how to actually register the LDAP clients with webapollo when I don't know their password. This is needed in order to assign them to groups. The only option I can think of is:

1. user logs in to webapollo and emails me.
2. i set the right organisms permissions and email user.
3. user can now work

i.e. not ideal. Any ideas?

thanks!

[alpapan]

BTW:

One of the things that @erasche had me do, was add this: https://genomearchitect.readthedocs.io/en/latest/Configure.html?highlight=register#register-admin-in-configuration

This is not actually needed if your apache is setup so it can do file authentication as well as ldap (see above for complete block):

       AuthBasicProvider file ntg-ldap
       AuthUserFile "/etc/apache2/passes/passfile"

[alpapan]

ooooh. I have an idea: perhaps first time around i could create the webapollo REMOTE_USER users by actually using this Apache AuthUserFile (with my own made up password). i.e.

1. Use:

      AuthBasicProvider file ntg-ldap
      AuthUserFile "/etc/apache2/passes/passfile"

2. Add my LDAP users to my AuthUserFile with a fake password
3. login to webapollo as each one (programmatically via a webservice). This will populate the webapollo DB. assign them to groups/organisms
4. remove these usernames from the AuthUserFile 5, Let them now login with their username using AuthBasicProvider ldap

I don't know how webapollo will handle the fact that the password has changed (i.e. the metadata column at step 3 will be different from step 5.

Off to teach but will try it tomorrow. Please let me know if it is stupid

[alpapan]

HURRAH!

But one (resolved) issue:

When using the LDAP way of setting REMOTE_USER, it will not work with users defined by a file (which is required if you want to have admin users who are not in the LDAP).

       # this will not work with FILE authentication, only LDAP
       RequestHeader set REMOTE_USER %{AUTHENTICATE_mail}e

rather, this is required instead in location tag:

....
       # this put file passwords as a priority (e.g. if the username is identical).
       AuthBasicProvider file ntg-ldap
....
       # this will work with FILE authentication, and it also works with LDAP (it seems)
        RewriteEngine on
        RewriteCond %{IS_SUBREQ} ^false$
        RewriteCond %{LA-U:REMOTE_USER} (.+)
        RewriteRule . - [E=RU:%1]
        RequestHeader set REMOTE_USER %{RU}e

I can also confirm that my proposed method above works fine:

• duplicate the LDAP users you'd like to create webapollo accounts for in a AuthUserFile using a fake password. By using AuthBasicProvider file ntg-ldap you can override the ldap password with the file (temporarily).
• Login as each user to create the webapollo account.
• Then setup their organism permissions.
• Then remove these usernames from the AuthUserFile to enable the users to login.

Also here is my apache file:

<AuthnProviderAlias ldap ntg-ldap>
  AuthLDAPURL "ldap://server:389/ou=People,dc=hortgenomics,dc=science?mail?sub?(objectClass=person)"
  AuthLDAPRemoteUserAttribute mail
  AuthLDAPBindDN "cn=admin,dc=hortgenomics,dc=science"
  AuthLDAPBindPassword "PASSWORD"
  AuthLDAPSubGroupAttribute memberUid
</AuthnProviderAlias>

<VirtualHost *:80>

    ProxyRequests Off
    ProxyPreserveHost On
    ProxyPass /apollo/stomp/info http://cory.westernsydney.edu.au:8080/apollo/stomp/info
    ProxyPassReverse /apollo/stomp/info http://cory.westernsydney.edu.au:8080/apollo/stomp/info
    ProxyPass /apollo/stomp ws://cory.westernsydney.edu.au:8080/apollo/stomp
    ProxyPassReverse /stomp ws://cory.westernsydney.edu.au:8080/apollo/stomp
    ProxyPass /apollo http://cory.westernsydney.edu.au:8080/apollo
    ProxyPassReverse /apollo http://cory.westernsydney.edu.au:8080/apollo

    <Location "/apollo">
       AuthName "NTG Genome Curation"
       AuthType Basic
       AuthBasicProvider file ntg-ldap
#       AuthBasicProvider file
       AuthUserFile "/etc/apache2/passes/ntg.2"
       Require valid-user
        RewriteEngine on
        RewriteCond %{IS_SUBREQ} ^false$
        RewriteCond %{LA-U:REMOTE_USER} (.+)
        RewriteRule . - [E=RU:%1]
        RequestHeader set REMOTE_USER %{RU}e
        #RequestHeader set REMOTE_USER %{AUTHENTICATE_cn}e
        #RequestHeader set REMOTE_USER %{AUTHENTICATE_mail}e
    </Location>

</VirtualHost>

I still have an unresolved REMOTE USER issue (Login without any genomes) but i think it is a proxy issue and for another day ...

[alpapan]

@nathandunn do you want me to document this? (if you have a github for the docs)

[hexylena]

@alpapan I'll reply to other points later, but for the remote_user case, we usually had an unauthenticated route that would let us access the API, it was a duplicate of the normal routing, but minute authentication. We could then use the python client / command line tools for management. e.g. https://github.com/galaxy-genome-annotation/dockerized-gmod-deployment/blob/master/nginx/default.conf#L27

[abretaud]

Same as @erasche, I use an unanthenticated route to use the api as admin.

At startup there is an admin user created automatically

Once Apollo is up, a bootstrap script is launched to create some default groups/users (using python-apollo). Users created this way are REMOTE_USER = I know they are in the LDAP, and they will login at some point later.

After that, users are logged in with their ldap account using the nginx ldap module setting REMOTE_USER. By default I add them to a group.

We also worked with @mboudet on https://github.com/genouest/ldapollo to synchronize users and groups between an ldap server and Apollo. It's just a croned python script that uses python-apollo to create users and check they are in correct groups. Not yet used in production, but should work fine.

[nathandunn]

Thanks all. @alpapan If you could document this, that would be amazing.

When we move to JBrowse 2, I hope to have a better way of doing this vis passport, etc..

Also, what does your authentication block look like (sorry, should have asked that first)?

I would image it should be like this, so it falls back and uses the next one, but just curious.

 authentications = [
            ["name"     : "Remote User Authenticator",
             "className": "remoteUserAuthenticatorService",
             "active"   : true,
            ]
            ,
            ["name"     : "Username Password Authenticator",
             "className": "usernamePasswordAuthenticatorService",
             "active"   : true,
            ]

    ]

[nathandunn]

@alpapan if you add documentation here via a PR (assuming everything is working how you want it), we can associate it with this ticket. (https://github.com/GMOD/Apollo/blob/develop/docs/Configure.md#other-authentication-strategies)

[alpapan]

Also, what does your authentication block look like (sorry, should have asked that first)?

I would image it should be like this, so it falls back and uses the next one, but just curious.

 authentications = [
            ["name"     : "Remote User Authenticator",
             "className": "remoteUserAuthenticatorService",
             "active"   : true,
            ]
            ,
            ["name"     : "Username Password Authenticator",
             "className": "usernamePasswordAuthenticatorService",
             "active"   : true,
            ]

    ]

yes it does

[alpapan]

@alpapan if you add documentation here via a PR (assuming everything is working how you want it), we can associate it with this ticket. (https://github.com/GMOD/Apollo/blob/develop/docs/Configure.md#other-authentication-strategies)

I will look into the ideas above and once working I will (maybe a new file linked from Configure.md)