Issue with moving from Lock 9 to Lock 11

[joshcanhelp]

Splitting this off from #48 by @coros-sanborn.

We used to get the app_metadata back which included the roles: data[:extra][:raw_info][:app_metadata][:myAppName][:roles]

Since you want to use Lock and you need to use v11 going forward, I think your best bet in this case is a custom claim for the ID token. Here is the Rules code you're looking for, modified from here:

function (user, context, callback) {
  const namespace = 'https://myapp.example.com/';
  context.idToken[namespace + 'app_metadata'] = user.app_metadata;
  callback(null, user, context);
}

One thing to note ... some browsers have a problem with the cross-browser authentication that Lock 11 does. To fix this, you'll need a custom domain setup for your tenant.

The universal login (which is v9) should at least work and return the same information as embedded lock v9 - which it does not (since I no longer get the app_metadata).

The actual version of Lock is less relevant than the API endpoints it uses. I wouldn't really even consider how the Lock widget on the universal page works since it behaves differently when it's located there. As you've said, that doesn't return the app_metadata either.

To be clear, I'm only looking at the universal login as a last resort.

Understood. That said, our recommendation stands to use a redirect to the universal login page with a web app like one built on Rails.

I have a rails 4 app that had the embedded login page with lock 9. It worked fine for years. It stopped working in the last couple weeks because lock 9 was deprecated. As soon as I upgraded to lock 11 the page will no longer redirect, even if the user is authenticated.

I'm clear on the missing app_metadata but can you clarify what you mean by "As soon as I upgraded to lock 11 the page will no longer redirect"?

This is the problem that I and many others are having and we really need a resolution.

That's what I'm here for :)

If lock 9 worked then upgrading to lock 11 should work, maybe with a few tweaks, but the entire app should not fail.

I agree. I think that's a shortcoming in our migration guide, to be honest. The new authentication endpoints only provide OIDC conformant fields and app_metadata is not one of them.

And to add to the challenge there is not any documentation on how to configure the embedded login with Rails. The quickstart example is only for using the universal login page.

That was intentional, RE: our recommendation to use the universal login page.

[coros-sanborn]

Thanks you for sticking with me on this. I'll speak with my manager about using the Universal Login.

With that in mind, everything from here forward will relate to using the universal login. I do have that very close to working. The confusing part is the claim. Thank you for providing the code chunk, but I'm not clear where it goes. Does that code go into the hosted page?

function (user, context, callback) {
  const namespace = 'https://myapp.example.com/';
  context.idToken[namespace + 'app_metadata'] = user.app_metadata;
  callback(null, user, context);
}

Do you mind walking me through where this code goes and how to use it? Below is the code I currenlty have setup on the universal login page. Thank you.

  <script src="https://cdn.auth0.com/js/lock-9.1.min.js"></script>  

  <script>   
    // Decode utf8 characters properly
    var config = JSON.parse(decodeURIComponent(escape(window.atob('@@config@@'))));
    var connection = config.connection;
    var prompt = config.prompt;
    var initializationOptions = {
      assetsUrl:  config.assetsUrl,
      cdn:        config.cdn
    };

    var lock = new Auth0Lock(
      config.clientID, 
      config.auth0Domain, 
      initializationOptions);

    lock.show({
      icon: 'https://www.mezurit.com/static_images/sanborn-arrow.png',
      callbackURL: config.callbackURL,
      responseType: config.callbackOnLocationHash ? 'token' : 'code',
      dict: config.dict,
      connections: connection ? [connection] : null,
      rememberLastLogin:  !prompt,
      container: 'widget-container',
      authParams: config.internalOptions
    });

   </script>

[coros-sanborn]

I figured out where to add the rule and how to try it. However I'm still not getting the "roles".
This is the output:

The profile is: 
{
  "name": "jdoe@foobar.com",
  "email": "jdoe@foobar.com",
  "nickname": "jdoe",
  "picture": "http://foobar.com/pictures/jdoe.png",
  "user_id": "auth0|0123456789",
  "identities": [
    {
      "provider": "auth0",
      "user_id": "0123456789",
      "connection": "Username-Password-Connection",
      "isSocial": false
    }
  ]
}
The rules context is: 
{
  "clientID": "123456789",
  "clientName": "MyWebApp",
  "connection": "MyDbConn",
  "connectionStrategy": "auth0",
  "protocol": "oidc-basic-profile",
  "request": {
    "query": {
      "scope": "openid"
    },
    "body": {},
    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36",
    "ip": "X.X.X.X",
    "geoip": {
      "country_code": "AR",
      "country_code3": "ARG",
      "country_name": "Argentina",
      "region": "08",
      "city": "Federal",
      "postal_code": "3180",
      "latitude": -30.954599380493164,
      "longitude": -58.78329849243164,
      "continent_code": "SA",
      "time_zone": "America/Argentina/Buenos_Aires"
    }
  },
  "samlConfiguration": {},
  "stats": {
    "loginsCount": 5
  },
  "accessToken": {},
  "idToken": {}
}

[coros-sanborn]

Please, I can use some help getting the user.app_metadata which includes the use roles.

Clearly user roles are important and they need to be included. I realize the standard does not include roles and that's why we're adding them to the idToken or accessToken, but it does not work.

I've added the following rule:

function (user, context, callback) {
  const namespace = 'https://www.mezurit.com/';
  context.idToken[namespace + 'app_metadata'] = user.app_metadata || {};
  context.accessToken[namespace + 'app_metadata'] = user.app_metadata || {};
  callback(null, user, context);
}

When I try this rule I get the following:

...
"accessToken": {
    "https://www.mezurit.com/data": {}
  },
  "idToken": {
    "https://www.mezurit.com/app_metadata": {}
  }

If I hard code a value just for testing I do get a result:

function (user, context, callback) {
  const namespace = 'https://www.mezurit.com/';
  context.idToken[namespace + 'app_metadata'] = user.app_metadata || {roles:["user"]};
  context.accessToken[namespace + 'app_metadata'] = user.app_metadata || {roles:["user"]};
  callback(null, user, context);
}

  "accessToken": {
    "https://www.mezurit.com/app_metadata": {
      "roles": [
        "user"
      ]
    }
  },
  "idToken": {
    "https://www.mezurit.com/app_metadata": {
      "roles": [
        "user"
      ]
    }
  }

This tells me that user.app_metadata is retuning nothing.

All the documentation, including several google searches, indicated that the rule is correct. Is there an other setting somewhere that needs to be set (that's not mentioned in the docs)?

[joshcanhelp]

@coros-sanborn - Looking now. Just FYI, urgent support is probably best run through our support group (assuming you're on a paid plan).

Will test a few things out now and respond ASAP.

[joshcanhelp]

@coros-sanborn - I tested this all the way through on my end and it works as expected. Here's what I did:

1. Added the Rule like so:

function (user, context, callback) {
  const namespace = 'http://localhost:3000/';
  context.idToken[namespace + 'app_metadata'] = user.app_metadata;
  callback(null, user, context);
}

No need to add the app_metadata to the accessToken, the the ID token.

2. Saved the Rule, then clicked Try This Rule. In the modal that popped up, I added this to the User data:

{
  // ..
    "app_metadata" : {
        "MyAppName": {
            "roles" : [
                "user", "editor"
            ]
        }
    }
}

... to match the structure on your end.

3. Clicked Try and see this:

{
  // ...
    "MyAppName": {
      "roles": [
        "user",
        "editor"
      ]
    }
  },
  // ...
}

This tells me that the Rule is functioning properly. Make sure to run this test yourself to diagnose.

4. I added that same app_metadata structure to my test used in the Auth0 dashboard.

5. Finally, I restarted Puma and ran through my local app (built straight from the quickstart) login process with the same test user. In the secured view I'm using for the redirect, I output session[:userinfo].inspect and see:

{
  // ...
  "http://localhost:3000/app_metadata"=>{"AppName"=>{"roles"=>["user", "editor"]}}
}

Besides the accessToken line, your Rule looks fine. Are you leaving out the app_metadata in the test user? Or if you're testing in your app, are you namespacing the claim and using a user that has that data stored?

[coros-sanborn]

I found the problem that was causing the app_metadata to be lost.

The documentation says to create a file called auth0.rb under config/initializers and to configure the OmniAuth middleware as shown below:

Rails.application.config.middleware.use OmniAuth::Builder do
  auth0_domain = ENV['AUTH0_DOMAIN']

  provider(
    :auth0,
    ENV['AUTH0_CLIENT_ID'],
    ENV['AUTH0_CLIENT_SECRET'],
    auth0_domain,
    callback_path: '/auth/auth0/callback',
    authorize_params: {
      scope: 'openid profile email',
      audience: 'https://myTenant.auth0.com/userinfo'
    }
  )
end

Part of the problem appears to be with the "audience" attribute.

If I specify the audience attribute as shown it will return the structure below:

  "extra"=>{
    "raw_info"=>{
      .....
      "https://app_meta_data"=>{
        "Mezurit"=>{
          "roles"=>[
            "user"
          ]
        }
      }
    }

So it returns the app_metadata nested under the namespace I assigned in the rule "https://app_meta_data".

If I remove the audience attribute from the configuration then I get the following structure:

  ....
  "extra"=>{
    "raw_info"=>{
      "app_metadata"=>{
        "Mezurit"=>{
          "roles"=>[
            "user"
          ]
        }
      },
      "Mezurit"=>{
        "roles"=>[
          "user"
        ]
      },
      "email_verified"=>true,
      ....
  }

As you can see the app_metadata is now included. I'm not sure why it seems to be repeated in the next block.

If I remove audience attribute AND disable the rule I still get the app_metadata:

  "extra"=>{
    "raw_info"=>{
      "app_metadata"=>{
        "Mezurit"=>{
          "roles"=>["user"]
        }
      }, 
      "Mezurit"=>{
        "roles"=>["user"]
      }, 
      .....
    }
  }

So following the documentation is what caused everything to go haywire. I'm not sure why I'm getting the different structures and it seems like leaving the audience attribute off in the config AND disabling/deleting the rule will also give the proper structure, so I'll go with that for simplicity, unless there's a compelling reason not to.

FYI, I was never able to get the rule testing to work - I had to run the app to see the results. It seems that updating the documentation would be quite helpful, since the way it is now has made things confusing and basically broke what was working before.

[joshcanhelp]

The audience parameter in the authorize URL is what forces you to use the newer authentication endpoints (that's what Lock 11 uses). Those endpoints only support OIDC claims for ID tokens and at the /userinfo endpoint, app_metadata not being one of those. When you take that parameter off, turn off (or leave off) OIDC compliance in your Application, and have the legacy user profile API turned on, you can do what you're doing here.

So following the documentation is what caused everything to go haywire.

That's in the documentation because it's how we recommend folks to authenticate. The quickstarts are meant for new applications/installs and new tenants. Anyone that signed up in the last year or less will not be able to do what you're describing here so we don't want to include that in the documentation.

it seems like leaving the audience attribute off in the config AND disabling/deleting the rule will also give the proper structure, so I'll go with that for simplicity, unless there's a compelling reason not to.

I would move to the new format at some point. APIs marked "legacy" could be deprecated in the future (with notice, of course). Our main goal is to get folks using the most secure and compliant authentication possible. Not only that but you mentioned wanting to use Lock for authentication and the only way to do that with your current setup is with the Rules. Try it out using the steps from my last comment on a test application and feel free to ask any additional questions you've got in this thread.

---

I'm glad this all got figured out and sincerely apologize for the trouble. This API deprecation was necessary for security reasons and we tried to provide as much notice and support as possible. I brought this up to the team that handled this process so they're aware of this gap.

[coros-sanborn]

Thank you for all your help.

One last thing I'd like to get clarity on. The Hosted Pages for the Universal login is using lock 9 are you suggesting that the hosted page should be using lock 11?

<script src="https://cdn.auth0.com/js/lock-9.1.min.js"></script>  
  <script>
    // Decode utf8 characters properly
    var config = JSON.parse(decodeURIComponent(escape(window.atob('@@config@@'))));

    var connection = config.connection;
    var prompt = config.prompt;

    var initializationOptions = {
      assetsUrl:  config.assetsUrl,
      cdn:        config.cdn
    };

    var lock = new Auth0Lock(
      config.clientID, 
      config.auth0Domain, 
      initializationOptions);

    lock.show({
      icon: 'https://www.example.com/static_images/sanborn-arrow.png',
      callbackURL: config.callbackURL,
      responseType: config.callbackOnLocationHash ? 'token' : 'code',
      dict: config.dict,
      connections: connection ? [connection] : null,
      rememberLastLogin:  !prompt,
      container: 'widget-container',
      authParams: config.internalOptions
    });

  </script>

I did try that yesterday and you mentioned that the lock version did not need to be upgraded to lock 11.

I don't mind updating to lock 11, I just need to know what the lock.show parameters are supposed to look like. When I tried it yesterday the page would not redirect.

Which is the best way to move forward to ensure the production app will continue to work in the future?

[joshcanhelp]

The Hosted Pages for the Universal login is using lock 9 are you suggesting that the hosted page should be using lock 11?

No, not at this time. Like I mentioned before, don't worry about the Lock version there.

I don't mind updating to lock 11, I just need to know what the lock.show parameters are supposed to look like. When I tried it yesterday the page would not redirect.

I asked about that earlier. Can you explain what you mean by "the page would not redirect"?

I have an example of using Lock that works with the quickstart-built app (just tested):

https://github.com/joshcanhelp/auth0-ruby-basic/blob/master/app/views/static_pages/lock.html.erb

A few notes:

• jQuery needs to be loaded in the header in this example
• You'll need to have @client_id and @domain defined in your controller
• A state value needs to be generated, stored in session['omniauth.state'], and then assigned to @state

Which is the best way to move forward to ensure the production app will continue to work in the future?

The universal login page is the recommended way to authenticate, both now and in the future. Lock will continue to be supported but ULP is what our quickstarts and architects use. The pros and cons of each are here.

[coros-sanborn]

To clarify - I'm only talking about using the universal login. We are not using the embedded login because of all the issues we've had.

So, everything I'm referring to is for the Hosted Pages - Universal Login with a rails app.

Above I mentioned that the "audience" parameter and the rule were returning different results and caused the initial problem because I added the "audience" attribute to the Omniauth config as indicated by the documentation.

I noticed that if I remove the rule and the audience attribute I get the "app_metadata" included (like we've had for years).

Above you mentioned:

The audience parameter in the authorize URL is what forces you to use the newer authentication endpoints (that's what Lock 11 uses). Those endpoints only support OIDC claims for ID tokens and at the /userinfo endpoint, app_metadata not being one of those. When you take that parameter off, turn off (or leave off) OIDC compliance in your Application, and have the legacy user profile API turned on, you can do what you're doing here. ....I would move to the new format at some point. APIs marked "legacy" could be deprecated in the future (with notice, of course).

Does your comment to "..move to the new format..." only refer to the embedded login? The universal login? Or both?

[coros-sanborn]

If someone signs up and verifies their email, shouldn't they get redirected back to the app? At the moment the user can verify but the verification page does not redirect back to the app.

It stays here: https://sanborn.auth0.com/lo/verify_email?ticket=PM4nHQYnKRmP5SWqTphKOqD8ldzJeHQb#

How can I get the user back to the production app once they sign up and verify?
I'm using the Universal Login.

[coros-sanborn]

Same question as above with the password reset. How can I get the user back to the production app once they verify their email or change their password?

[joshcanhelp]

I'm only talking about using the universal login.

Thank you for the clarification.

Does your comment to "..move to the new format..." only refer to the embedded login? The universal login? Or both?

I meant using ULP with an audience parameter and the custom claims for the ID token.

If someone signs up and verifies their email, shouldn't they get redirected back to the app?

Not from the email link, no.

How can I get the user back to the production app once they sign up and verify?

They should just have the app open in another tab, right? You can also format the email template if you're using your own email provider.

Same question as above with the password reset.

Same response RE: email templates. Password reset also has a hosted page you can customize.

[coros-sanborn]

Thanks for the responses. I really do appreciate your help, however it appears that Auth0 does not really care about the problem they have caused their existing customers.

After checking with my team it's clear that we received no notification of these changes. Sure there is a notice on the dashboard, but we don't stare at that on a daily or even monthly basis. We set up Auth0 a while back, it worked and we expect it to continue working.

If Auth0 makes a drastic change they should provide a migration path that matches the options that were available before; in our case a migration path and documentation for an embedded login page in a rails app. It's unfathomable that Auth0 would intentionally withhold documentation (as you shared in a previous post) for embedded logins because they want people to migrate to the universal login.

Login, Email Verification, Password reset. All these worked when we were using the embedded login with Rails. They should continue to work in the same manner, with minor tweaks, if the requirement is to migrate to the Universal login.

Since this required migration, our logins, email verification and password resets do not work as before. I get it that Auth0 wants everyone to use the universal login. I'm trying to do that, but I still have to do a lot of extra work in order to get my production app back online. Password reset and Email Verification used to work and should continue to work after the migration! The documentation is dismal and as you're aware it's taken several posts to try to get things to work again.

This migration has been horrible for our business. Our production app has been down for over a week and has affected our current and new customers in a unacceptable manner, in fact, we have lost customers. Working with a third party authentication platform should be reliable. This is the gateway for our customers and we rely on it working. I'm sure there are several people that would agree with me based on all the google and stack overflow questions.

At this point we will have to drop Auth0 and find an other solution. It's really a shame since it used to work so well.

Thanks again for your help. I'm frustrated that Auth0 has made these changes without thinking of existing customers and how this might affect their business.

[joshcanhelp]

@coros-sanborn - I'm sorry to hear this but I understand. If you're open to working with one of our technical support engineers on getting your system running smoothly, please reach out to me directly josh.cunningham@auth0.com.

Please know that we do take your concerns seriously and I've passed your feedback along to the right group here at Auth0. This deprecation was necessary from a security standpoint but your specific use case could have been covered better.