How to find the bidirectional map between Cognito identity ID and Cognito user information?

[baharev]

Given the Cognito identity ID, I would like to programmatically find the user name, e-mail address, etc. For example, one issue is that each user gets his/her own folder in S3 (e.g. private/${cognito-identity.amazonaws.com:sub}/ according to the myproject_userfiles_MOBILEHUB_123456789 IAM policy) but I cannot relate that folder name (S3 prefix) to the user attributes in my user pool. The closest thing that I have found is this rather complicated code:

AWS Lambda API gateway with Cognito - how to use IdentityId to access and update UserPool attributes?

Is it my best bet? Is it really this difficult?

(As a workaround, I would be happy with a post confirmation lambda trigger that creates for example a ${cognito-identity.amazonaws.com:sub}/info.txt file in some S3 bucket, and in the info.txt file it could place the user sub from the user pool. I am not sure that this is feasible at all, it was just an idea.)

[jonsmirl]

When the identity is created in the identity pool, can't you add 'logins' to the call? Those logins get stored in the identity pool providing a way to map between Google/Facebook/User Pool ID and the Identity pool ID. 'Logins' are optional, but they are very useful.

--logins (map) A set of optional name-value pairs that map provider names to provider tokens.

Edit - you have the logins in Auth.ts/setCredentialsFromFederation. Why aren't these getting stored in the identity pool along with the ID from federation, I'm sure that data used to be in my identity pools, so where did it go?

Here's an identity pool entry from my mobile app, it has the User Pool ID in it. The mobile app is based on code from Mobile Hub.

jonsmirl@ubuntu-16:~$ aws cognito-identity describe-identity --identity-id us-east-1:bea3f57c-4564-47cf-b82c-a515f719d8a5 --profile bill { "Logins": [ "cognito-idp.us-east-1.amazonaws.com/us-east-1_Cf6AJpe20" ], "LastModifiedDate": 1512005509.277, "CreationDate": 1512005509.237, "IdentityId": "us-east-1:bea3f57c-4564-47cf-b82c-a515f719d8a5" }

I checked Google and FB logins and they don't display the Google/FB sub like the user pool entry does.

So my mobile app is storing the User Pool ID. This seems to be missing from amplify. If that data was in the identity pool this issue would be solved.

In Auth.ts there is this code: private setCredentialsFromFederation(provider, token, user) { const domains = { 'google': 'accounts.google.com', 'facebook': 'graph.facebook.com', 'amazon': 'www.amazon.com' };

I wonder if that string is fixed format? Maybe it is legal to say... 'google': 'accounts.google.com/34455444444',

ie add in the Google sub id? That appears to be what the mobile hub code did for Cognito.

[richardzcode]

Hi @baharev lookup user attributes by identityId is application specific. Some app may not want this for security concerns, some want to store in S3, and some prefer database. As a library we can't make those assumptions.

With aws-amplify it is pretty easy to implement. Depend on your needs, maybe pick one of the below two implementation.

Save private so only owner of the info can access

When user sign up, or maybe sign in depend on your choice,

  Auth.currentUserInfo()
    .then(info => {
      Storage.put(
        'userInfo.json',
        JSON.stringify(info),
        { level: 'private', contentType: 'application/json' }
      );
    });

Then do this to load user attributes,

  Storage.get('userInfo.json', { level: 'private',  download: true})
    .then(data => console.log('info...', data.Body.toString()));

Save public so just lookup by identityId

  Auth.currentUserInfo()
    .then(info => {
      Storage.put(
        identityId + '_userInfo.json',
        JSON.stringify(info),
        { level: 'public', contentType: 'application/json' }
      );
    });

Then do this to retrieve user attributes,

  Storage.get(identityId + '_userInfo.json', { level: 'public',  download: true})
      .then(data => console.log('info...', data.Body.toString()));

[baharev]

@richardzcode Thanks, your first suggestion is an acceptable workaround for the time being. What I don't like about it is that the userInfo.json in your code comes from the user, and therefore cannot be trusted.

lookup user attributes by identityId is application specific.

There is a misunderstanding here: I need this bidirectional map purely on the backend. The user must not be able to access it exactly for security reasons.

The use cases are the followings.

1. If a user is complaining about a bug, I have to be able to find his/her folder to look at the data and to reproduce the bug (if any). In this use case I know the user but I don't know which folder is his/her folder.
2. Say I noticed something strange in certain folders of the S3 bucket, and I would like to reach the corresponding users in e-mail and warn them. In this use case I know the folder but I don't know the users' e-mail address.

Note that both use cases must happen purely on the backend, and without any interaction from the user.

The true solution would be to retrieve this bidirectional map purely on the backend, without any user interaction. I think there should be an AWS API for backend code to do that. In my first comment I link to a horribly complicated "solution", but I find that unacceptably complex.

[baharev]

I looked at the Cognito API reference, and it is weird too. For example:

1. AdminGetUser does not seem to give me the identity ID.
2. DescribeIdentity does not seem give me any user attributes.
3. Somehow the mapping between users and identity IDs seem to be hidden. The only way I could recover it is through the login tokens.

What is going on here? Or what did I miss / misunderstand?

[richardzcode]

@baharev In my knowledge there is no backend direct mapping between identityId and username. Backend is depend on what services provide. We can only suggest client side solution at this moment.

[jonsmirl]

The old Mobile Hub code for Android built a two way map. This Indentity pool entry was created by that code when I did a user pool login:

jonsmirl@ubuntu-16:~$ aws cognito-identity describe-identity --identity-id us-east-1:bea3f57c-4564-47cf-b82c-a515f719d8a5 --profile bill { "Logins": [ "cognito-idp.us-east-1.amazonaws.com/us-east-1_Cf6AJpe20" ], "LastModifiedDate": 1512005509.277, "CreationDate": 1512005509.237, "IdentityId": "us-east-1:bea3f57c-4564-47cf-b82c-a515f719d8a5" }

Also, if you use the current hosted UI under user pool it creates a back link like above. It is only Amplify that is not creating the back link.

I just used Amplify to create an entry in my identity pool corresponding to a userpool entry. The entry in the identity pool does not have the linked login info shown above.

jonsmirl@ubuntu-16:~/aosp/demo/foobar/client$ aws cognito-identity describe-identity --identity-id us-east-1:f763ea29-41ce-4b86-bc58-31de68d7cce8 { "LastModifiedDate": 1517513923.798, "CreationDate": 1517513923.798, "IdentityId": "us-east-1:f763ea29-41ce-4b86-bc58-31de68d7cce8" }

[baharev]

@richardzcode This map must exist on the backend, because a given user always gets the same ${cognito-identity.amazonaws.com:sub}. It must be solvable purely on the backend.

[baharev]

@jonsmirl Sorry, I don't understand, but it seems to me that you are also struggling with this issue, or with a very similar one.

[jonsmirl]

When amplify creates an entry in identity pool corresponding to a user pool entry, why is this part missing? Other Amazon Cognito code makes this entry:

"Logins": [ "cognito-idp.us-east-1.amazonaws.com/us-east-1_Cf6AJpe20" ],

That entry lets you map from an identity pool ID back into a user pool ID. You can query the logins off from an identity pool entry to get that string. Then use that string to query all of the attributes for the user out of the user pool.

[baharev]

@jonsmirl OK, now we are getting somewhere. Please explain:

Then use that string to query all of the attributes for the user out of the user pool.

How? Which API call will consume this mysterious "cognito-idp.us-east-1.amazonaws.com/us-east-1_Cf6AJpe20" and give me the user attributes?

[jonsmirl]

We don't use this feature, and I see now that I was misunderstanding what I was seeing. This is the Cognito user pool id, not the sub of the user: cognito-idp.us-east-1.amazonaws.com/us-east-1_Cf6AJpe20 I was thinking that last part was the sub of the user in the pool and it is not.

You would need to be able to access the token that was submitted with this identity and I don't see any obvious way to get to it. 99% of our logins are via Google/FB with only a few using User Pool.

So to make this work, after you get logged in you will need a dynamodb table that maps the cognito identify id to the cognito user id. Then you will be able to find the user in the user pool.

Or you will just end up doing what we did. Since the attributes on Google, FB and user pool are all different we just store a copy of the attributes we care about in our internal user database. Which is what richard said to do.

This might be what you are looking for:

https://stackoverflow.com/questions/42386180/aws-lambda-api-gateway-with-cognito-how-to-use-identityid-to-access-and-update

[baharev]

@jonsmirl Thanks for the prompt reply.

As I said, yes, it would be an acceptable workaround for the time being, but it requires a user login and the info comes from the user (hence cannot be trusted). What pisses me is that this map must be available in the backend, so I see no reason why the user pool owner cannot access it. It just does not make sense to me, and I am wondering why the others aren't complaining about it too.

This might be what you are looking for:

https://stackoverflow.com/questions/42386180/aws-lambda-api-gateway-with-cognito-how-to-use-identityid-to-access-and-update

That is the link in my very first comment. :) I know you can do it, but it is unacceptably complex.

Thanks again for the prompt feedback!

[WolfWalter]

Totally agreeing with @baharev. In my use case I have stored the Cognito Identity IDs in my database and now tried to get the username of the associated userpool user. None of the APIs seems to expose this functionality.

[ffxsam]

I was advised to retrieve the Cognito identity ID via Auth.currentUserInfo() and store it as an attribute in the user object in the user pool.

[ffxsam]

However, I can't figure out when the right time is to grab this info. Upon login, when I call Auth.currentUserInfo(), the id property is undefined.

  async signIn({ commit }, { username, password }) {
    const user = await Auth.signIn(username, password);
    const userInfo = await Auth.currentUserInfo();
    console.log(userInfo);
    authenticate(commit, user);
  },

[ffxsam]

Ok, this works for me:

  async signIn({ commit }, { username, password }) {
    const user = await Auth.signIn(username, password);
    const credentials = await Auth.currentCredentials();
    console.log('Cognito identity ID:', credentials.identityId);
    authenticate(commit, user);
  },

So within this code block, you could just update the user's attributes and set their Cognito identity ID in there, and you'd have immediate access to it once they're authenticated.

[baharev]

@ffxsam Interesting, thanks for letting us know. I recognize Auth.signIn() and Auth.currentCredentials() but could you explain what authenticate is in your code?

If I understand correctly what you are suggesting, it is essentially the same as richardzcode's suggestion. See my previous objections why I am not happy with it as a solution. It is an acceptable workaround for the time being though.

[ffxsam]

authenticate isn't relevant here, just my own function that commits data to a Vuex store.

[gbrits]

This is how I overcame the obstacle of not having the identityId on hand for my users:

login() {
    let loading = this.loadingCtrl.create({
      content: 'Please wait...'
    });
    loading.present();

    let details = this.loginDetails;
    Auth.signIn(details.username, details.password)
      .then(user => {
        Auth.currentCredentials().then(credentials => {
          var identityId = credentials.identityId;
          let params = {
             "AccessToken": user.signInUserSession.accessToken.jwtToken,
             "UserAttributes": [
                {
                   "Name": "custom:identityId",
                   "Value": identityId
                }
             ]
          }
          // Save the identityId custom attribute to the user
          this.db.getCognitoClient()
          .then((client) => {
            client.updateUserAttributes(params, function(err, data) {
                if (err) console.log(err, err.stack);// an error occurred
                else {
                  console.log(data);
                }
            });
          });
        });
        logger.debug('signed in user', user);

        if (user.challengeName === 'SMS_MFA') {
          this.navCtrl.push(ConfirmSignInPage, { 'user': user });
        } else {
          this.navCtrl.setRoot(TabsPage);
        }
      })
      .catch(err => {
        logger.debug('errrror', err);
        this.error = err;
      })
      .then(() => loading.dismiss());
  }

Ref: inside of my DynamoDB (public db) class, my getCognitoClient method invokes the CognitoIdentityServiceProvider endpoint from the aws-sdk

getCognitoClient() {
      return Auth.currentCredentials()
        .then(credentials => new AWS.CognitoIdentityServiceProvider({ credentials: credentials }))
        .catch(err => logger.debug('error getting document client', err));
  }

Important Note You have to log into Cognito's User Pool and click Attributes and add IdentityId (as a string) to your custom attributes, for it to be populated.

Hope this helps someone. Because it had me thinking I'm an idiot who should stop programming, so hopefully someone out there can reassure me to continue!

[baharev]

@gbrits I think that we should get official support through the SDK, and we really should not be implementing our workarounds. Especially not in ways that involve data coming from the user (untrusted data).

As far as I understand your code, it has the same issues as richardzcode's workaround.

[ffxsam]

Totally agree. The SDK should handle these sorts of lower level operations for us.

[keithdmoore]

I agree this is extremely complicated. Why can't the Cognito user attributes just be passed in through the request. Even digging out the sub is a pain in the rear.

Accessing this attribute: req.apiGateway.event.requestContext.identity.cognitoAuthenticationProvider yields this as a result cognito-idp.us-east-1.amazonaws.com/us-east-1_blah,cognito-idp.us-east-1.amazonaws.com/us-east-1_blah:CognitoSignIn:XXXXXXX-XXXXX-XXXX-XXX-XXXXXX

Really? I have to parse that to get the 'subject'? I would really like the username and email address but in order to do that I have to change the authorizer on the api gateway to use Cognito instead of IAM and then somehow get the API class methods to provide the token in such a way that it can be used. This is crazy!

[mlabieniec]

Hi @keithdmoore there are a number of ways you can simplify this, but we are also looking into how we can do this on both the library and service side. Until then, the api gateway body mapping templates can help you pass/organize things on the lambda context, and as far as the sub, this is available in the jwt token, there is a pretty indepth blog post on some of this here: https://aws.amazon.com/blogs/compute/secure-api-access-with-amazon-cognito-federated-identities-amazon-cognito-user-pools-and-amazon-api-gateway/

Also, on the client side, you can retrieve the sub from Amplify Auth with:

let session = await Auth.currentSession()
// session will be a json obj with your tokens

{
    "idToken": {
        "jwtToken": "...", 
        "payload": {
                "sub": "...", // <-- here is your subject
                "email_verified": false,
                "iss": "https://cognito-idp.us-east-1.amazonaws.com/us-east-1_xxx",
                "phone_number_verified": true,
                "cognito:username": "uname",
                "aud": "...",
                "event_id": "...",
                "token_use": "id",
                "auth_time": 1525294669,
                "phone_number": "...",
                "exp": 1525551200,
                "iat": 1525547600,
                "email": "..."
        },
        // as well as other tokens and payloads
        "refreshToken": {}, "accessToken": {}
    }
}

[keithdmoore]

@mlabieniec Thanks for the info. However, I am using a LAMBDA_PROXY to route to Express so I don't think I can modify the request body mappings. Also, passing those parameters from the client is not secure. What I really wanted to do was to retrieve dynamoDB entries for the currently logged in user's username. As a workaround, I guess I will just use the sub instead and parse it out of the the provider attribute and run my dynamoDB query using the DocumentClient.

[shagabay]

I also struggled with this weirdly lacking feature (seems so basic ha?)

My eventual solution is to use a DynamoDB table in order to store user information (but not password, of course). I had a feeling i shouldn’t trust Cognito to easily retrieve that information upon request.. apparently I was right. Since those attributes cannot change (I.e. phone number), there׳s no issue with syncing. However, I chose to create a trigger on Cognito that will update db upon use signup to reduce risk of problems.

The more important part is to use COGNITO_USER_POOLS as my authorizer on apig. All you need is to pass along a token in the headers, and you’ll have the user name available in Lambda. From there you can lookup the db and voila.

[christopherlwilliamsm]

What brings me here is the fact that the Amplify documentation for storage gives instructions for accessing other users' files by means of the identityId but does not provide the means for obtaining said identityId or linking it to a userpool identity. The documentation says the following:

To get other users’ objects

Storage.list('photos/', { 
    level: 'protected', 
    identityId: 'xxxxxxx' // the identityId of that user
})
.then(result => console.log(result))
.catch(err => console.log(err));

[powerful23]

Guys confirmed from Cognito service team that unfortunately this feature is not supported.

[baharev]

@powerful23 OK, where or how I can submit a feature request?

People need this feature.

[yuntuowang]

I will create a feature request on your behalf. @baharev

[baharev]

@powerful23 I understand that there is nothing the Amplify team can do about it, but please tell me how I can ask for this feature from the Cognito team.

[baharev]

@yuntuowang Perfect, thank you!

[vahdet]

This identity ID is also referred for getting other users’ objects in the docs here like:

Storage.get('test.txt', { 
    level: 'protected', 
    identityId: 'xxxxxxx' // the identityId of that user
})
.then(result => console.log(result))
.catch(err => console.log(err));

I tried to get that the identityId of that user via a post confirmation lambda trigger to store in DynamoDB, but no chance.

Even if it was returned by Storage API after a put, it would make everything easier. But it is not available neither.

[thrixton]

Hi @yuntuowang, is there a feature request for this publicly accessible that we can track?

[jonmifsud]

@yuntuowang the above is a key flaw which essentially makes S3 Storage nearly unusable for anyone with proper application structures; unless you do some dirty work-arounds.

The cognito event trigger on new user registration does not supply the IdentityId (so it cannot be placed in say DynamoDB for access) and the only way to obtain the IdentityId is when a user is logged in having to re-submit a separate request to update the information.

[baharev]

@yuntuowang Any news on the progress?

[abirtley]

@yuntuowang any updates?

[hectomg]

Any updates??? I have this problem also and as @baharev has said before:

As I said, yes, it would be an acceptable workaround for the time being, but it requires a user login and the info comes from the user (hence cannot be trusted). What pisses me is that this map must be available in the backend, so I see no reason why the user pool owner cannot access it. It just does not make sense to me, and I am wondering why the others aren't complaining about it too.

Totally identified with that!

[TomiLahtinen]

Having this issue too. Everything about the possible workarounds has been said already so I'll just leave it at that.

[baharev]

@yuntuowang This issue the second most commented open issue, with 19 participants. Could you give it a higher priority, please?

[behrooziAWS]

We don't comment on the status or priority of feature requests. I'll suggest one more approach which should address the security concerns expressed above.

1. Add a custom attribute to your user pools schema called identityId
2. Make it read-only to the client that your end users are authenticating with
3. Authenticate the end user
4. Get AWS credentials for the end user
5. If there is no identity id present in the user profile, call a Lambda using the AWS credentials from step 4 and provide the id token from the end user as a parameter.
6. In the lambda call Cognito Federated Identity's GetId and pass the id token for the end user in the logins map, this will return the identity id
7. In the lambda call AdminUpdateUser to set the IdentityId profile attribute to the value returned in step 6
8. Optionally refresh the end users' user pool tokens so they will have the identity id in them

Since the attribute cannot be set by the end user and there are no assumptions about what the end user provides the Lambda this should be secure.

[ffxsam]

That seems kind of excessive. Why not just update the user's attributes in the user pool all in the client side, without the Lambda calls?

      const user = await Auth.signIn(username, password);
      const credentials = await Auth.currentCredentials();

      await Auth.updateUserAttributes(user, {
        'custom:identity_id': credentials.identityId,
      });

[baharev]

@behrooziAWS Thanks, but it is still a workaround at best. Far from simple, requires the user to login, I have to set up yet another service (Lambda), implement the workaround, test it, etc.

[baharev]

@behrooziAWS I am not sure your approach is safe: I think you do assume that the user will pass in his/her own id token. As a malicious user, I could create two users, and pass in the valid id token of the other user, messing things up on the backend. I do not see how your workaround would prevent this.

[behrooziAWS]

@baharev If the malicious user passes a valid id token for the other user, because we are getting the identity by using the token they passed in, and the username from the token they passed in, it will just set the identity id for the other user correctly in the other users' profile.

[lestephane]

This dichotomy between user pool and identity pool accounts for the most confusing part of the aws-api-gateway-developer-portal codebase.

export function init() {
  // attempt to refresh credentials from active session
  userPool = new CognitoUserPool(poolData)
  cognitoUser = userPool.getCurrentUser()

  if (cognitoUser !== null) {
    cognitoUser.getSession(function(err, session) {
      if (err) {
        logout()
        console.error(err)
        return
      }

      const cognitoLoginKey = getCognitoLoginKey()
      const Logins = {}
      Logins[cognitoLoginKey] = session.getIdToken().getJwtToken()
      AWS.config.credentials = new AWS.CognitoIdentityCredentials({
        IdentityPoolId: cognitoIdentityPoolId,
        Logins: Logins
      })

      AWS.config.credentials.refresh((error) => {
        if (error) {
          logout()
          console.error(error)
        } else {
          initApiGatewayClient(AWS.config.credentials)
          updateAllUserData()
        }
      })
    })
  } else {
    initApiGatewayClient()
  }
}

Wot?

I've for now recommended my client to stay away from a userpool + identitypool solution, and to have a userpool only.

@behrooziAWS does that seem like a good workaround for this problem? Since I don't need federated web identities, and I don't need the single-page-app client to ever assume an iam role using the identity pool mechanism, that seems OK.

[baharev]

@lestephane Interesting. If your client app never has to assume an IAM role then why are you using Cognito at all? The sole reason why I am using Cognito is that my client app has to assume an IAM role.

I've for now recommended my client to stay away from a userpool + identitypool solution, and to have a userpool only.

I wonder how you get away without the identitypool if you need IAM role for the client.

[baharev]

@behrooziAWS Maybe it is "safe" but I still don't like this workaround. It somehow reminds me to Meltdown and Spectre. And as I mentioned, this workaround is unnecessarily complicated.

[lestephane]

@baharev I use this because a user registration and login system is not my core competency. Why would I contemplate implementing one of my own? My client app invokes an endpoint that verifies the JWT token issued by Cognito. And then, based on that, it returns an apiKey, which the client can use inside of an x-api-key header. It means i don't overly rely on AWS's sdk, and keep things simple. Is there any flaw in my logic? I'm happy to stand corrected...

[baharev]

@lestephane There is some misunderstanding: I would like to understand your use case. That is all. I did not say that your approach was wrong.

If I understand correctly: If you do not use the identity pool part of Cognito, you can only check whether the user is logged in or not. And apparently that is enough for your use case. (?)

It is not clear to me how you make sure that the user can save his/her files on S3, and only he/she can access them. If your use case involves something like that, you must be re-implemeting something that IAM would give you practically for free. Or I am missing something obvious here...

In any case, I would like to know more about your use case. Thanks!

[lestephane]

If I understand correctly: If you do not use the identity pool part of Cognito, you can only check whether the user is logged in or not. And apparently that is enough for your use case. (?)

Correct. I don't understand what identity pools would bring me (yet). And translating one id from user pool subject id to identity pool id is apparently nigh impossible.

It is not clear to me how you make sure that the user can save his/her files on S3, and only he/she can access them

There is no need for us (yet) to invoke any aws api from the client single page app using an assumed role. We just want to offload the user registration, login, storage of usernames / emails and passwords to aws. Not our core business. We have a developer api portal (see the aws-api-gateway-developer-portal repo) where we manage api keys for our logged in users. The api key will decide whether they can invoke an our developer api (an apigateway endpoint) or not.

However, I will grant you that, if designed properly, the identity pool can give you some amazing capabilities, like for example restricting the right of the logged in user to read and write only DynamoDB entries that he / she owns. But, this is so hard to get right, that you end up with a role that is more permissive than it should be. In fact, the default policies you see when looking for CloudFormation template samples use way too many wildcards ('*'). And if you use a permissive policy, you end up negating the benefits of using roles, and therefore using identity pools.

So our use case in short:

• Our developer api portal frontend (a single-page-app) uses amplify-js to call out to cognito for registration and login steps.
• Cognito return an id token, which our portal frontend transmits to our portal backend api.
• The portal backend api verifies the jwt token using the aws jwks, then uses the subid to look up the apiKey we've got on record for that logged-in user in a dynamodb table.
• The api key is sent back to the portal frontend, and displayed to the end-user (a developer).
• The developer then uses that apiKey when invoking our developer API (using an x-api-key header). The apiKey is associated with a usage plan in AWS API Gateway. It's a black or white situation, either the user can invoke the developer api or not.

I don't understand what an identity pool would bring, especially not in a developer api portal.