aws identitystore list-users for all users

[iainelder]

Is your feature request related to a problem? Please describe.

I can't figure out how to get a list of all the SSO users using the API.

The documentation for the list-users command is too generic to be useful.

The documentation for the ListUsers API is slightly more specific but still vague.

Lists the attribute name and value of the user that you specified in the search. We only support UserName as a valid filter attribute path currently, and filter is required.

Which filter do I need to provide to match all users? Does such a filter exist?

Describe the solution you'd like

I've tried using * as a a wildcard with no success.

aws identitystore list-users \
--identity-store-id d-12354567890 \
--filters AttributePath=UserName,AttributeValue="*"

Ideally the match-everything filter should be the default option, like the aws ec2 describe-instances API call.

aws identitystore list-users \
--identity-store-id d-12354567890 \

Describe alternatives you've considered

I suppose it would be possible to scrape this info from the AWS SSO web console, but that really shouldn't be the only solution.

[es-arif]

If you don't want to click through page by page with JavaScript, you can pull every user as JSON in a few of clicks.

1. Go to Developer Tools -> Network (or Ctrl+Shift+E)
2. Refresh page
3. Click on "file"/request that says identitystore
4. Click tab that says "Response"
5. Right click on "Users" JSON object and;
6. manipulate that to your hearts content.

Just as an example, I ran it through jq to get all users and status:

$ cat json.json| jq '.Users[]|{"email": .UserName,"active": .Active}'
....
{
  "email": "redacted@redacted.com",
  "active": false
}
{
  "email": "redacted@redacted.com",
  "active": true
}
{
  "email": "redacted@redacted.com",
  "active": true
}

[andrisro]

+1

[peraltadavidtrvlx]

Actually, this api already there, it's just not documented and implemented as part of boto3:

{"method":"POST","path":"/identitystore/","headers":{"Content-Type":"application/json; charset=UTF-8","Content-Encoding":"amz-1.0","X-Amz-Target":"com.amazonaws.identitystore.AWSIdentityStoreService.SearchGroups","X-Amz-Date":"Mon, 07 Feb 2022 16:45:51 GMT","Accept":"application/json, text/javascript, */*"},"region":"eu-west-1","operation":"SearchGroups","contentString":"{\"IdentityStoreId\":\"d-<id>\",\"NextToken\":null}"}

And response:

{
    "Groups": [
        {
            "DisplayName": "<name>",
            "GroupAttributes": {
                "description": {
                    "StringValue": "<value>"
                }
            },
            "GroupId": "<guid>",
            "Meta": {
                "..."
            }
        },
        {
            "DisplayName": "<name>",
            "GroupAttributes": {
                "description": {
                    "StringValue": "<text>"
                }
            },
            "GroupId": "<...>",
            "Meta": {
                "..."
            }
        }
    ],
    "TotalGroupCount": 2
}

It does work, but the documentation describes a limitation that may block production use, depends on your use case.

You can use the /Groups endpoint to filter queries on a list of existing groups by making a GET request with additional filter information. Only a maximum of 50 results can be returned. See the Constraints section for a list of available filters.

Source: https://docs.aws.amazon.com/singlesignon/latest/developerguide/listgroups.html.

How do you run this in python tho?

[malaval]

If that can help, here is a Javascript console script I made (and tested in Chrome) to get the list of users, groups and group attachments from the console. Native AWS SSO API would be much easier though...

I have something similar to parse the Applications page if needed.

/* Before running this script:
1. Go to https://eu-west-1.console.aws.amazon.com/singlesignon/identity/home?region=eu-west-1#!/users (change the region in all the code)
2. Choose to display the columns in the User page:
  - Username
  - Display name
  - Status
  - MFA devices
  - First name
  - Last name
  - Created by
  - Created
*/

const waitBetweenPages = 1500;
const listGroups = [];
const listUserIds = [];
const listUsers = [];
const listUserGroups = [];

function parseId(innerHtml) {
  return innerHtml
    .match('(/groupDetails/[^"]*|/userDetails/[^&]*)')[0]
    .split("/")[2]
    .replace("userId=", "");
}

function processUsersPage() {
  console.log('Processing users page')

  // Next button in the navigation pane
  var nextButtonPane = document.querySelector('*[class^="awsui_tools-pagination"]');
  var nextButtonLIs = nextButtonPane.getElementsByTagName('ul')[0].getElementsByTagName('li');
  var nextButton = nextButtonLIs[nextButtonLIs.length-1].getElementsByTagName('button')[0];

  // Table that lists the users
  var usersTable = document.getElementsByTagName("table")[0];

  // For each line of the table, except the header
  for (let i = 1; i < usersTable.rows.length; i++) {

    // Parse the table content to get the user ID, the user name and the MFA devices
    var userId = parseId(usersTable.rows[i].innerHTML);
    var userName = usersTable.rows[i].getElementsByTagName('td')[1].innerText;
    var userEnabled = usersTable.rows[i].getElementsByTagName('td')[3].innerText;
    var userMfa = usersTable.rows[i].getElementsByTagName('td')[4].innerText;
    var userFirstName = usersTable.rows[i].getElementsByTagName('td')[5].innerText;
    var userLastName = usersTable.rows[i].getElementsByTagName('td')[6].innerText;
    var userCreated = usersTable.rows[i].getElementsByTagName('td')[8].innerText;
    listUserIds.push(`${userId}`);
    listUsers.push(`  "${userId};${userName};${userEnabled};${userMfa};${userFirstName};${userLastName};${userCreated}",`);

  }

  // Click the next button if more pages are available
  // Or continue and retrieve the user groups
  if (nextButton.disabled) {
    goToUserPage(0);
  }
  else {
    nextButton.click();
    setTimeout(
      function() {
        processUsersPage();
      }, waitBetweenPages);
  }
}

function goToUserPage(userIndex) {
  console.log(`Processing user ${userIndex} / ${listUserIds.length} page`)

  var userId = listUserIds[userIndex];
  var userGroupUrl = "https://eu-west-1.console.aws.amazon.com/singlesignon/identity/home?region=eu-west-1#!/users/users/userDetails/userId={id}&directoryId=&realm=/groups"
  window.location.href = userGroupUrl.replace('{id}', userId);
  setTimeout(
    function(){
      processUserPage(userIndex);
    },waitBetweenPages);

}

function processUserPage(userIndex) {
  var userId = listUserIds[userIndex];

  // Table that lists the user's groups
  var groupsTable = document.getElementsByTagName("table")[0];

  // For each line of the table, except the header
  for (let i = 1; i < groupsTable.rows.length; i++) {

    // Check that it is the row has more than one column
    // Otherwise it might be a "No groups" row
    var rowTDs = document.getElementsByTagName("table")[0].rows[1].getElementsByTagName('td');
    if (rowTDs.length > 1) {

      // Parse the table content to get the group ID and the group name
      var groupId = parseId(groupsTable.rows[i].innerHTML);
      var groupName = groupsTable.rows[i].getElementsByTagName('td')[1].innerText;
      listUserGroups.push(`  "${userId};${groupId};${groupName}",`);

    }

  }

  if (userIndex+1 < listUserIds.length) {
    goToUserPage(userIndex+1);
  }
  else {
    goToGroupsPage();
  }
}

function goToGroupsPage() {
  console.log('Redirecting to groups page')

  window.location.href = "https://eu-west-1.console.aws.amazon.com/singlesignon/identity/home?region=eu-west-1#!/groups";
  setTimeout(
    function(){
      processGroupsPage();
    },
    waitBetweenPages);

}

function processGroupsPage() {
  console.log('Processing groups page')

  // Next button in the navigation pane
  var nextButtonPane = document.querySelector('*[class^="awsui_tools-pagination"]');
  var nextButtonLIs = nextButtonPane.getElementsByTagName('ul')[0].getElementsByTagName('li');
  var nextButton = nextButtonLIs[nextButtonLIs.length-1].getElementsByTagName('button')[0];

  // Table that lists the groups
  var groupsTable = document.getElementsByTagName("table")[0];

  // For each line of the table, except the header
  for (let i = 1; i < groupsTable.rows.length; i++) {

    // Parse the table content to get the group ID and the group name
    var groupId = parseId(groupsTable.rows[i].innerHTML);
    var groupName = groupsTable.rows[i].getElementsByTagName('td')[1].innerText;
    listGroups.push(`  "${groupId};${groupName}",`);

  }

  // Click the next button if more pages are available
  // Or display the results
  if (nextButton.disabled) {
    printResult();
  }
  else {
    nextButton.click();
    setTimeout(
      function() {
        processGroupsPage();
      },
      waitBetweenPages);
  }
}

function printResult() {
  console.log(
    '{'
    + '\n"Groups":[\n' + listGroups.join("\n").slice(0, -1)+'\n],'
    + '\n"Users":[\n'+listUsers.join("\n").slice(0, -1)+'\n],'
    + '\n"UserGroups":[\n'+listUserGroups.join("\n").slice(0, -1)+'\n]'
    + '}' );
}

processUsersPage()

[peraltadavidtrvlx]

Try using SCIM API endpoint. https://docs.aws.amazon.com/singlesignon/latest/developerguide/listgroups.html

[demetriusmoro]

Try using SCIM API endpoint. https://docs.aws.amazon.com/singlesignon/latest/developerguide/listgroups.html

The doc itself says there are limitations:

• You can use the /Groups endpoint to filter queries on a list of existing groups by making a GET request with additional filter information. Only a maximum of 50 results can be returned. See the Constraints section for a list of available filters.
• GetGroup and ListGroups return an empty member list. To see group info for a certain member, call ListGroups with a member filter. (See the examples that follow.)

[liebman]

I created a fork where I added a "datastore" to keep a list of users/group in a local file, consul, or S3. https://github.com/liebman/ssosync/tree/patched-with-pr-55-58-47-45-and-datastore (a few additional patches that we needed are also applied) Its not perfect but it is functional.

[asharafzwift]

This is a real limitation with managing users in aws sso. Is there going to be any movement on this in the near future?

[tgmatt]

Another user here suffering without this API. Quite astonished it's not available.

[dustintodd123]

Written because I needed it. Solved my reporting problem, it's not pretty. But it works. https://github.com/dustintodd123/aws-ssoreporting To run this you need:

1. A file with every email address in your directory. The CSV needs a header row. Put the email address column name in --colname parameter.
2. AWS SSO SCIM URL stored in env variable URL. Found in the AWS SSO console.
3. AWS SSO SCIM auth token stored in env variable SCIMTOKEN. Found in the AWS SSO console.

python aws-ssoreporting.py --infile= --outfile= --colname=

Output rpt file is one row for every group a user is a member of email,firstname,lastname,groupname

[rmanny]

This isn't a feature request, this is a bug. This is just broken functionality - the list-users and list-groups functionality just do not work. We're now about 13 months after the original report. Why is this so hard to implement? The console can certainly get this data, so it's clearly talking to the appropriate backing store directory services.

You'd be better off just removing the functions so developers don't keep trying to make it work like it should and wasting time on it.

+1 on this. PLEASE remove list_groups from the API docs, or at least add a huge warning. Right now its only purpose is to cause confusion. Just wasted a few hours on this, not believing it could possibly be designed like this. Until the filters can match on something besides an exact string, its a copy of describe_group with syntax that makes no sense

[devopssankar]

Yes, It is a bug. More than a year now AWS Service Team didn't care to fix it.

[demetriusmoro]

For eveyrone still waiting, with hope, remember to 👍🏻 on the first comment. It helps this issue's priority get higher.

[devopssankar]

@demetriusmoro It is a Myth. Not everyone follows to a provide priority based on how many users wanted the feature. I don't think AWS Team care about this - They will have their own development agenda. Look at recent feature request PRs you know

[dtg123nayya]

I think this capability is intentionally omitted , but not sure why. Being able to report out the data via SCIM or AWS API appears to specifically prevented. The only way was able to meet compliance reporting needs is to take every email address from my IdP and query each one individually. Doesn't look like an accident that it was built this way.

[devopssankar]

@dtg123nayya May be. I'm able to list individual users (or) group but I couldn't find anything to link between user & group. I'm looking for something that gives me [a] if I describe a group, it should give a list of users who member of that group (OR) [b] if I describe a user, it should give a list of groups that user part of.

[dustintodd123]

I used the SCIM API in this project: aws-ssoreporting.py and for every group in the system, you can query the group API with the group id and user id as the filter. If nothing returns the user is not a member, if it returns data the user is a member. This is ugly because you have to test each user against every group. But it's accepted by audit. Also, I had to start with a list of every possible email address to query each one individually. Which is also ridiculous.

[devopssankar]

@dustintodd123 Thank you very much for the script.

[WanderSurfer]

+1 to this request, I don't really want to use some janky scripts/incorporate some limited API queries to accomplish what should be a basic functionality for SSO and provided by the CLI and Boto3. Not sure why AWS is dragging their feet so badly. Azure handles this fine.

[tim-finnigan]

Hi all, thanks for your patience. I reached out to the SSO team for an update and they responded:

We are currently working on launching an updated version of the AWS SSO User and Group Management APIs, which will enable you to query all users/groups in AWS SSO via the API. Please reach out to AWS Support or your account team to track the AWS SSO Product Feature Request for more details on timelines.

[sasuperdev]

Absolutely ridiculous this is a basic operation. More then a year later and still no update. Why are they making list users so difficult jump through a dozen hoops, fake limits ?

[AmitArie]

any news?

[brendan-snyk]

Any news indeed? This is dumb.

[megaproaktiv]

A new API is published: https://docs.aws.amazon.com/singlesignon/latest/IdentityStoreAPIReference/API_ListUsers.html This should fix it?!

[iainelder]

Well spotted, @megaproaktiv !

I can now call list-users without the --filter parameter.

aws identitystore list-users \
--identity-store-id d-1234567890

{
    "Users": [
        {
            "UserName": "user1",
            "UserId": "..."
        },
        {
            "UserName": "user2",
            "UserId": "..."
        }
    ]
}

Today the ListUsers API reference says of Filters that this "parameter has been deprecated."

It works for me using aws-cli version 2.7.28. The changelog mentions documentation updates to the identitystore service.

  {
    "category": "``identitystore``",
    "description": "Documentation updates for the Identity Store CLI Reference.",
    "type": "api-change"
  },

@tim-finnigan , please pass on my thanks to the service team.

[LiranV]

I was able to run the following to fetch all users:

aws identitystore list-users --identity-store-id="<ID>"

This wasn't possible before (running without --filters)

My issue now is that the only fields I get back are UserName and UserId, even when using describe-user instead of list-users (AWS CLI), any solution?

[mustafaakin]

Yes, the describe-user response is not like the API reference at all: https://docs.aws.amazon.com/singlesignon/latest/IdentityStoreAPIReference/API_DescribeUser.html

But glad it finally exists.

[iainelder]

My issue now is that the only fields I get back are UserName and UserId, even when using describe-user instead of list-users (AWS CLI), any solution?

@LiranV , @mustafaakin , I suppose you can create a new issue in this repository to ask for improvements to the describe-user command.

The feature I requested in this issue is now implemented, so I'm going to close it.

[github-actions[bot]]

This issue is now closed.

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

[mrbitsdcf]

I'm reopening this discussion because it seems that the "Filter" issue was not completely solved. We can run

aws identitystore list-users --identity-store-id="<ID>"

or

aws identitystore list-groups --identity-store-id="<ID>"

without the --filter parameter IF our IdentityStore is the IAM Identity Center default one. If I'm using AWS Managed AD as Identity Provider, AWS CLI still asks for --filter.

In addition, for list-users action, AWS CLI is asking for a parameter User@Domain and it only works with the Domain created in AWS Managed AD. In my use case here I have UserNames being users' e-mails, so we have user@lots_of_domains.

Is it possible to change AWS CLI to avoid to ask for --filter if Identity Provider is not the SSO default?

Thanks a lot.

[iainelder]

@mrbitsdcf , you'll get better feedback from the SDK team if you open a new issue that references this one.

[femto113]

sorry for out of topic, how to get --identity-store-id in cli? currently I take from http browser request

Replying here even though off-topic because this is where Google landed me when searching for exactly this question. The identity-store-id can be retrieved from the CLI via aws sso-admin list-instances

[ksmnv]

This was not addressed for the AD Connector type.