get google users googleapi: Error 400: Invalid Input, invalid

[ricpruss]

Describe the bug Running ssosync fails with an Error 400: Invalid Input on get google users.

To Reproduce Set up ssosync according to the instructions. while exporting the environment variables for AWS stuff It seemed to be short an identity store number in the instructions so I set that SSOSYNC_IDENTITY_STORE_ID but the run seems to work for the test but dies when getting the users

Additional context Add any other context about the problem here. INFO[0000] Syncing AWS users and groups from Google Workspace SAML Application INFO[0001] Test call for groups successful Groups="{\n Groups: [{\n DisplayName: ,\n GroupId: \"REDACTED\",\n IdentityStoreId: \"d-REDACTED\"\n }]\n}" INFO[0001] syncing sync_method=groups INFO[0001] get google groups queryGroup="*" INFO[0001] get google users queryUsers= DEBU[0001] preparing list of google users, groups and their members DEBU[0001] Fetching ALL users from google, to use as cache FATA[0001] googleapi: Error 400: Invalid Input, invalid

[ChrisPates]

A 400 error from the Google api indicates the query string (in this case groupMatch I think) is invalid. Take a look at the examples on the ReadMe. The simplest is '*' which matchs all groups.

Chris

[tyrannosaurus-becks]

Thanks for the suggestion! I am also encountering this error. I've just tried the advice above and am still getting the same error.

Steps to reproduce

In Github Actions, this is the job:

  sync_google_and_aws_groups:
    name: Sync Google and AWS groups
    continue-on-error: true
    runs-on: ubuntu-20.04
    env:
      AWS_REGION: 'us-east-1'

      # This secret maps to the service account entitled "Platform repo, Google & AWS group sync"
      # in the "Administration" project in Google Cloud.
      GOOGLE_CREDENTIALS_JSON_FILE_BODY: ${{ secrets.GOOGLE_CREDENTIALS_JSON_FILE_BODY }}

      # This is the ID of the identity store inside AWS. Always will start with 'd-' and is
      # shown in their UI.
      IDENTITY_STORE_ID: ${{ vars.IDENTITY_STORE_ID }}

      # Irritatingly, this access token expires annually inside the AWS IAM Identity Center
      # UI, and that will cause this script to break. We will simply need to create
      # a new access token and place it in this secret.
      SSOSYNC_SCIM_ACCESS_TOKEN: ${{ secrets.SSOSYNC_SCIM_ACCESS_TOKEN }}

      # This is static and non-secret.
      SSOSYNC_SCIM_ENDPOINT: ${{ vars.SSOSYNC_SCIM_ENDPOINT }}
    steps:
      - uses: actions/checkout@v4
        with:
          repository: 'awslabs/ssosync'
          ref: '6cb78e1225f5193ea996b215d1eca378046701dd' # Version 2.2.5.
      - name: Configure AWS
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID_STAGING }}
          aws-secret-access-key: ${{ secrets.AWS_ACCESS_KEY_SECRET_STAGING }}
          aws-region: ${{ env.AWS_REGION }}
      - name: Configure Google Oauth
        uses: 'google-github-actions/auth@v2'
        with:
          service_account: ${{ vars.GOOGLE_ADMIN }}
          credentials_json: ${{ env.GOOGLE_CREDENTIALS_JSON_FILE_BODY }}
          token_format: 'access_token'
          access_token_scopes: 'https://www.googleapis.com/auth/admin.directory.user.readonly'
      - name: Sync Google groups into AWS
        run: |
          echo '${{ env.GOOGLE_CREDENTIALS_JSON_FILE_BODY }}' > credentials.json
          make go-build
          ./ssosync \
            --access-token ${{ env.SSOSYNC_SCIM_ACCESS_TOKEN }} \
            --endpoint ${{ env.SSOSYNC_SCIM_ENDPOINT }} \
            --region ${{ env.AWS_REGION }} \
            --identity-store-id ${{ env.IDENTITY_STORE_ID }} \
            --google-admin ${{ vars.GOOGLE_ADMIN }} \
            --google-credentials credentials.json \
            --user-match '*' \
            --debug

Logs

Here's the output I receive from the Sync Google Groups into AWS step:

go build -o ssosync main.go
go: downloading github.com/aws/aws-lambda-go v1.23.0
go: downloading github.com/aws/aws-sdk-go v1.[44](https://github.com/hipcamp/platform/actions/runs/9161872803/job/25187672612#step:5:45).102
go: downloading github.com/pkg/errors v0.9.1
go: downloading github.com/sirupsen/logrus v1.8.1
go: downloading github.com/spf13/cobra v1.1.3
go: downloading github.com/spf13/viper v1.7.1
go: downloading github.com/hashicorp/go-retryablehttp v0.7.0
go: downloading google.golang.org/api v0.46.0
go: downloading golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e
go: downloading github.com/spf13/pflag v1.0.5
go: downloading github.com/fsnotify/fsnotify v1.4.9
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading github.com/magiconair/properties v1.8.5
go: downloading github.com/mitchellh/mapstructure v1.4.1
go: downloading github.com/pelletier/go-toml v1.9.0
go: downloading github.com/spf13/afero v1.6.0
go: downloading github.com/spf13/cast v1.3.1
go: downloading github.com/spf13/jwalterweatherman v1.1.0
go: downloading github.com/subosito/gotenv v1.2.0
go: downloading gopkg.in/ini.v1 v1.62.0
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading github.com/jmespath/go-jmespath v0.4.0
go: downloading github.com/BurntSushi/toml v1.0.0
go: downloading golang.org/x/oauth2 v0.0.0-20210427180440-81ed05c6b58c
go: downloading github.com/hashicorp/go-cleanhttp v0.5.2
go: downloading golang.org/x/text v0.3.7
go: downloading google.golang.org/grpc v1.37.0
go: downloading cloud.google.com/go v0.81.0
go: downloading github.com/googleapis/gax-go/v2 v2.0.5
go: downloading go.opencensus.io v0.23.0
go: downloading golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd
go: downloading github.com/golang/protobuf v1.5.2
go: downloading google.golang.org/genproto v0.0.0-202104291814[45](https://github.com/hipcamp/platform/actions/runs/9161872803/job/25187672612#step:5:46)-86c259c2b4ab
go: downloading github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e
go: downloading google.golang.org/protobuf v1.26.0
time="2024-05-20T16:30:22Z" level=info msg="Syncing AWS users and groups from Google Workspace SAML Application"
time="2024-05-20T16:30:22Z" level=info msg="Test call for groups successful" Groups="***\n  Groups: [\n    ***\n      DisplayName: <sensitive>,\n      GroupId: \"c458b458-f071-7038-d731-80f11e2b6c9d\",\n      IdentityStoreId: \"d-9067f2fdbb\"\n    ***,\n    ***\n      DisplayName: <sensitive>,\n      GroupId: \"d4a884b8-b091-70e1-7f63-1dfc415c08a4\",\n      IdentityStoreId: \"d-9067f2fdbb\"\n    ***,\n    ***\n      DisplayName: <sensitive>,\n      GroupId: \"54c81[47](https://github.com/hipcamp/platform/actions/runs/9161872803/job/25187672612#step:5:48)8-c091-7056-943b-0cd69d37e5d8\",\n      IdentityStoreId: \"d-9067f2fdbb\"\n    ***,\n    ***\n      DisplayName: <sensitive>,\n      GroupId: \"94[48](https://github.com/hipcamp/platform/actions/runs/9161872803/job/25187672612#step:5:49)5408-7061-702d-0d94-354d45b85174\",\n      IdentityStoreId: \"d-9067f2fdbb\"\n    ***,\n    ***\n      DisplayName: <sensitive>,\n      GroupId: \"04b85428-f071-7017-2ab8-367d16eb1ef0\",\n      IdentityStoreId: \"d-9067f2fdbb\"\n    ***\n  ]\n***"
time="2024-05-20T16:30:22Z" level=info msg=syncing sync_method=groups
time="2024-05-20T16:30:22Z" level=info msg="get google groups" queryGroup="*"
time="2024-05-20T16:30:22Z" level=info msg="get google users" queryUsers="*"
time="2024-05-20T16:30:22Z" level=debug msg="preparing list of google users, groups and their members"
time="2024-05-20T16:30:22Z" level=debug msg="Fetching ALL users from google, to use as cache"
time="2024-05-20T16:30:22Z" level=fatal msg="googleapi: Error 400: Invalid Input, invalid"

Let me know if you have any other ideas! Thanks!

[hanswesterbeek]

We are also encountered this, the solution was to pass --google-admin foo@bar.com where foo@bar.com is an admin user in your Google Workspace

[ChrisPates]

Is the mail address your supplying for Google-admin a super-admin?

[tyrannosaurus-becks]

@ChrisPates mine was!

[ChrisPates]

Have you tried deploying from the serverless application repository? (Link in the README.md). It is by far the easiest deployment route.

I notice you are running it under git actions, you will need to establish an aws session to allow the IAM Identity Center - Identity Store api.

[tyrannosaurus-becks]

Thanks for continuing to respond here @ChrisPates ! I actually ended up just writing a Python script to do what I needed, since I hit a wall here.

I ended up opening https://github.com/awslabs/ssosync/issues/197 for other folks in the future. Not sure if @hanswesterbeek or @ricpruss are still encountering the issue as well, but I will let them follow up with more details if so.

[ChrisPates]

Fair enough, whatever works.

I saw that issue and your right the read me now contains too much information on too many topics. I'll look at shifting the details to a separate page and refocus on the read me on being a QuickStart.