rahulpsd18 / cognito-backup-restore

AIO Tool for backing up and restoring AWS Cognito User Pools
MIT License
196 stars 64 forks source link

Can't restore a google account #38

Open toddb opened 3 years ago

toddb commented 3 years ago

I am doing some migrations. All was good until I hit a google_xxx account. I haven't been able to locate the error. Is it possible? Links to documentation would be great.

Note: I am raising it here because these value came out of a backup and then I tried to restore and can imagine other people having this problem. In practice what happens is that the non external accounts (ie cognito) restore but it is very hard to notice this.

My assumption is that either we need to strip values of external accounts on restore or setup the pool differently. The former seems more probable because I don't/can't change my pool without deleting it.

UPDATE: Looking at linking cognito user to federated accounts it would seem that it is not surprising we can't out of the box.

General Settings > Attributes

Federation > Google > Attribute mapping

Here's my script, sample json from a backup and the error message.

➜  cognito-migrate node ./node_modules/.bin/cbr restore \
>      --file to_add.json \
>     -r xxxx \
>     -p default \
>     --pool XXXXXXX
✔ Users imported successfully to XXXXXXX
[
  {
    "Username": "google_100051227669222736914",
    "Attributes": [
      {
        "Name": "sub",
        "Value": "93085109-16ac-4997-9680-0b61dbd4771d"
      },
      {
        "Name": "identities",
        "Value": "[{\"userId\":\"100051227669222736914\",\"providerName\":\"Google\",\"providerType\":\"Google\",\"issuer\":null,\"primary\":true,\"dateCreated\":1609832494628}]"
      },
      {
        "Name": "email_verified",
        "Value": "true"
      },
      {
        "Name": "name",
        "Value": "J T"
      },
      {
        "Name": "given_name",
        "Value": "J"
      },
      {
        "Name": "family_name",
        "Value": "T"
      },
      {
        "Name": "email",
        "Value": "J.T@gmail.com"
      }
    ],
    "UserCreateDate": "2021-01-05T07:41:34.635Z",
    "UserLastModifiedDate": "2021-01-05T07:41:34.635Z",
    "Enabled": true,
    "UserStatus": "EXTERNAL_PROVIDER"
  }
]
cognito-migrate node ./node_modules/.bin/cbr restore \
>      --file to_add.json \
>     -r xxxx \
>     -p default \
>     --pool xxxxxx
✔ Users imported successfully to xxxxxx

cognito-migrate/node_modules/aws-sdk/lib/protocol/json.js:52
  resp.error = util.error(new Error(), error);
                          ^

InvalidParameterException: Cannot modify the non-mutable attribute identities
    at Request.extractError (cognito-migrate/node_modules/aws-sdk/lib/protocol/json.js:52:27)
    at Request.callListeners (cognito-migrate/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (cognito-migrate/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (cognito-migrate/node_modules/aws-sdk/lib/request.js:688:14)
    at Request.transition (cognito-migrate/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (cognito-migrate/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at cognito-migrate/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (cognito-migrate/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (cognito-migrate/node_modules/aws-sdk/lib/request.js:690:12)
    at Request.callListeners (cognito-migrate/node_modules/aws-sdk/lib/sequential_executor.js:116:18) {
  code: 'InvalidParameterException',
  time: 2021-06-18T06:40:27.805Z,
  requestId: '90f09f62-7b7e-412f-a3c4-eb6296e2f727',
  statusCode: 400,
  retryable: false,
  retryDelay: 64.47731523692524
matej-topolovec commented 2 years ago

@toddb Did you manage to find a workaround for this? I'm also struggling with the same issue. Thanks

toddb commented 2 years ago

I have just looked at the scripts/project I used for the migration. It was all pretty awful and too embarrassing to post here! It is undocumented with commented out code and looks for practical purposes one-off (although looking at it I kept running it over a period of time of the migration). I have put in some effectively pseudo code to help. It was painful.

From what I can tell.

Good luck.

#!/usr/bin/env bash

set -e

# generate restore json files for each environment
node ./node_modules/.bin/cbr backup -r ap-southeast-2 -p default --pool ap-southeast-2_src --dir .
node ./node_modules/.bin/cbr backup -r ap-southeast-2 -p default --pool ap-southeast-2_dest --dir .
# diff the two files to return on non-google with an email address (note: not using phone)
node ./diff.js  
# add new addresses without google
node ./node_modules/.bin/cbr restore \
    --pwdModule ./cognito-migrate/pwd_module.js \
    --file to_add_cognito.json \
    -r ap-southeast-2 \
    -p default \
    --pool ap-southeast-2_dest
const fs = require('fs');

export function readPool(poolId) {
    const data = fs.readFileSync(__dirname + '/ap-southeast-2_' + poolId + '.json');
    return JSON.parse(data);
}

const src = readPool('ap-southeast-2_src');
const dest = readPool('ap-southeast-2_dest');

let emailPredicate = x => x.Attributes.filter(x => x.Name === 'email')[0].Value;

const prodEmails = dest.map(x => emailPredicate(x));

const nonGoogleAccounts = src.filter(x => !x.Username.includes('google'));

const toAddCognito = nonGoogleAccounts.filter(x => !prodEmails.includes(emailPredicate(x)))

fs.writeFileSync(__dirname + '/to_add_cognito.json', JSON.stringify(toAddCognito));