search

studionone / cognito-export

Export user data from an AWS Cognito user pool
7 stars 8 forks source link

Cognito limitation? #1

Open carlsonjf opened 4 years ago

carlsonjf commented 4 years ago

We are running into an issue with exporting users from Cognito.
Have tried cognito-export-users and have some ideas that may be causing the error below...

We have > 300 users in our Cognito user pool. When we run the node package cognito-export-users we get 282 users and at the end, instead of a properly formed json stream the last character is a % (rather than a ]). In looking for other packages that may get around this issue, I found yours and would fix the issue. It looks like we have the same issue here. Are you aware of a limitation of the APIs you are using in exporting more than 282 users?

File "cognito-export.py", line 93, in data = CognitoExport(args[''], args['--profile']) File "cognito-export.py", line 51, in init data = json.loads(result.stdout.decode('utf-8')) File "/Users/jfcarlso/opt/anaconda3/lib/python3.7/json/init.py", line 348, in loads return _default_decoder.decode(s) File "/Users/jfcarlso/opt/anaconda3/lib/python3.7/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/Users/jfcarlso/opt/anaconda3/lib/python3.7/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None

JonathanHolvey commented 4 years ago

Without looking into it too deeply, I'd say the AWS CLI is returning invalid JSON (or an error message) that the json library is unable to decode. If you add print(result.stdout.decode('utf-8')) before line 51 of cognito-export.py, you should be able to see what is coming back from each CLI call.