Root login fails: "Your authentication information is incorrect"

iainelder commented 1 year ago

I'm evaluating awsapilib to see whether it can help me automate the building of a landing zone.

I have root user credentials for an AWS account that I just created today.

I know the credentials are valid because I used them to log into the console and retrieve the account ID.

I try to do the same using the AccountManager class.

from awsapilib import AccountManager

email="..."
password="..."
region="eu-west-1"
am = AccountManager(email, password, region)

print(am.account_id)

I expect this to print an account ID like 111111111111.

Instead it prints a traceback and an error about invalidate credentials.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/isme/.cache/pypoetry/virtualenvs/control-tower-RVRvYoiX-py3.8/lib/python3.8/site-packages/awsapilib/console/console.py", line 765, in account_id
    session = self._get_billing_session(self.email,
  File "/home/isme/.cache/pypoetry/virtualenvs/control-tower-RVRvYoiX-py3.8/lib/python3.8/site-packages/awsapilib/console/console.py", line 559, in _get_billing_session
    redirect_url = self._get_root_console_redirect(email, password, session, mfa_serial=mfa_serial)
  File "/home/isme/.cache/pypoetry/virtualenvs/control-tower-RVRvYoiX-py3.8/lib/python3.8/site-packages/awsapilib/console/console.py", line 552, in _get_root_console_redirect
    raise InvalidAuthentication(f'Unable to authenticate, response received was: {response.text} '
awsapilib.console.consoleexceptions.InvalidAuthentication: Unable to authenticate, response received was: {"state":"FAIL","properties":{"Message":"Your authentication information is incorrect. Please try again.","Title":"Authentication failed"}} with status code: 200

iainelder commented 1 year ago

I read similar issues #48 and #43 but didn't see anything that might solve my problem.

costastf commented 1 year ago

Hi @iainelder , thank you very much for reporting! I am sorry i did not respond earlier, was away on trainings on site. Next week I will be focusing on going through the backlog of issues and see if I can tackle them. Not sure if you are aware about this but if you you are trying to automate the creation of a landing zone, shall I suggest to have a look at https://superwerker.cloud/? I will probably ping you for more details on your issue next week when I work on it.

iainelder commented 1 year ago

Yes, Superwerker is on my list of things to try!

awsapilib is also interesting because it promises to automate actions without a public API. I don't know whether Superwerker also handles that.

costastf commented 1 year ago

Superwerker actually uses awsapilib under the hood for some actions, like setting up control tower and setting up the billing. I thing a mix and match approach would work best, where you would use superwerker and get a lot of stuff and maybe have another pass to customise if you want some stuff differently. But honestly the guys behind superwerker have done so much stuff that it does not make sense to try to reinvent that. :heart:

On Wed, May 31, 2023 at 11:52 AM Iain Samuel McLean Elder < @.***> wrote:

Yes, Superwerker is on my list of things to try!

awsapilib is also interesting because it promises to automate actions without a public API. I don't know whether Superwerker also handles that.

— Reply to this email directly, view it on GitHub https://github.com/schubergphilis/awsapilib/issues/49#issuecomment-1569864971, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDMK23QBYVKE5OUZNS2KWDXI4IEHANCNFSM6AAAAAAYGWQLOM . You are receiving this because you commented.Message ID: @.***>

iainelder commented 1 year ago

Superwerker actually uses awsapilib under the hood for some actions, like setting up control tower and setting up the billing.

Thanks for sharing. It sounds then like Superwerker is a good place to start.

I'll leave this issue open anyway so that you may investigate and decide what to do with it. Let me know if you need more details from me.

costastf commented 1 year ago

Will do next week when i work on this. Good luck with Superwerker!

iainelder commented 1 year ago

It seems that awsapilib is failing to detect the captcha response from the console login flow.

In my browser, I would do this:

Open console home in my browser (https://us-east-1.console.aws.amazon.com/console/home)
Select "Root user" sign in type
Paste my root user email address into the text field
Click the Next button

The console presents me with a "Security check" page that displays the text "Type the chatacters seen in the image below" and a captcha image.

When I enable debug messages in awsapilib, it says it doesn't detect a captcha.

In my terminal:

Enter a virtualenv with awsapilib installed

Save this to debug_account_manager.py.

import logging
from awsapilib import AccountManager

logging.basicConfig()
logging.getLogger("authentication.AccountManager").setLevel("DEBUG")

with open("/tmp/pass") as f:
  email, password = f.read().split(",")
region="us-east-1"
am = AccountManager(email, password, region)

print(am.account_id)

Save credentials in email,password format in /tmp/pass.
Execute the script to see the debug message before the failure.
```
python debug_account_manager.py
```

awsapilib prints DEBUG:authentication.AccountManager:No Captcha information found.. The complete output follows.

DEBUG:authentication.AccountManager:Trying to get url: https://us-east-1.console.aws.amazon.com/console/home with parameters :{'hashArgs': '#a', 'skipRegion': 'true', 'region': 'us-east-1'}
DEBUG:authentication.AccountManager:No Captcha information found.
Traceback (most recent call last):
  File "debug_account_manager.py", line 12, in <module>
    print(am.account_id)
  File "/home/isme/.cache/pypoetry/virtualenvs/landing-zone-accelerator-8fwM1A1d-py3.8/lib/python3.8/site-packages/awsapilib/console/console.py", line 765, in account_id
    session = self._get_billing_session(self.email,
  File "/home/isme/.cache/pypoetry/virtualenvs/landing-zone-accelerator-8fwM1A1d-py3.8/lib/python3.8/site-packages/awsapilib/console/console.py", line 559, in _get_billing_session
    redirect_url = self._get_root_console_redirect(email, password, session, mfa_serial=mfa_serial)
  File "/home/isme/.cache/pypoetry/virtualenvs/landing-zone-accelerator-8fwM1A1d-py3.8/lib/python3.8/site-packages/awsapilib/console/console.py", line 552, in _get_root_console_redirect
    raise InvalidAuthentication(f'Unable to authenticate, response received was: {response.text} '
awsapilib.console.consoleexceptions.InvalidAuthentication: Unable to authenticate, response received was: {"state":"FAIL","properties":{"Message":"Your authentication information is incorrect. Please try again.","Title":"Authentication failed"}} with status code: 200

costastf commented 1 year ago

Hi, i am working on this issue since yesterday. The problem is that aws requires the sending of a metadata1 argument that holds all kinds of fingerprinting metrics to accept the submission of the form. If that is not sent, or is not correct (or correct looking) the flow does not really work. I have made it all the way to logging in so I have a POC. I hope to be able to fix this and the other stuff during this week. Currently have been distracted with something else but will get back to it shortly.

iainelder commented 1 year ago

Thank you, @costastf. I appreciate the work you are doing on this. Let me know if I can help in any way.

costastf commented 1 year ago

Your feedback is help enough :)

iainelder commented 1 year ago

I just discovered the metadata1 hidden input parameter today in my browser's DOM inspector.

I searched on Google for metadata1 and "ECdITeCs:" to find some possible solutions to processing it.

costastf commented 1 year ago

Thanks! I have already solved this by seeing some of these myself. I will hopefully next week be able to wrap everything up. Random metadata1 along with lsubid are being created just fine and authentication flow continues. I need to match all the headers and cookies and clean up. It is just quite tedious and confusing work because there are a lot of handshakes and many exchanges, cookie invalidation and every service in AWS has it's own flow. Thanks for thinking along though ❤️ @iainelder

iainelder commented 1 year ago

Happy to help out!

The creation of a stand-alone AWS account includes an identity verification step. It appears to use the same captcha. I've been working on a way to automate the account creation. That's how I discovered the parameter now.

My approach is to use Selenium and Firefox to fill in all the forms. In that way I hope the browser automatically submits the correct metadata1 value. (I say "hope" because the UI apparently changed again to break my automation!)

I'm interested to see how your approach works. It seems that one way or another, some reverse engineering is required, either of the fingerprinting or of the UI elements.

costastf commented 1 year ago

Hi @iainelder , I have pushed a WIP branch of mostly working code (in not great shape, it needs a lot of refactoring and many things are not aligned throughout the code base, but it proves the case) https://github.com/schubergphilis/awsapilib/tree/fix-root-login. On that branch, under the console package there is a metadata module that has all the required code for the authentication to work. I will continue on this and clean it up and make it more robust, but I thought I'd share the progress in case it helps you with your efforts.

iainelder commented 1 year ago

Thanks! I'll try it out this week and give some feedback.

iainelder commented 1 year ago

I used your branch to run my debug script. There is more debugging output, but I get the same error in the end.

Here's the output I get:

DEBUG:authentication.AccountManager:Received response {"state":"SUCCESS","properties":{"resolvedAccountType":"Decoupled"}} with status code 200
DEBUG:authentication.AccountManager:No Captcha information found.
DEBUG:authentication.AccountManager:Resolved account type successfully with response :{"state":"SUCCESS","properties":{"resolvedAccountType":"Decoupled"}}
DEBUG:authentication.AccountManager:Received response {"mfaType":"NONE"} with status code 200
Traceback (most recent call last):
  File "/home/isme/tmp/tmp.2023-06-20.LLQoNwnm/awsapilib/debug_account_manager.py", line 12, in <module>
    print(am.account_id)
  File "/home/isme/tmp/tmp.2023-06-20.LLQoNwnm/awsapilib/awsapilib/console/console.py", line 811, in account_id
    session = self._get_billing_session(self.email,
  File "/home/isme/tmp/tmp.2023-06-20.LLQoNwnm/awsapilib/awsapilib/console/console.py", line 604, in _get_billing_session
    redirect_url = self._get_root_console_redirect(email, password, mfa_serial=mfa_serial)
  File "/home/isme/tmp/tmp.2023-06-20.LLQoNwnm/awsapilib/awsapilib/console/console.py", line 596, in _get_root_console_redirect
    raise InvalidAuthentication(f'Unable to authenticate, response received was: {response.text} '
awsapilib.console.consoleexceptions.InvalidAuthentication: Unable to authenticate, response received was: {"state":"FAIL","properties":{"Message":"Your authentication information is incorrect. Please try again.","Title":"Authentication failed"}} with status code: 200

Here's what I did:

Change to a clean directory
Clone your repo
Switch to the fix-root-login branch
Run pipenv install
Run pipenv shell
Follow the steps above to run debug_account_manager.py

Am I missing something?

costastf commented 1 year ago

No, you are not missing anything. AWS has a pretty extensive fingerprinting system so it might throw all kinds of hurdles along your way if it thinks you are a bot. So figuring out all the possible permutations of the auth flow is quite tricky and quite time consuming. I have the need to close around 100 accounts (so org api limits reached with boto) and have been at it since the morning and i get pretty inconsistent responses were the flow works fine almost 50% of the time. I need to be able to reproduce all the forks along the way and solve for it. Currently the code is a huge mess and it needs quite a redesign before this work becomes easy enough. This is my last working week and then I am off for three weeks so in all honesty, i do not think that I will manage to solve this 100% by friday.

iainelder commented 1 year ago

There is no deadline for this on my side. Enjoy your holiday!

costastf commented 1 year ago

I have had no failure with the currently pushed code for quite a few runs so give that a go.

iainelder commented 1 year ago

It's getting closer! Here's what I learned from two attempts.

The first attempt printed these lines.

DEBUG:authentication.AccountManager:Received response {"mfaType":"NONE"} with status code 200
DEBUG:authentication.AccountManager:Received response {"state":"SUCCESS","properties":{"resolvedAccountType":"Decoupled"}} with status code 200
DEBUG:authentication.AccountManager:No Captcha information found.
DEBUG:authentication.AccountManager:Resolved account type successfully with response :{"state":"SUCCESS","properties":{"resolvedAccountType":"Decoupled"}}

Then it was quiet for several seconds.

Finally it ended with a stack trace showing a JSONDecodeError.

Traceback (most recent call last):
  File "/home/isme/.local/share/virtualenvs/awsapilib-bgEL6V9t/lib/python3.10/site-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "/home/isme/.pyenv/versions/3.10.10/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/home/isme/.pyenv/versions/3.10.10/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/isme/.pyenv/versions/3.10.10/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/debug_account_manager.py", line 12, in <module>
    print(am.account_id)
  File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 818, in account_id
    session = self._get_billing_session(self.email,
  File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 610, in _get_billing_session
    redirect_url = self._get_root_console_redirect(email, password, mfa_serial=mfa_serial)
  File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 593, in _get_root_console_redirect
    success = self._validate_response(response)
  File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 468, in _validate_response
    any([response.json().get('state', '') == 'SUCCESS',
  File "/home/isme/.local/share/virtualenvs/awsapilib-bgEL6V9t/lib/python3.10/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

On my second attempt I used the command python debug_account_manager.py | ts -s "%.S" to prefix the output with elapsed seconds. This is so I can give you a better idea of how long it took to execute. Only the lines starting "Captcha: " got the prefix, but that's good enough to see that it took about 55 seconds to complete.

The second attempt prompted me for two captchas (I must have failed to read the first!) before it crashed another JSONDecodeError.

I redacted the unique parts of the output just in case.

DEBUG:authentication.AccountManager:Received response {"mfaType":"NONE"} with status code 200
DEBUG:authentication.AccountManager:Received response {"state":"SUCCESS","properties":{"CES":"...","Captcha":"true","CaptchaURL":"https://opfcaptcha-prod.s3.amazonaws.com/....jpg?AWSAccessKeyId\u003d...\u0026Expires\u003d...\u0026Signature\u003d...%3D","email":"...","captchaObfuscationToken":"{\"b64KeyCipherData\":\"...\",\"b64CipherData\":\"...\"}"}} with status code 200
DEBUG:authentication.AccountManager:Getting the resolve account type captcha.
17.379435 Please follow https://opfcaptcha-prod.s3.amazonaws.com/....jpg?AWSAccessKeyId=...&Expires=...&Signature=...%3D and provide the solution.
brzbgs
DEBUG:authentication.AccountManager:Resolved account type successfully with response :{"state":"SUCCESS","properties":{"resolvedAccountType":"Unknown","captchaStatusToken":"{\"b64KeyCipherData\":\"...\",\"b64CipherData\":\"...\"}"}}
DEBUG:authentication.AccountManager:Getting the after login type captcha.
36.520390 Captcha: Please follow https://opfcaptcha-prod.s3.amazonaws.com/....jpg?AWSAccessKeyId=...&Expires=...&Signature=...%3D and provide the solution.
563d8x
Traceback (most recent call last):
  File "/home/isme/.local/share/virtualenvs/awsapilib-bgEL6V9t/lib/python3.10/site-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "/home/isme/.pyenv/versions/3.10.10/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/home/isme/.pyenv/versions/3.10.10/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/isme/.pyenv/versions/3.10.10/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/debug_account_manager.py", line 12, in <module>
    print(am.account_id)
  File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 818, in account_id
    session = self._get_billing_session(self.email,
  File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 610, in _get_billing_session
    redirect_url = self._get_root_console_redirect(email, password, mfa_serial=mfa_serial)
  File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 593, in _get_root_console_redirect
    success = self._validate_response(response)
  File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 468, in _validate_response
    any([response.json().get('state', '') == 'SUCCESS',
  File "/home/isme/.local/share/virtualenvs/awsapilib-bgEL6V9t/lib/python3.10/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
54.639909 Captcha:

This is awesome progress! Let me know when I should try again :-)

costastf commented 1 year ago

I think they have flagged the User-Agent I have hardcoded on some parts because I have closed like 40 accounts since yesterday. I need to make that dynamic like the other part. I am planning that as part of the code clean up and refactoring. Not sure when, I hope to do this sometime until end of day tomorrow. The code is a huge mess and does not allow for easy refactoring so it needs a major rehaul.

On Thu, Jun 22, 2023 at 12:15 PM Iain Samuel McLean Elder < @.***> wrote:

It's getting closer! Here's what I learned from two attempts.

The first attempt printed these lines.

DEBUG:authentication.AccountManager:Received response {"mfaType":"NONE"} with status code 200 DEBUG:authentication.AccountManager:Received response {"state":"SUCCESS","properties":{"resolvedAccountType":"Decoupled"}} with status code 200 DEBUG:authentication.AccountManager:No Captcha information found. DEBUG:authentication.AccountManager:Resolved account type successfully with response :{"state":"SUCCESS","properties":{"resolvedAccountType":"Decoupled"}}

Then it was quiet for several seconds.

Finally it ended with a stack trace showing a JSONDecodeError.

Traceback (most recent call last): File "/home/isme/.local/share/virtualenvs/awsapilib-bgEL6V9t/lib/python3.10/site-packages/requests/models.py", line 971, in json return complexjson.loads(self.text, **kwargs) File "/home/isme/.pyenv/versions/3.10.10/lib/python3.10/json/init.py", line 346, in loads return _default_decoder.decode(s) File "/home/isme/.pyenv/versions/3.10.10/lib/python3.10/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/home/isme/.pyenv/versions/3.10.10/lib/python3.10/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/debug_account_manager.py", line 12, in print(am.account_id) File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 818, in account_id session = self._get_billing_session(self.email, File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 610, in _get_billing_session redirect_url = self._get_root_console_redirect(email, password, mfa_serial=mfa_serial) File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 593, in _get_root_console_redirect success = self._validate_response(response) File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 468, in _validate_response any([response.json().get('state', '') == 'SUCCESS', File "/home/isme/.local/share/virtualenvs/awsapilib-bgEL6V9t/lib/python3.10/site-packages/requests/models.py", line 975, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

On my second attempt I used the command python debug_account_manager.py | ts -s "%.S" to prefix the output with elapsed seconds. This is so I can give you a better idea of how long it took to execute. Only the lines starting "Captcha: " got the prefix, but that's good enough to see that it took about 55 seconds to complete.

The second attempt prompted me for two captchas (I must have failed to read the first!) before it crashed another JSONDecodeError.

I redacted the unique parts of the output just in case.

DEBUG:authentication.AccountManager:Received response {"mfaType":"NONE"} with status code 200 DEBUG:authentication.AccountManager:Received response {"state":"SUCCESS","properties":{"CES":"...","Captcha":"true","CaptchaURL":"https://opfcaptcha-prod.s3.amazonaws.com/....jpg?AWSAccessKeyId\u003d...\u0026Expires\u003d...\u0026Signature\u003d...%3D","email":"...","captchaObfuscationToken":"{\"b64KeyCipherData\":\"...\",\"b64CipherData\":\"...\"}"}} with status code 200 DEBUG:authentication.AccountManager:Getting the resolve account type captcha. 17.379435 Please follow https://opfcaptcha-prod.s3.amazonaws.com/....jpg?AWSAccessKeyId=...&Expires=...&Signature=...%3D and provide the solution. brzbgs DEBUG:authentication.AccountManager:Resolved account type successfully with response :{"state":"SUCCESS","properties":{"resolvedAccountType":"Unknown","captchaStatusToken":"{\"b64KeyCipherData\":\"...\",\"b64CipherData\":\"...\"}"}} DEBUG:authentication.AccountManager:Getting the after login type captcha. 36.520390 Captcha: Please follow https://opfcaptcha-prod.s3.amazonaws.com/....jpg?AWSAccessKeyId=...&Expires=...&Signature=...%3D and provide the solution. 563d8x Traceback (most recent call last): File "/home/isme/.local/share/virtualenvs/awsapilib-bgEL6V9t/lib/python3.10/site-packages/requests/models.py", line 971, in json return complexjson.loads(self.text, **kwargs) File "/home/isme/.pyenv/versions/3.10.10/lib/python3.10/json/init.py", line 346, in loads return _default_decoder.decode(s) File "/home/isme/.pyenv/versions/3.10.10/lib/python3.10/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/home/isme/.pyenv/versions/3.10.10/lib/python3.10/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/debug_account_manager.py", line 12, in print(am.account_id) File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 818, in account_id session = self._get_billing_session(self.email, File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 610, in _get_billing_session redirect_url = self._get_root_console_redirect(email, password, mfa_serial=mfa_serial) File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 593, in _get_root_console_redirect success = self._validate_response(response) File "/home/isme/tmp/tmp.2023-06-22.PCrQXURb/awsapilib/awsapilib/console/console.py", line 468, in _validate_response any([response.json().get('state', '') == 'SUCCESS', File "/home/isme/.local/share/virtualenvs/awsapilib-bgEL6V9t/lib/python3.10/site-packages/requests/models.py", line 975, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) 54.639909 Captcha:

This is awesome progress! Let me know when I should try again :-)

— Reply to this email directly, view it on GitHub https://github.com/schubergphilis/awsapilib/issues/49#issuecomment-1602295100, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDMK2YST3BVWER3GUCEZMLXMQEJ5ANCNFSM6AAAAAAYGWQLOM . You are receiving this because you were mentioned.Message ID: @.***>

costastf commented 1 year ago

Final update before I drop off for vacation. There seems to be a fork in the road where in certain circumstances the authentication does not go through the signin service but actually gets credentials like STS. This needs more looking into so it will have to wait for after the vacation.

iainelder commented 1 year ago

Thanks @costastf for your efforts. Enjoy the holiday. I'll be happy to review or test this again next month.

costastf commented 1 year ago

Hi @iainelder, branch https://github.com/schubergphilis/awsapilib/tree/fix-root-login kinda fixes the authentication but I get flagged pretty quickly as suspicious and can't really figure out why. It could be a billion things and not sure how to fix this. I am constructing a metadata1 variable with all kinds of automatic created variables, I have implemented the creation of the Lsubid variable and the first time I use the code everything works like a charm. I have a number of accounts that need closing and I am using this code to terminate those accounts but I keep getting flagged as suspicious and get responses like {"state":"FAIL","properties":{"Message":"Invalid request","Title":"Bad Request"}} . Every call is also send over to a telemetry end point at the same time. It could even be that since these calls are not being sent over to the telemetry end point they are getting flagged. Not sure how to proceed from this to be honest.

costastf commented 1 year ago

@iainelder I have refactored the code completely in the above mentioned branch and have had almost 100% success in closing 35 accounts using root access (the failures I had were with failed auto captchas). Do you mind giving that branch a try and let me know if it works for you?

iainelder commented 1 year ago

Welcome back, @costastf :-)

I ran debug_account_manager.py a few times between 08:10 and 08:22 UTC.

The first attempts fail fast. The later attempts go as far as the captcha. I'm not sure why it changed behavior.

In the end all attempts fail with the same message: {'state': 'FAIL', 'properties': {'Message': 'Your authentication information is incorrect. Please try again.', 'Title': 'Authentication failed'}}.

Would some extra debugging output in the awsapilib code or in my test program help here? I'm not sure what to add to mine apart from a start and end timestamp.

Run 1:

Traceback (most recent call last):
  File "/home/isme/tmp/tmp.2023-08-26.585oUgjL/awsapilib/debug_account_manager.py", line 12, in <module>
    print(am.account_id)
  File "/home/isme/tmp/tmp.2023-08-26.585oUgjL/awsapilib/awsapilib/console/console.py", line 445, in account_id
    session = authenticator.get_billing_root_session(self.email,
  File "/home/isme/tmp/tmp.2023-08-26.585oUgjL/awsapilib/awsapilib/authentication/authentication.py", line 421, in get_billing_root_session
    session = RootAuthenticator.get_root_console_session(self._solver,
  File "/home/isme/tmp/tmp.2023-08-26.585oUgjL/awsapilib/awsapilib/authentication/authentication.py", line 392, in get_root_console_session
    success = RootAuthenticator.validate_response(response)
  File "/home/isme/tmp/tmp.2023-08-26.585oUgjL/awsapilib/awsapilib/authentication/authentication.py", line 533, in validate_response
    raise InvalidAuthentication(response.json())
awsapilib.authentication.authenticationexceptions.InvalidAuthentication: {'state': 'FAIL', 'properties': {'Message': 'Your authentication information is incorrect. Please try again.', 'Title': 'Authentication failed'}}

Run 2: same as run 1.

Run 3: same as run 1.

Run 4: same as run 1.

Run 5:

Please follow https://opfcaptcha-prod.s3.amazonaws.com/910a80140d814543af54e6b1685896a9.jpg?AWSAccessKeyId=AKIA5WBBRBBBTTRJTAGZ&Expires=1693038060&Signature=0da0txVjpJq4YjrYjQLbHfwNaU8%3D and provide the solution.
Captcha: 83r7w4
Please follow https://opfcaptcha-prod.s3.amazonaws.com/155719c47024481980fcfea59fcd41e3.jpg?AWSAccessKeyId=AKIA5WBBRBBBTTRJTAGZ&Expires=1693038077&Signature=zHMxQbLr83c6Dt7Y6o3A7PmWkpw%3D and provide the solution.
Captcha: nww3dy
Traceback (most recent call last):
  File "/home/isme/tmp/tmp.2023-08-26.585oUgjL/awsapilib/debug_account_manager.py", line 13, in <module>
    print(am.account_id)
  File "/home/isme/tmp/tmp.2023-08-26.585oUgjL/awsapilib/awsapilib/console/console.py", line 445, in account_id
    session = authenticator.get_billing_root_session(self.email,
  File "/home/isme/tmp/tmp.2023-08-26.585oUgjL/awsapilib/awsapilib/authentication/authentication.py", line 421, in get_billing_root_session
    session = RootAuthenticator.get_root_console_session(self._solver,
  File "/home/isme/tmp/tmp.2023-08-26.585oUgjL/awsapilib/awsapilib/authentication/authentication.py", line 395, in get_root_console_session
    success = RootAuthenticator.validate_response(response)
  File "/home/isme/tmp/tmp.2023-08-26.585oUgjL/awsapilib/awsapilib/authentication/authentication.py", line 533, in validate_response
    raise InvalidAuthentication(response.json())
awsapilib.authentication.authenticationexceptions.InvalidAuthentication: {'state': 'FAIL', 'properties': {'Message': 'Your authentication information is incorrect. Please try again.', 'Title': 'Authentication failed'}}

Run 6:

Please follow https://opfcaptcha-prod.s3.amazonaws.com/231a6018c8e044298a42898896ae7e33.jpg?AWSAccessKeyId=AKIA5WBBRBBBTTRJTAGZ&Expires=1693038376&Signature=QjMb6vk%2FPARnlkGVEAQ7I2YmfC4%3D and provide the solution.
Captcha: 6mwfs3
Please follow https://opfcaptcha-prod.s3.amazonaws.com/42560eb340294c3388eac64e6b25d622.jpg?AWSAccessKeyId=AKIA5WBBRBBBTTRJTAGZ&Expires=1693038391&Signature=5Bjm0tzJxAaBVg%2Fl3lsuybZGXTA%3D and provide the solution.
Captcha: nyxc7t
Traceback (most recent call last):
  File "/home/isme/tmp/tmp.2023-08-26.585oUgjL/awsapilib/debug_account_manager.py", line 13, in <module>
    print(am.account_id)
  File "/home/isme/tmp/tmp.2023-08-26.585oUgjL/awsapilib/awsapilib/console/console.py", line 445, in account_id
    session = authenticator.get_billing_root_session(self.email,
  File "/home/isme/tmp/tmp.2023-08-26.585oUgjL/awsapilib/awsapilib/authentication/authentication.py", line 421, in get_billing_root_session
    session = RootAuthenticator.get_root_console_session(self._solver,
  File "/home/isme/tmp/tmp.2023-08-26.585oUgjL/awsapilib/awsapilib/authentication/authentication.py", line 395, in get_root_console_session
    success = RootAuthenticator.validate_response(response)
  File "/home/isme/tmp/tmp.2023-08-26.585oUgjL/awsapilib/awsapilib/authentication/authentication.py", line 533, in validate_response
    raise InvalidAuthentication(response.json())
awsapilib.authentication.authenticationexceptions.InvalidAuthentication: {'state': 'FAIL', 'properties': {'Message': 'Your authentication information is incorrect. Please try again.', 'Title': 'Authentication failed'}}

Captcha images:

83r7w4

nww3dy

6mwfs3

nyxc7t

costastf commented 1 year ago

Did you check whether your authentication is indeed incorrect? What I had happened a few times is that was flagged my efforts to login and automatically changed the account password and send me an email that went to spam. I am off course aiming 5 also that you are using the latest code on the fix branch.

costastf commented 1 year ago

The reason why your first attempts fail fast is probably because they are not flagged and they do not get any captcha, while later they get a captcha. I would actually first validate the credentials because it seems very consistent.

iainelder commented 1 year ago

I thought that too, but I did use the same credentials to log in using the console in Firefox.

In Firefox I didn't get any captchas. The console just accepted my name and password in the browser and logged me in as the root user.

I did try it with only one account. I'll try it with more to see whether I have better luck with the others.

PacoVK commented 1 year ago

I could use the code on the fix-root-login branch 👍 However, i now run into another issue when i try to update multiple account names, even with a wait of 60 seconds between password reset action and update account name. The reason is that AWS sends a verification code via Email, but the tool does break here. Would it be possible to check and ask for the code instead of crashing?

context

The script i run (imports redacted):

solver = Iterm()
password_manager = PasswordManager(solver=solver)

for key, value in account_batch.items():
    print("Request password reset for account", key, "with Email", value.rootEmail)

    password_manager.request_password_reset(value.rootEmail)

    email_reset_link = input("Please enter the reset link: ")

    characters = string.ascii_letters + string.digits + string.punctuation
    password = ''.join(secrets.choice(characters) for _ in range(24))

    print("password to be set for account:", password)

    password_manager.reset_password(email_reset_link, password)

    print("Did change password for account:", key, "waiting 60 seconds for propagation")

    time.sleep(60)

    account_manager = AccountManager(value.rootEmail, password, "eu-central-1", None, solver=solver)

    print("Update account name to new value ", value.newAccountName)

    account_manager.update_account_name(value.newAccountName)

    print("Did change account name for account:", key, "to", value.newAccountName)

    time.sleep(60)

print("Done with current batch of accounts")

Error message

awsapilib.authentication.authenticationexceptions.InvalidAuthentication: {'state': 'FAIL', 'properties': {'Message': 'Your authentication information is incorrect. Please try again.', 'Title': 'Authentication failed'}}

I get an email with the verification code. If i login via webconsole manually, i can see AWS asks for that code, after entering it, i can do any operation in the webconsole as expected. The error message is therefore a bit missleading here.

costastf commented 1 year ago

Hey @PacoVK , thanks for reporting! Oof this is very tricky. This is the back end identifying that something is off. The response send has 'Authentication failed' so that is what is reported. I have never come accross this code that you mention. This probably happens because the pseudo random Metadata that get send over the wire get flagged and there are extra checks put in place. It would be great to be able to fix that and not get flagged, that would be the ideal situation. I have not bumped into this code situation so cannot really ideate on how to handle it. Could you play around with the metadata creation code and see if you can get more robust behavior maybe? https://github.com/schubergphilis/awsapilib/blob/2ab78924c6f9619c6f2faa29f0578393790693d8/awsapilib/authentication/metadata.py#L296 this is where most of the pseudo random things are happening. Maybe disable other platforms than the one you actually running on and the random resolutions?

schubergphilis / awsapilib

Root login fails: "Your authentication information is incorrect" #49

context

Error message