Major Codebase Refactor for Easier Maintenance

mide commented 6 years ago

Breaking Changes

Removed Python 2.6 Support. (Also see #41 for this specific issue)
- pip doesn't support 2.6 since fall 2016 https://github.com/pypa/pip/issues/3955
- pytestdoesn't support 2.6 since fall 2017 https://github.com/pytest-dev/pytest/issues/2812
- setuptools doesn't support 2.6 since fall 2017 https://github.com/pypa/setuptools/issues/878
- Many other tools look like they're moving in this direction (and eventually moving to remove all 2.* support, but I'm not going that far yet.)
~~Remove persistent (to file) profiles.~~
- Previously, aws-google-auth would write to a user's ~/.aws/config file and read the values when run the next time. I feel this adds a complexity in trying to configure the tool to determine defaults. aws-google-auth supports command line params, user input, environment variables and adding another may be too much to maintain. Of course, we can add this back if desired. (Edit: Per feedback in #38, I will add this back in)

Under-the-hood Changes

Broke apart __init__.py into the following files (along with their purproses):
- google.py is for all Google related functions. It includes the logic to perform the page scraping and SAML fetching.
- amazon.py for AWS related fucntions. This performs the role extraction from the SAML and performs the AWS API call to get the access tokens.
- configuration.py - This only maintains user options (anything the user specifies). Breaking this into it's own object allows us to perform much more robust testing and just pass around a single object instead of a handful of options.
- util.py for common tooling

Added the following tests

Flake8 Python style testing. This is added into the TravisCI build script (.travis.yml).
test_configuration.py:
- Test that invalid duration values get rejected.
- Test that valid duration values are accepted.
- Test that the default value of duration is max_duration
- Test that invalid ask_role values get rejected.
- Test that valid ask_role values are accepted.
- Test that ask_role is an optional setting.
- Test that invalid idp_id values get rejected.
- Test that valid idp_id values are accepted.
- Test that invalid sp_id values get rejected.
- Test that valid username values are accepted.
- Test that invalid username values get rejected.
- Test that valid sp_id values are accepted.
- Test that the default value of profile is sts
- Test that invalid profile values get rejected.
- Test that valid profile values are accepted.
- Test all profile_defaults.
- Test that invalid region values get rejected.
- Test that valid region values are accepted.
- Test that the default value of region is ap_southeast_2
- Test that invalid role_arn values get rejected.
- Test that role_arn_is is an optional setting.
- Test that valid role_arn values are accepted.
- Test that invalid u2f_disabled values get rejected.
- Test that valid u2f_disabled values are accepted.
- Test that u2f_disabled_is is an optional setting.
test_amazon.py
- Test that the sts boto client properly returns an STS client object.
- Test that role extraction happens correctly given a valid SAML response.
- Test that role extraction happens correctly given a SAML response with too many commas (See #12)

Needed Work

In order to get tests to pass, I had to ignore Flake8 rule E722. We should go back and determine what the correct exceptions are to catch and only catch those.

Note

Please don't feel you need to accept this, but do please let me know. If this breaks Cevo's workflows, I may just end up maintaining my own fork.

nonspecialist commented 6 years ago

Hmmm, I have a profile in ~/.aws/config with the profile prefix in the name; e.g.

[profile None]
region = ap-southeast-2
output = json
...

It's quite possible that this was due to an intermediate version, but the result is confusing. It breaks with:

(aws-google-auth) cmp@tak.local aws-google-auth $ bin/aws-google-auth 
Failed to import U2F libraries, U2F login unavailable. Other methods can still continue.
Google Password: 
Invalid parameters.
Traceback (most recent call last):
  File "bin/aws-google-auth", line 11, in <module>
    load_entry_point('aws-google-auth==0.0.16', 'console_scripts', 'aws-google-auth')()
  File "/home/cmp/Build/Cevo/aws-google-auth/lib/python2.7/site-packages/aws_google_auth-0.0.16-py2.7.egg/aws_google_auth/__init__.py", line 52, in main
    cli(sys.argv[1:])
  File "/home/cmp/Build/Cevo/aws-google-auth/lib/python2.7/site-packages/aws_google_auth-0.0.16-py2.7.egg/aws_google_auth/__init__.py", line 107, in cli
    config.raise_if_invalid()
  File "/home/cmp/Build/Cevo/aws-google-auth/lib/python2.7/site-packages/aws_google_auth-0.0.16-py2.7.egg/aws_google_auth/configuration.py", line 72, in raise_if_invalid
    assert (self.profile.__class__ is str), "Expected profile to be a string. Got {}.".format(self.profile.__class__)
AssertionError: Expected profile to be a string. Got <type 'NoneType'>.

If I edit out the profile prefix (so it's just [None]) then all's well.

If I specify a profile on the command-line, eg aws-google-auth -p cevo-dev then it works too; but not if I just run it without setting a default profile.

$AWS_PROFILE is unset.

I think it's because config.read(args.profile) gets a NoneType first, before any sanity checking is done and then, after environment variables are read again, it's overridden by default_if_none which is passed a NoneType in as the default

Adding default='default' to line 27 of __init__.py fixes this but it would be useful to report on which profile is having issues, so the user can adjust/edit their ~/.aws/config file if needed.

nonspecialist commented 6 years ago

I think my description is confusing, so I'll adjust it.

Missing any default value for the profile argument results in a NoneType being used to load, and re-load the profile, instead of using the profile "default"

mide commented 6 years ago

Hmmm, okay. That's an interesting case. I'll take a look at that in the morning @nonspecialist. (I'm in the US - New York time zone)