search

trek10inc / awsume

A utility for easily assuming AWS IAM roles from the command line.
https://awsu.me
MIT License
494 stars 90 forks source link

TypeError: datetime.datetime(2018, 3, 19, 12, 52, 54, tzinfo=tzutc()) is not JSON serializable #29

Closed felipe1982 closed 6 years ago

felipe1982 commented 6 years ago

Getting this error with 3.0.9 in WSL environment (opensuse)

felipe@DESKTOP-TCPCKB9:~/Downloads$ awsume -a velocity-ipaas-nonprod
AWSume: User profile credentials will expire at: 2018-03-19 12:52:54+00:00
AWSume: Role profile credentials will expire at: 2018-03-19 11:52:55
Traceback (most recent call last):
  File "/usr/bin/awsumepy", line 11, in <module>
    load_entry_point('awsume==3.0.9', 'console_scripts', 'awsumepy')()
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 1415, in main
    awsume.run(command_line_arguments)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 1383, in run
    func(self, arguments, profiles, user_session, role_session)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 862, in get_role_session_callback
    start_auto_awsume(args, app, profiles, AWS_CREDENTIALS_FILE, user_session, role_session)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 892, in start_auto_awsume
    write_auto_awsume_session(args.target_profile_name, auto_profile, credentials_file_path)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 958, in write_auto_awsume_session
    LOG.debug('AutoAwsume profile: %s', json.dumps(auto_profile, indent=2))
  File "/usr/lib64/python2.7/json/__init__.py", line 251, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/usr/lib64/python2.7/json/encoder.py", line 209, in encode
    chunks = list(chunks)
  File "/usr/lib64/python2.7/json/encoder.py", line 434, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/usr/lib64/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib64/python2.7/json/encoder.py", line 442, in _iterencode
    o = _default(o)
  File "/usr/lib64/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: datetime.datetime(2018, 3, 19, 12, 52, 54, tzinfo=tzutc()) is not JSON serializable
felipe1982 commented 6 years ago

sudo pip install awsume==2.1.5

problems go away

mbarneyjr commented 6 years ago

This is likely from an issue with the update from 2.x to 3.x. The way AWSume cache credentials has changed since 2.x, so what you'll want to do is to stop autoawsume with awsume -k, and then either remove all of AWSume's cache files in your ~/.aws/cli/cache/ directory, or run your AWSume commands with the -r flag, to ignore the cache and refresh it with the correct, current format (preferably removing the cache files).

However it would be better if AWSume handled backwards-incompatibility gracefully, so I will work on an update to handle these kinds of situations.

If removing the cache or -r doesn't work, please let me know!

felipe1982 commented 6 years ago

solved.

[sudo] password for root:
Collecting awsume
  Using cached awsume-3.0.9.tar.gz
Requirement already up-to-date: boto3 in /usr/lib/python2.7/site-packages (from awsume)
Requirement already up-to-date: psutil in /usr/lib64/python2.7/site-packages (from awsume)
Requirement already up-to-date: yapsy in /usr/lib/python2.7/site-packages (from awsume)
Requirement already up-to-date: future in /usr/lib/python2.7/site-packages (from awsume)
Requirement already up-to-date: jmespath<1.0.0,>=0.7.1 in /usr/lib/python2.7/site-packages (from boto3->awsume)
Requirement already up-to-date: botocore<1.10.0,>=1.9.11 in /usr/lib/python2.7/site-packages (from boto3->awsume)
Requirement already up-to-date: s3transfer<0.2.0,>=0.1.10 in /usr/lib/python2.7/site-packages (from boto3->awsume)
Requirement already up-to-date: python-dateutil<2.7.0,>=2.1 in /usr/lib/python2.7/site-packages (from botocore<1.10.0,>=1.9.11->boto3->awsume)
Requirement already up-to-date: docutils>=0.10 in /usr/lib/python2.7/site-packages (from botocore<1.10.0,>=1.9.11->boto3->awsume)
Requirement already up-to-date: futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" in /usr/lib/python2.7/site-packages (from s3transfer<0.2.0,>=0.1.10->boto3->awsume)
Requirement already up-to-date: six>=1.5 in /usr/lib/python2.7/site-packages (from python-dateutil<2.7.0,>=2.1->botocore<1.10.0,>=1.9.11->boto3->awsume)
Installing collected packages: awsume
  Found existing installation: awsume 2.1.5
    Uninstalling awsume-2.1.5:
      Successfully uninstalled awsume-2.1.5
  Running setup.py install for awsume ... done
Successfully installed awsume-3.0.9
felipe@DESKTOP-TCPCKB9:~/repos/vff-ipaas-infrastructure/ansible (feature/syslog-apigee *$=)$ awsume -k
awsumefelipe@DESKTOP-TCPCKB9:~/repos/vff-ipaas-infrastructure/ansible (feature/syslog-apigee *$=)$ awsume -r velocity-ipaas-nonprod
AWSume: User profile credentials will expire at: 2018-03-20 12:01:35+00:00
AWSume: Role profile credentials will expire at: 2018-03-20 12:01:42
felipe@DESKTOP-TCPCKB9:~/repos/vff-ipaas-infrastructure/ansible (feature/syslog-apigee *$=)$
felipe1982 commented 6 years ago

Sorry. False alarm. I hit this

elipe@DESKTOP-TCPCKB9:~/repos/vff-ipaas-infrastructure/ansible (feature/syslog-apigee *$=)$ sudo pip install -U awsume
[sudo] password for root:
Collecting awsume
  Using cached awsume-3.0.9.tar.gz
Requirement already up-to-date: boto3 in /usr/lib/python2.7/site-packages (from awsume)
Requirement already up-to-date: psutil in /usr/lib64/python2.7/site-packages (from awsume)
Requirement already up-to-date: yapsy in /usr/lib/python2.7/site-packages (from awsume)
Requirement already up-to-date: future in /usr/lib/python2.7/site-packages (from awsume)
Requirement already up-to-date: jmespath<1.0.0,>=0.7.1 in /usr/lib/python2.7/site-packages (from boto3->awsume)
Requirement already up-to-date: botocore<1.10.0,>=1.9.11 in /usr/lib/python2.7/site-packages (from boto3->awsume)
Requirement already up-to-date: s3transfer<0.2.0,>=0.1.10 in /usr/lib/python2.7/site-packages (from boto3->awsume)
Requirement already up-to-date: python-dateutil<2.7.0,>=2.1 in /usr/lib/python2.7/site-packages (from botocore<1.10.0,>=1.9.11->boto3->awsume)
Requirement already up-to-date: docutils>=0.10 in /usr/lib/python2.7/site-packages (from botocore<1.10.0,>=1.9.11->boto3->awsume)
Requirement already up-to-date: futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" in /usr/lib/python2.7/site-packages (from s3transfer<0.2.0,>=0.1.10->boto3->awsume)
Requirement already up-to-date: six>=1.5 in /usr/lib/python2.7/site-packages (from python-dateutil<2.7.0,>=2.1->botocore<1.10.0,>=1.9.11->boto3->awsume)
Installing collected packages: awsume
  Found existing installation: awsume 2.1.5
    Uninstalling awsume-2.1.5:
      Successfully uninstalled awsume-2.1.5
  Running setup.py install for awsume ... done
Successfully installed awsume-3.0.9
felipe@DESKTOP-TCPCKB9:~/repos/vff-ipaas-infrastructure/ansible (feature/syslog-apigee *$=)$ awsume -k
awsumefelipe@DESKTOP-TCPCKB9:~/repos/vff-ipaas-infrastructure/ansible (feature/syslog-apigee *$=)$ awsume -r velocity-ipaas-nonprod
AWSume: User profile credentials will expire at: 2018-03-20 12:01:35+00:00
AWSume: Role profile credentials will expire at: 2018-03-20 12:01:42
felipe@DESKTOP-TCPCKB9:~/repos/vff-ipaas-infrastructure/ansible (feature/syslog-apigee *$=)$ awsume -a velocity-ipaas-nonprod
set | grep AWS_
AWSume: User profile credentials will expire at: 2018-03-20 12:03:06+00:00
AWSume: Role profile credentials will expire at: 2018-03-20 12:03:12
Traceback (most recent call last):
  File "/usr/bin/awsumepy", line 11, in <module>
    load_entry_point('awsume==3.0.9', 'console_scripts', 'awsumepy')()
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 1415, in main
    awsume.run(command_line_arguments)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 1383, in run
    func(self, arguments, profiles, user_session, role_session)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 862, in get_role_session_callback
    start_auto_awsume(args, app, profiles, AWS_CREDENTIALS_FILE, user_session, role_session)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 892, in start_auto_awsume
    write_auto_awsume_session(args.target_profile_name, auto_profile, credentials_file_path)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 958, in write_auto_awsume_session
    LOG.debug('AutoAwsume profile: %s', json.dumps(auto_profile, indent=2))
  File "/usr/lib64/python2.7/json/__init__.py", line 251, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/usr/lib64/python2.7/json/encoder.py", line 209, in encode
    chunks = list(chunks)
  File "/usr/lib64/python2.7/json/encoder.py", line 434, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/usr/lib64/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib64/python2.7/json/encoder.py", line 442, in _iterencode
    o = _default(o)
  File "/usr/lib64/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: datetime.datetime(2018, 3, 20, 12, 3, 6, tzinfo=tzutc()) is not JSON serializable
felipe@DESKTOP-TCPCKB9:~/repos/vff-ipaas-infrastructure/ansible (feature/syslog-apigee *$=)$ set | grep AWS_
AWS_ACCESS_KEY_ID=ASIAJTNXUHTJ4WKM3PBA
AWS_DEFAULT_REGION=ap-southeast-2
AWS_REGION=ap-southeast-2
AWS_SECRET_ACCESS_KEY=obfuscated
AWS_SECURITY_TOKEN=obfuscated
AWS_SESSION_TOKEN=obfuscated
felipe@DESKTOP-TCPCKB9:~/repos/vff-ipaas-infrastructure/ansible (feature/syslog-apigee *$=)$ rm ~/.aws/cli/
rm: cannot remove '/home/felipe/.aws/cli/': Is a directory
felipe@DESKTOP-TCPCKB9:~/repos/vff-ipaas-infrastructure/ansible (feature/syslog-apigee *$=)$ awsume -k
^[[Afelipe@DESKTOP-TCPCKB9:~/repos/vff-ipaas-infrastructure/ansible (feature/syslog-apigee *$=)$ awsume -a velocity-ipaas-nonprod
AWSume: User profile credentials will expire at: 2018-03-20 12:03:38+00:00
AWSume: Role profile credentials will expire at: 2018-03-20 12:03:44
Traceback (most recent call last):
  File "/usr/bin/awsumepy", line 11, in <module>
    load_entry_point('awsume==3.0.9', 'console_scripts', 'awsumepy')()
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 1415, in main
    awsume.run(command_line_arguments)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 1383, in run
    func(self, arguments, profiles, user_session, role_session)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 862, in get_role_session_callback
    start_auto_awsume(args, app, profiles, AWS_CREDENTIALS_FILE, user_session, role_session)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 892, in start_auto_awsume
    write_auto_awsume_session(args.target_profile_name, auto_profile, credentials_file_path)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 958, in write_auto_awsume_session
    LOG.debug('AutoAwsume profile: %s', json.dumps(auto_profile, indent=2))
  File "/usr/lib64/python2.7/json/__init__.py", line 251, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/usr/lib64/python2.7/json/encoder.py", line 209, in encode
    chunks = list(chunks)
  File "/usr/lib64/python2.7/json/encoder.py", line 434, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/usr/lib64/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib64/python2.7/json/encoder.py", line 442, in _iterencode
    o = _default(o)
  File "/usr/lib64/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: datetime.datetime(2018, 3, 20, 12, 3, 38, tzinfo=tzutc()) is not JSON serializable
mbarneyjr commented 6 years ago

Can you upload the output of running the failing AWSume commands with --debug?

felipe1982 commented 6 years ago

debug on

awsume.awsumepy.pre_awsume : Debug logs are visible
awsume.awsumepy.pre_awsume : Using profilename: velocity-ipaas-nonprod
awsume.awsumepy.get_aws_profiles : Getting AWS profiles
awsume.awsumepy.read_ini_file : Reading ini file from /home/felipe/.aws/config
awsume.awsumepy.read_ini_file : Reading ini file from /home/felipe/.aws/credentials
awsume.awsumepy.mix_role_and_source_profiles : Combining role and source profiles
awsume.awsumepy.merge_role_and_source_profile : merging config and credentials profile for [velocity-cloudops-build]
awsume.awsumepy.merge_role_and_source_profile : merging config and credentials profile for [velocity-ipaas-nonprod]
awsume.awsumepy.merge_role_and_source_profile : merging config and credentials profile for [velocity-cloudops-nonprod]
awsume.awsumepy.merge_role_and_source_profile : merging config and credentials profile for [velocity-ipaas-prod]
awsume.awsumepy.run : Collected aws profiles:
{
  "velocity-cloudops-build": {
obfuscated  },
  "velocity-ipaas-nonprod": {
 obfuscated },
  "saml": {
  obfuscated },
  "default": {
    "__name__": "default",
    "region": "ap-southeast-2",
    "output": "table"
  },
  "velocity": {
obfuscated
  },
  "velocity-cloudops-nonprod": {
obfuscated
  },
  "velocity-ipaas-prod": {
obfuscated
  }
}
awsume.awsumepy.get_aws_profiles_callback : Validating Profile
awsume.awsumepy.get_user_session : Getting user session credentials
awsume.awsumepy.read_aws_cache : Reading aws cache file
awsume.awsumepy.read_aws_cache : cache file does not exist
awsume.awsumepy.valid_cache_session : {}
awsume.awsumepy.valid_cache_session : Session is invalid
awsume.awsumepy.create_sts_client : Creating an STS client
awsume.awsumepy.get_user_session : profile does not require mfa
awsume.awsumepy.run : User session:
{
  "SecretAccessKey": "obfuscated",
  "SessionToken": "obfuscated",
  "Expiration": "2018-03-20 12:09:05+00:00",
  "AccessKeyId": "obfuscated"
}
AWSume: User profile credentials will expire at: 2018-03-20 12:09:05+00:00
awsume.awsumepy.get_role_session : Getting role session credentials
awsume.awsumepy.create_sts_client : Creating an STS client
awsume.awsumepy.fix_session_credentials : Converting session expiration to local timezone
awsume.awsumepy.run : Role session:
{
  "SecretAccessKey": "obfuscated",
  "region": "ap-southeast-2",
  "SessionToken": "obfuscated",
  "Expiration": "2018-03-20 12:09:12",
  "AccessKeyId": "obfuscated"
}
AWSume: Role profile credentials will expire at: 2018-03-20 12:09:12
awsume.awsumepy.start_auto_awsume : starting auto refresher
awsume.awsumepy.write_auto_awsume_session : Writing auto-awsume session
awsume.awsumepy.write_auto_awsume_session : Profile name: velocity-ipaas-nonprod
Traceback (most recent call last):
  File "/usr/bin/awsumepy", line 11, in <module>
    load_entry_point('awsume==3.0.9', 'console_scripts', 'awsumepy')()
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 1415, in main
    awsume.run(command_line_arguments)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 1383, in run
    func(self, arguments, profiles, user_session, role_session)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 862, in get_role_session_callback
    start_auto_awsume(args, app, profiles, AWS_CREDENTIALS_FILE, user_session, role_session)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 892, in start_auto_awsume
    write_auto_awsume_session(args.target_profile_name, auto_profile, credentials_file_path)
  File "/usr/lib/python2.7/site-packages/awsume/awsumepy.py", line 958, in write_auto_awsume_session
    LOG.debug('AutoAwsume profile: %s', json.dumps(auto_profile, indent=2))
  File "/usr/lib64/python2.7/json/__init__.py", line 251, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/usr/lib64/python2.7/json/encoder.py", line 209, in encode
    chunks = list(chunks)
  File "/usr/lib64/python2.7/json/encoder.py", line 434, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/usr/lib64/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib64/python2.7/json/encoder.py", line 442, in _iterencode
    o = _default(o)
  File "/usr/lib64/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: datetime.datetime(2018, 3, 20, 12, 9, 5, tzinfo=tzutc()) is not JSON serializable
mbarneyjr commented 6 years ago

I found the origin of the problem, it wasn't formatting correctly for profiles that don't require mfa.

3.0.10 has been released to fix this bug, thanks for the tip!

felipe1982 commented 6 years ago

working in 3.0.10. thank you sir!