resero-labs / docker-ds

Docker image for essential data science notebook environment (Jupyterlab)
0 stars 1 forks source link

requiring ~/.aws/config with profile "ds-notebook" is very fragile #8

Closed rappdw closed 5 years ago

rappdw commented 5 years ago

This has ripple effects into docker-utils, team-view-notebook and elsewere. Look into alternatives

Consider:

default_region = boto3.session.Session().region_name


**Note: This doesn't work on a kubernetes docker-ds image**
rappdw commented 5 years ago

Proposal for resolving region that works on laptop, ec2 instance and in container running in kubernetes on AWS

def get_session_params():
    import boto3
    import requests
    from credstash import get_session_params

    # first try boto3 (This will work on both an ec2 and laptop, but doesn't
    # work in kubernetes on AWS for some reason
    region = boto3.session.Session().region_name
    if not region:
        # next try the AWS config route (This works in kubernets)
        # set a short time out on this
        try:
            r = requests.get("http://169.254.169.254/latest/dynamic/instance-identity/document", timeout=0.2)
            response_json = r.json()
            region = response_json.get('region')
        except requests.exceptions.ConnectTimeout:
            pass
    if region:
        return {'region': region}
    # finally as a fallback try to get session parameters from ~/.aws/config under ds-notebook profile
    return get_session_params('ds-notebook', None)