botocore.exceptions.UnknownServiceError for glue

[Dave-Poole-60103]

I am running Python 3.6.2 with boto3 version 1.4.7 and am getting the error

botocore.exceptions.UnknownServiceError: Unknown service: 'glue'

I note that the closed issue https://github.com/boto/boto3/issues/1242 mentions upgrading Boto3 however 1.4.7 is the latest version

[danlindow]

Can you paste the output from the script as you run it? I'd like to see how this is being caught.

Thanks!

[Dave-Poole-60103]

My Python environment is a virtual environment built using Gradle.

Ignore the assert boto3.__version__ == "1.4.7" I put this in to see if boto3 was the version I was expecting and to force the test to fail early if it wasn't.

        app_config = get_configuration()
        assert boto3.__version__ == "1.4.7"
        client = boto3.client(
            'glue',
            aws_access_key_id=app_config._aws_credentials.access_key_id,
            aws_secret_access_key=app_config._aws_credentials.secret_access_key,
>           aws_session_token=app_config._aws_credentials.session_token)

test/test_scratchpad.py:45: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
build/venv/lib/python3.6/site-packages/boto3/__init__.py:83: in client
    return _get_default_session().client(*args, **kwargs)
build/venv/lib/python3.6/site-packages/boto3/session.py:263: in client
    aws_session_token=aws_session_token, config=config)
build/venv/lib/python3.6/site-packages/botocore/session.py:836: in create_client
    client_config=config, api_version=api_version)
build/venv/lib/python3.6/site-packages/botocore/client.py:64: in create_client
    service_model = self._load_service_model(service_name, api_version)
build/venv/lib/python3.6/site-packages/botocore/client.py:96: in _load_service_model
    api_version=api_version)
build/venv/lib/python3.6/site-packages/botocore/loaders.py:132: in _wrapper
    data = func(self, *args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <botocore.loaders.Loader object at 0x7ff12e5855c0>, service_name = 'glue'
type_name = 'service-2', api_version = None

    @instance_cache
    def load_service_model(self, service_name, type_name, api_version=None):
        """Load a botocore service model

            This is the main method for loading botocore models (e.g. a service
            model, pagination configs, waiter configs, etc.).

            :type service_name: str
            :param service_name: The name of the service (e.g ``ec2``, ``s3``).

            :type type_name: str
            :param type_name: The model type.  Valid types include, but are not
                limited to: ``service-2``, ``paginators-1``, ``waiters-2``.

            :type api_version: str
            :param api_version: The API version to load.  If this is not
                provided, then the latest API version will be used.

            :type load_extras: bool
            :param load_extras: Whether or not to load the tool extras which
                contain additional data to be added to the model.

            :raises: UnknownServiceError if there is no known service with
                the provided service_name.

            :raises: DataNotFoundError if no data could be found for the
                service_name/type_name/api_version.

            :return: The loaded data, as a python type (e.g. dict, list, etc).
            """
        # Wrapper around the load_data.  This will calculate the path
        # to call load_data with.
        known_services = self.list_available_services(type_name)
        if service_name not in known_services:
            raise UnknownServiceError(
                service_name=service_name,
>               known_service_names=', '.join(sorted(known_services)))
E           botocore.exceptions.UnknownServiceError: Unknown service: 'glue'. Valid service names are: acm, apigateway, application-autoscaling, appstream, athena, autoscaling, batch, budgets, clouddirectory, cloudformation, cloudfront, cloudhsm, cloudsearch, cloudsearchdomain, cloudtrail, cloudwatch, codebuild, codecommit, codedeploy, codepipeline, codestar, cognito-identity, cognito-idp, cognito-sync, config, cur, datapipeline, dax, devicefarm, directconnect, discovery, dms, ds, dynamodb, dynamodbstreams, ec2, ecr, ecs, efs, elasticache, elasticbeanstalk, elastictranscoder, elb, elbv2, emr, es, events, firehose, gamelift, glacier, greengrass, health, iam, importexport, inspector, iot, iot-data, kinesis, kinesisanalytics, kms, lambda, lex-models, lex-runtime, lightsail, logs, machinelearning, marketplace-entitlement, marketplacecommerceanalytics, meteringmarketplace, mturk, opsworks, opsworkscm, organizations, pinpoint, polly, rds, redshift, rekognition, resourcegroupstaggingapi, route53, route53domains, s3, sdb, servicecatalog, ses, shield, sms, snowball, sns, sqs, ssm, stepfunctions, storagegateway, sts, support, swf, waf, waf-regional, workdocs, workspaces, xray

build/venv/lib/python3.6/site-packages/botocore/loaders.py:378: UnknownServiceError
===================== 1 failed, 21 passed in 0.84 seconds ======================
:pytest FAILED

The output from the virtual environment build is as follows:-

:bootstrapPython SKIPPED
:setupPython
Fetching package metadata ...........
Solving package specifications: .

Package plan for installation in environment /home/datadev/code/EDW_DB_HOUSEKEEPING/parquet_experiment/build/venv:

The following NEW packages will be INSTALLED:

    arrow-cpp:       0.7.1-py36_2       conda-forge
    boto3:           1.4.7-py36_0       conda-forge
    botocore:        1.5.92-py36_0                 
    certifi:         2016.2.28-py36_0              
    dnspython:       1.15.0-py_0        conda-forge
    docutils:        0.14-py36_0                   
    ecdsa:           0.13-py36_0                   
    email_validator: 1.0.2-py_0         conda-forge
    faker:           0.8.1-py_0         conda-forge
    flake8:          3.3.0-py36_0                  
    future:          0.16.0-py36_1                 
    idna:            2.6-py36_0                    
    jmespath:        0.9.0-py36_0                  
    libpq:           9.5.4-0                       
    mccabe:          0.6.1-py36_0                  
    mkl:             2017.0.3-0                    
    numpy:           1.12.1-py36_0                 
    openssl:         1.0.2l-0                      
    pandas:          0.19.2-np112py36_1            
    parquet-cpp:     1.3.0.post-2       conda-forge
    pip:             9.0.1-py36_1                  
    psycopg2:        2.7.1-py36_0                  
    py:              1.4.34-py36_0                 
    pyarrow:         0.7.0-py36_1       conda-forge
    pycodestyle:     2.3.1-py36_0                  
    pycrypto:        2.6.1-py36_6                  
    pyflakes:        1.5.0-py36_0                  
    pyodbc:          4.0.17-py36_0                 
    pytest:          3.0.5-py36_0                  
    python:          3.6.2-0                       
    python-dateutil: 2.6.1-py36_0                  
    pytz:            2017.2-py36_0                 
    readline:        6.2-2                         
    s3transfer:      0.1.10-py36_0                 
    setuptools:      36.4.0-py36_1                 
    six:             1.10.0-py36_0                 
    sqlalchemy:      1.1.13-py36_0                 
    sqlite:          3.13.0-0                      
    tk:              8.5.18-0                      
    unixodbc:        2.3.4-0                       
    vertica-python:  0.7.3-py36_0       conda-forge
    wheel:           0.29.0-py36_0                 
    xz:              5.2.3-0                       
    zlib:            1.2.11-0           

[Dave-Poole-60103]

I notice that Glue was introduced in botocore 1.6.1 but if I try and force a load of botocore 1.6.1 or higher I get a message telling me that boto3 1.4.7 expects botocore <1.6.0

[danlindow]

I think you will want to open this issue under one of the following repos: https://github.com/boto/boto3 https://github.com/boto/botocore

This package covers a few scripts to assist with migrations.

[Dave-Poole-60103]

Am closing the issue on this board and re-entering it on https://github.com/boto/boto3 as suggested.

Thanks Dan