awslabs / aws-glue-libs

AWS Glue Libraries are additions and enhancements to Spark for ETL operations.
Other
635 stars 299 forks source link

Using Glue Docker container for local development via SDK (boto3) or CLI #190

Open babaMar opened 1 year ago

babaMar commented 1 year ago

Wondering if the image I'm using: amazon/aws-glue-libs:glue_libs_4.0.0_image_01 supports all Glue functionalities. Specifically I'd like to test boto3 Glue methods like: client.create_connection, create_job, ... where client points to the running docker container client = boto3.client('glue', endpoint_url=f'http://localhost:{port}', use_ssl=False).

Now for port I tried 4040 and 18080 found in this AWS blog, without success.

Is there a port where the container is listening to such API calls? (e.g. aws glue get-jobs --no-verify-ssl --endpoint-url http://localhost:...)