Closed JeevaTM closed 2 years ago
Currently we do not have formal instructions to set up the marketplace connectors for the Docker image.
Here's couple of available options:
For the second one, this blog post has appendix to set up extra library dependencies. https://aws.amazon.com/blogs/big-data/develop-and-test-aws-glue-version-3-0-jobs-locally-using-a-docker-container/ Hope it helps.
I am using Iceberg Connector for Glue 3.0 on AWS Glue. Obviously works as expected, i.e., reads Iceberg table, write to Iceberg table, etc.
I created a new connection using Iceberg Connector for Glue 3.0 and added it in AWS Glue Job details under connections.
How do you do that in local Glue image?
Checking the AWS Glue for previous run logs, it does hints towards what is happening in the background
Is it possible to add some sort of docker-compose.yml with Glue image as one service and connector as another service? Making life easier to work with instead of fumbling through logs