search

awslabs / aws-glue-libs

AWS Glue Libraries are additions and enhancements to Spark for ETL operations.
Other
635 stars 299 forks source link

is there a plan to update the docker image with glue version 4.0 #165

Open mpulaparthi opened 1 year ago

singlewind commented 1 year ago

Yes, I think so. I'm waiting the image to be released as well so I can update in pipeline. I suggest open source the Dockerfile repo as well, so we can update by yourself. https://github.com/awslabs/aws-glue-libs/issues/158#issuecomment-1353889589

dmschauer commented 1 year ago

Indeed, please release the dockerfile. It would help lots of people with their development workflow, see this discussion: https://github.com/awslabs/aws-glue-libs/issues/81#issuecomment-1240107699

PykupeJIbc commented 1 year ago

I join the previous speakers!

michal-ziarnik commented 1 year ago

upvoting request

saviodsouza29 commented 1 year ago

This issue has been resolved. - https://hub.docker.com/r/amazon/aws-glue-libs/tags?page=1&name=4.0.0

dmschauer commented 1 year ago

@saviodsouza29 What about the Dockerfile?

singlewind commented 1 year ago

Can we rebase the container to Amazon Linux 2022 instead of Amazon Linux 2?

singlewind commented 1 year ago

The reason why I ask this is by default the python version on the container is 3.10.2 and glibc 2.26 which is OK. If we would like to install a newer version like 3.10.9 which depends on glibc 2.34. Amazon linux 2022 has glibc 2.34 already.