AWSGlueDQ not available in glue 4.0 image

awslabs / aws-glue-libs

AWS Glue Libraries are additions and enhancements to Spark for ETL operations.

Other

635 stars 299 forks source link

AWSGlueDQ not available in glue 4.0 image #200

Open MattHardner opened 7 months ago

RodiScheele commented 3 months ago

We have the same issue. Did anyone find a workaround?

I did have contact with AWS support, they told me it's on their roadmap. So that can mean anything really.

DanielSastoque commented 3 months ago

Hi there, we are using the docker image for local development. We are building a DQ infrastructure that uses heavily the AWS Glue DQ framework. Therefore, not having it available for local testing slows our development process and increases our AWS bills.

Is there any plan to have it in the next releases?

kevzha commented 3 months ago

Hi, Similarly, I was trying to work with the AWSGlueDQ in a local environment but ran into this issue. Was wondering if there is any plans to update this?