search

awslabs / aws-glue-libs

AWS Glue Libraries are additions and enhancements to Spark for ETL operations.
Other
635 stars 299 forks source link

Wrong Spark 3.3.0 distribution for Glue 4.0 #171

Open alvsanand opened 1 year ago

alvsanand commented 1 year ago

It seems that the tgz from https://aws-glue-etl-artifacts.s3.amazonaws.com/glue-4.0/spark-3.3.0-amzn-1-bin-3.3.3-amzn-0.tgz is not right. It lacks from several basic Spark directories like "conf" and it is contained in a build directory in contrast to the other versions.

neilagupta commented 1 year ago

@alvsanand it was updated recently, does this help a bit? conf folder was not added back but there are several templates that can be used from here.