search

aws-samples / aws-glue-samples

AWS Glue code samples
MIT No Attribution
1.42k stars 812 forks source link

Bump up awssdk.version to 1.11.903 to fix java.lang.NoSuchFieldError #116

Closed satyapreddy closed 2 years ago

satyapreddy commented 2 years ago

Issue #, if available:

Description of changes:

With OOTB POM that has 1.11.901</awssdk.version> I was running into following exception when starting the docker container:

Caused by: java.lang.NoSuchFieldError: SERVICE_ID at com.amazonaws.services.s3.AmazonS3Client.createRequest(AmazonS3Client.java:4772) at com.amazonaws.services.s3.AmazonS3Client.createRequest(AmazonS3Client.java:4758) at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1434) at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:1374) at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$verifyBucketExists$1(S3AFileSystem.java:381) at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109) at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)

With 1.11.903</awssdk.version> the issue is resolved and I am able start the docker container and view the details of the completed spark jobs

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

zdf1230 commented 2 years ago

I faced the same issue and I changed to other versions to solve the issue. I tested this change as well. 903 works fine. Hope this can be solved earlier.

moomindani commented 2 years ago

Merged. Thank you for your contribution!