With OOTB POM that has 1.11.901</awssdk.version> I was running into following exception when starting the docker container:
Caused by: java.lang.NoSuchFieldError: SERVICE_ID
at com.amazonaws.services.s3.AmazonS3Client.createRequest(AmazonS3Client.java:4772)
at com.amazonaws.services.s3.AmazonS3Client.createRequest(AmazonS3Client.java:4758)
at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1434)
at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:1374)
at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$verifyBucketExists$1(S3AFileSystem.java:381)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)
With 1.11.903</awssdk.version> the issue is resolved and I am able start the docker container and view the details of the completed spark jobs
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
I faced the same issue and I changed to other versions to solve the issue. I tested this change as well. 903 works fine. Hope this can be solved earlier.
Issue #, if available:
Description of changes:
With OOTB POM that has1.11.901</awssdk.version> I was running into following exception when starting the docker container:
Caused by: java.lang.NoSuchFieldError: SERVICE_ID at com.amazonaws.services.s3.AmazonS3Client.createRequest(AmazonS3Client.java:4772) at com.amazonaws.services.s3.AmazonS3Client.createRequest(AmazonS3Client.java:4758) at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1434) at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:1374) at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$verifyBucketExists$1(S3AFileSystem.java:381) at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109) at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)
With1.11.903</awssdk.version> the issue is resolved and I am able start the docker container and view the details of the completed spark jobs
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.