search

Alluxio / alluxio

Alluxio, data orchestration for analytics and machine learning in the cloud
https://www.alluxio.io
Apache License 2.0
6.87k stars 2.94k forks source link

Fix error caused by Jersey version mismatch on enabling webhdfs schema #18701

Open iwasakims opened 1 month ago

iwasakims commented 1 month ago

What changes are proposed in this pull request?

Fix the following error caused by mismatch of Jersey version between Alluxio and Hadoop when mounting HDFS by webhdfs:// schema (enabled by adding webhdfs:// and swebhdfs:// to alluxio.underfs.hdfs.prefixes).

java.lang.LinkageError: ClassCastException: attempting to castjar:file:/usr/lib/alluxio/assembly/server/target/alluxio-assembly-server-2.9.3-jar-with-dependencies.jar!/javax/ws/rs/ext/RuntimeDelegate.classtojar:file:/usr/lib/alluxio/lib/alluxio-underfs-hdfs-3.3.1-2.9.3.jar!/javax/ws/rs/ext/RuntimeDelegate.class
        at javax.ws.rs.ext.RuntimeDelegate.findDelegate(RuntimeDelegate.java:116)
        at javax.ws.rs.ext.RuntimeDelegate.getInstance(RuntimeDelegate.java:91)
        at javax.ws.rs.core.MediaType.<clinit>(MediaType.java:44)
        at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.jsonParse(WebHdfsFileSystem.java:482)
        at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$FsPathResponseRunner.getResponse(WebHdfsFileSystem.java:956)
        at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:815)
        at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:637)
        at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:675)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
        at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:671)
        at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getHdfsFileStatus(WebHdfsFileSystem.java:1104)
        at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getFileStatus(WebHdfsFileSystem.java:1115)
        at org.apache.hadoop.fs.FileSystem.isDirectory(FileSystem.java:1777)
        at alluxio.underfs.hdfs.HdfsUnderFileSystem.isDirectory(HdfsUnderFileSystem.java:500)
        at alluxio.underfs.UnderFileSystemWithLogging$31.call(UnderFileSystemWithLogging.java:712)
        at alluxio.underfs.UnderFileSystemWithLogging$31.call(UnderFileSystemWithLogging.java:709)
        at alluxio.underfs.UnderFileSystemWithLogging.call(UnderFileSystemWithLogging.java:1262)
        at alluxio.underfs.UnderFileSystemWithLogging.isDirectory(UnderFileSystemWithLogging.java:709)
        at alluxio.master.file.DefaultFileSystemMaster.prepareForMount(DefaultFileSystemMaster.java:3398)
        ...

Why are the changes needed?

webhdfs:// and swebhdfs:// need the same fix as #14950.

Does this PR introduce any user facing changes?

n/a

alluxio-bot commented 1 month ago

Thank you for your pull request. In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement (CLA). It's all electronic and will take just a few minutes. Please download CLA form here, sign, and e-mail back to cla@alluxio.org