oracle / oci-hdfs-connector

HDFS Connector for Oracle Cloud Infrastructure
https://cloud.oracle.com/cloud-infrastructure
Other
27 stars 26 forks source link

hadoop fs -cat doesn't work with Instance Principal authenticator #12

Closed xdobiasl closed 5 years ago

xdobiasl commented 5 years ago

I have Hadoop 2.6.0-cdh5.15.1, with OCI HDFS driver 2.7.7.0.

I cannot get content of a file on OCI using:

$ hadoop fs -text oci://bdw-demo-bucket@oraclebigdatadb/pocitani_pismen.txt

I writes an error:

text: Unable to fetch file status for: pocitani_pismen.txt

However, I can list the file, so the driver somehow works:

$ hadoop fs -text oci://bdw-demo-bucket@oraclebigdatadb/pocitani_pismen.txt
-rw-rw-rw-   1        838 2019-01-04 18:21 oci://bdw-demo-bucket@oraclebigdatadb/pocitani_pismen.txt

And I can get the file content using OCI Python SDK. So it is not a permission/policies issue.

In core-site.xml, I have:

  <property>
    <name>fs.oci.client.hostname</name>
    <value>https://objectstorage.us-phoenix-1.oraclecloud.com</value>
  </property>
  <property>
    <name>fs.oci.client.custom.authenticator</name>
    <value>com.oracle.bmc.hdfs.auth.InstancePrincipalsCustomAuthenticator</value>
  </property>

When I run the hadoop with more verbose logging:

HADOOP_ROOT_LOGGER="DEBUG,console" hadoop fs -text oci://bdw-demo-bucket@oraclebigdatadb/pocitani_pismen.txt

I can see error:

19/01/08 16:39:52 DEBUG internal.RequestSignerImpl: Could not sign request
java.lang.SecurityException: class "javax.annotation.Nonnull"'s signer information does not match signer information of other classes in the same package

Full console log from the command: hadoop-fs-text-instance-principal-auth-problem.log

xdobiasl commented 5 years ago

When I used 2.7.7.1, then it works. Closing issue.