search

oracle / oci-hdfs-connector

HDFS Connector for Oracle Cloud Infrastructure
https://cloud.oracle.com/cloud-infrastructure
Other
27 stars 26 forks source link

Potential data corruption issue for OCI HDFS Connector with RefreshableOnNotAuthenticatedProvider #35

Open mricken opened 3 years ago

mricken commented 3 years ago

Update 2020-11-03: We believe we have fixed this issue in version 3.2.1.3. If you are using a RefreshableOnNotAuthenticatedProvider (Instance Principals or Resource Principals, for example), we urge you to update to version 3.2.1.3 or later.

Original issue If you are using any of the OCI HDFS Connector and you use a RefreshableOnNotAuthenticatedProvider (e.g. InstancePrincipalsCustomAuthenticator, or generally for Resource Principals or Instance Principals) you may be affected by silent data corruption.

Description

When using a RefreshableOnNotAuthenticatedProvider, the OCI Java SDK will attempt to acquire a new security token when receiving a 401 response, which indicates authentication problems, and then automatically retry the request. In this situation, the clients fail to reset the stream position for requests that upload streams. As a result, an empty stream is successfully uploaded, and success may be returned, even though the actual stream content was not uploaded.

Affected and Not Affected Code

It only happens when an authentication details provider that implements RefreshableOnNotAuthenticatedProvider is used, e.g.:

It does not happen for other authentication details provider that do not implement said interface, e.g.:

Workarounds

We believe we have fixed this issue in OCI Java SDK version 1.25.2. If you are using a RefreshableOnNotAuthenticatedProvider (InstancePrincipalsCustomAuthenticator in particular, or generally Instance Principals or Resource Principals, for example) with the OCI HDFS Connector, we urge you to update to version 1.25.2 of the OCI Java SDK by overriding the version of the OCI Java SDK that the OCI HDFS Connector uses.

We apologize for any inconvenience this issue may cause you. We will release an updated version of the OCI HDFS Connector, which uses the fixed version 1.25.2 of the OCI Java SDK, shortly. Please watch this GitHub issue for updates.

y-chandra commented 3 years ago

We have released a new version of the OCI HDFS Connector i.e. 3.2.1.3 (https://github.com/oracle/oci-hdfs-connector/releases/tag/v3.2.1.3) that uses the fixed version 1.25.2 of the OCI Java SDK.