BmcDataStore -> createDirectory is not threadsafe. throwing java.io.IOException: Unable to put object

[piyushg]

https://github.com/oracle/oci-hdfs-connector/blob/master/hdfs-connector/src/main/java/com/oracle/bmc/hdfs/store/BmcDataStore.java

BmcDataStore -> createDirectory is not thread safe and throwing IOException. when using spark to store dataset to object storage using this connector we are hitting across this issue. data access to object storage through connector should be thread safe.

Stacktrace: ExceptionStack_1.txt

Code reference from where exception is getting thrown:

/**

• Creates a pseudo directory at the given path.
• @param path
• The path to create a directory object at.
• @throws IOException

• if the operation cannot be completed. */ public void createDirectory(final Path path) throws IOException { // nothing to do for the "root" directory if (this.isRootDirectory(path)) { LOG.debug("Root directory specified, nothing to create"); return; }

     final String key = this.pathToDirectory(path);
  
     LOG.debug("Attempting to create directory {} with object {}", path, key);
  
     final ByteArrayInputStream bais = new ByteArrayInputStream(new byte[0]);
     final PutObjectResponse response;
     try {
         response = this.objectStorage.putObject(this.requestBuilder.putObject(key, bais, 0L));
         this.statistics.incrementWriteOps(1);
         LOG.debug("Created directory at {} with etag {}", path, response.getETag());
     } catch (final BmcException e) {
         // if running jobs in parallel, it's possible multiple threads try to
         // create the directory at the same time, which might lead to 409 conflicts.
         // also allowing 412 (even though we don't set im/inm headers) as we
         // basically just want to ensure the placeholder directory object exists
         if (e.getStatusCode() != 409 && e.getStatusCode() != 412) {
             LOG.debug("Failed to create directory for {}", key, e);
             throw new IOException("Unable to put object", e);              <-------- ERROR line ------>
         }
         LOG.debug(
                 "Exception while creating directory, ignoring {} {}",
                 e.getStatusCode(),
                 e.getMessage());
     }

  }

[andy-miles]

Thanks for reporting this issue. We'll look into reproducing this issue and addressing it.

[sumitkdey]

Per the attached callstack, the createDirectory operation failed due to the service returning the below failure. Caused by: com.oracle.bmc.model.BmcException: (404, ObjectNotFound, false) null (opc-request-id: 951de845-9b75-c5b1-7074-0a9fa1a51e3b) The service here returns a 404, ObjectNotFound error. The connector does not attempt to detect, prevent or serialize access to objects from multiple threads. It is the responsibility of the caller to ignore expected failures (ex createDirectory failure if the parent directory was deleted by a separate thread).