Parameter --awsCredentialsProvider doesn't work

[alberttwong]

[root@spark-hudi auxjars]# java -jar utilities-0.1.0-SNAPSHOT-bundled.jar --datasetConfig onetable.yaml --awsCredentialsProvider MinioEnvironmentProvider
usage: onetable.jar [-c <arg>] -d <arg> [-h] [-i <arg>] [-p <arg>]
 -c,--clientsConfig <arg>          The path to a yaml file containing
                                   OneTable client configurations. These
                                   configs will override the default
 -d,--datasetConfig <arg>          The path to a yaml file containing
                                   dataset configuration
 -h,--help                         Displays help information to run this
                                   utility
 -i,--icebergCatalogConfig <arg>   The path to a yaml file containing
                                   Iceberg catalog configuration. The
                                   configuration will be used for any
                                   Iceberg source or target.
 -p,--hadoopConfig <arg>           Hadoop config xml file path containing
                                   configs necessary to access the file
                                   system. These configs will override the
                                   default configs.

[ashvina]

Hi @alberttwong.

Support for awsCredentialsProvider was deprecated in favor of a more generic config input based on hadoopConfig in PR #5. If you haven't already, could you please try setting the Hadoop configuration properties fs.s3.aws.credentials.provider or fs.s3a.aws.credentials.provider with the appropriate credentials provider class and confirm whether this solution works for you?

      conf.set("fs.s3.impl", "org.apache.hadoop.fs.s3a.S3AFileSystem");
      conf.set("fs.s3.aws.credentials.provider", awsCredentialsProvider);
      conf.set("fs.s3a.impl", "org.apache.hadoop.fs.s3a.S3AFileSystem");
      conf.set("fs.s3a.aws.credentials.provider", awsCredentialsProvider);

The Readme was not updated correctly earlier and could be fixed now.