[Improvement] Update Hadoop version from 3.1 to 3.1 to make fileset catalog up-to-date

apache / gravitino

World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.

https://gravitino.apache.org

Apache License 2.0

1.1k stars 348 forks source link

Open yuqi1129 opened 2 weeks ago

yuqi1129 commented 2 weeks ago

Hadoop 3.1 is very old and lacks some key features.

hadoop-azure 3.1 only support wasbs protocol and DOES not support ADLS(abfs)
hadoop-aws 3.1 does not contain S3 delegation-related feature, which will a hinder for us to use credential mechanism to access S3.

So, I suggest we can update the Hadoop version from 3.1.0 to 3.3.0.

No response