minio / docs

MinIO Object Storage Documentation
https://docs.min.io/minio/baremetal
Creative Commons Attribution 4.0 International
547 stars 297 forks source link

Update the Spark/Hive integration page #1278

Open feorlen opened 1 month ago

feorlen commented 1 month ago

This page:

https://min.io/docs/minio/linux/integrations/disaggregated-spark-and-hadoop-hive-with-minio.html

hasn't been updated in years and the actual content apparently predates its existence at this location in the docs. The information is very inaccurate:

those are for HDP 3.1.x which predates the entire CDP releases.
HDP 3.1 and CDH 6.x are equivalent to Apache Hadoop 3.1
CDP 7.1.2+ is (mostly) equivalent to Apache Hadoop 3.2
The recommendations for the Staging committer are still valid (but tuning for a workload is likely required)
CLDR does not provide any guidance for using the Magic committer and recommends staying away from it.

The Docs team doesn't currently have the needed context, so we will need to work closely with eng/customer eng for this.

feorlen commented 1 month ago

Maybe also coordinate with updates to these blog posts: