search

Alluxio / alluxio

Alluxio, data orchestration for analytics and machine learning in the cloud
https://www.alluxio.io
Apache License 2.0
6.87k stars 2.94k forks source link

Rename property key alluxio.underfs.s3.threads.max #18670

Closed apc999 closed 3 months ago

apc999 commented 3 months ago

What changes are proposed in this pull request?

Rename property key alluxio.underfs.s3.threads.max to alluxio.underfs.s3.connections.max and bump the default number to 1024

Why are the changes needed?

The property alluxio.underfs.s3.threads.max has a misleading name, as it actually represents the size of a connection pool rather than the max number of threads. Consequently, the value of this property is typically much larger (e.g., Trino uses 256 for the equivalent property to ensure sufficient bandwidth when reading from S3).

However, including "threads" in the name causes some users to worry, as they believe it should be calculated based on the number of vCores (as we found in one PoC that users were unwilling to set this value large) .

This commit changes the name but retains the original property name as an alias to maintain backward compatibility.

Does this PR introduce any user facing changes?

Yes but backwards compatible

Xenorith commented 3 months ago

alluxio-bot, force-merge this please