search

facebookincubator / velox

A composable and fully extensible C++ execution engine library for data management systems.
https://velox-lib.io/
Apache License 2.0
3.53k stars 1.16k forks source link

Add support for S3 Region #11627

Open majetideepak opened 2 days ago

majetideepak commented 2 days ago

Description

We will follow the Hadoop aws library rules to be consistent with other applications. https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/connecting.html#s3_endpoint_region_details

  1. Configs hive.s3.endpoint and hive.s3.endpoint.region are used to set values for S3 endpoint and region respectively.
  2. If hive.s3.endpoint.region is configured with valid AWS region value, S3FS will use this value.
  3. If hive.s3.endpoint.region is not set and hive.s3.endpoint is set with valid endpoint value, S3FS will attempt to parse the region from the endpoint and configure the client to use the region value.