Add support for AWS S3 as a Iceberg table location through s3a:// as a filesystem to store Iceberg data and metadata.
Included are updates to pre-package the core-site.xml in the correct location so that AWS IAM Role policies will work, regardless to if they are pulling from an EC2 instance IAM Profile, Environment Variables, a credential file or Kubernetes IRSA Web Token authentication.
Thanks @Ralnoc , but I don't think we want to add the Hadoop dependencies here. If we want to add S3 support, it should be through iceberg-aws and the S3FileIO implementation to avoid dragging in Hadoop.
Add support for AWS S3 as a Iceberg table location through s3a:// as a filesystem to store Iceberg data and metadata.
Included are updates to pre-package the core-site.xml in the correct location so that AWS IAM Role policies will work, regardless to if they are pulling from an EC2 instance IAM Profile, Environment Variables, a credential file or Kubernetes IRSA Web Token authentication.