search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
37.13k stars 14.31k forks source link

Support specify s3 object ACL args for remote logging #8917

Open kuhnhu opened 4 years ago

kuhnhu commented 4 years ago

Description Support specify s3 object ACL args for remote logging, like 'bucket-owner-full-control'

Use case / motivation We are deploying an airflow server in the client's AWS server, using EC2 instance role(cross account) send logging to our S3 bucket. We want to specify ACL arg "bucket-owner-full-control" to grant owner permission on logs.

Related Issues

boring-cyborg[bot] commented 4 years ago

Thanks for opening your first issue here! Be sure to follow the issue template!

RNHTTR commented 1 year ago

I suspect this is already possible given S3Hook has the acl_policy parameter for many of its methods.

Taragolis commented 1 year ago

I suspect this is already possible given S3Hook has the acl_policy parameter for many of its methods.

Yep parameter in hook exists but remote logging still do not have configurations options and implementations for that ¯\_(ツ)_/¯

update: Since Airflow 2.7 we have an options for create documented configurations for providers, might be this is a good time to move all provider specific remote logging options to the providers, and keep only remote-logging and remote-base-log-folder and maybe some other in [logging]. @eladkal @potiuk WDYT?

@o-nikolas maybe also could interested because this particular one affected Amazon Provider.