apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
35.24k stars 13.77k forks source link

Added chunk_size parameter to LocalFilesystemToGCSOperator #40379

Closed jsjasonseba closed 1 week ago

jsjasonseba commented 1 week ago

closes: https://github.com/apache/airflow/issues/38958 related: https://github.com/apache/airflow/issues/38958

This PR added a chunk_size parameter to the LocalFilesystemToGCSOperator. This parameter is already implemented in the GCSHook.upload function. This change ensures that the chunk_size is passed to the upload function.


^ Add meaningful description above Read the Pull Request Guidelines for more information. In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed. In case of a new dependency, check compliance with the ASF 3rd Party License Policy. In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

boring-cyborg[bot] commented 1 week ago

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst) Here are some useful points:

potiuk commented 1 week ago

Some spell-check issue.

jsjasonseba commented 1 week ago

It seems like the spell-check detects 'KiB' as an invalid term, even tough it should be valid. I have removed it.

potiuk commented 1 week ago

There is a list of allowed words. KiB is not on it. you can add it there, or you can KiB quote the word to avoid spellchecking it.

jsjasonseba commented 1 week ago

I wasn't aware of the list of allowed words. Let me add it to the list. Thanks for letting me know.

boring-cyborg[bot] commented 1 week ago

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

potiuk commented 1 week ago

And merged :)