Nike-Inc / koheesio

Python framework for building efficient data pipelines. It promotes modularity and collaboration, enabling the creation of complex pipelines from simple, reusable components.

https://engineering.nike.com/koheesio/

Apache License 2.0

602 stars 29 forks source link

[FEATURE] Enable adding options to DeltaReader both streaming and writing #111

Closed mikita-sakalouski closed 3 days ago

mikita-sakalouski commented 4 days ago

Description

Introduce private attributes for batch and stream readers to enable adding options to DeltaReader both streaming and writing.

Related Issue

110

Motivation and Context

Provide possibility to override readers, e.g. add more options to readers.

How Has This Been Tested?

Current tests

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.