Nike-Inc / koheesio

Python framework for building efficient data pipelines. It promotes modularity and collaboration, enabling the creation of complex pipelines from simple, reusable components.

https://engineering.nike.com/koheesio/

Apache License 2.0

599 stars 19 forks source link

Ability to read Excel files #37

Closed dannymeijer closed 4 months ago

dannymeijer commented 4 months ago

introduces pandas module
abstracted Reader class to a common BaseReader (as part of models module)
re-implemented spark.readers.Reader based on BaseReader
introduces ExcelReader at koheesio.pandas.readers.excel.ExcelReader
introduces ExcelReader at koheesio.spark.reader.excel.ExcelReader
added unittests to cover the above
added excel extra dependency
added docs

Description

Related Issue

3

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

maxim-mityutko commented 4 months ago

Potentially can be integrated with Box. We already have CSV reader there, Excel could be a valuable addition.