Create python bindings

[jimexist]

i guess this can be combined in datafusion-python repo

[matthewmturner]

@Jimexist that would certainly decrease time to market. In the short run I think its a great idea.

and just to confirm, are you referring to make it part of the same datafusion python package? or a separate package like datafusion-s3?

@seddonm1 any thoughts?

[seddonm1]

Sorry I have zero knowledge of the Python bindings.

The only benefit of separating it from the datafusion-python repo may be if users don't want to install certain components (for example a GCP user may not need S3)?

[jimexist]

something we can consider is to make sure this repo's python API is compatible with the one from datafusion-python, and then allow it to be installed via extras:

• https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#optional-dependencies
• https://python-poetry.org/docs/pyproject/#extras

In this way, users can do:

pip install datafusion datafusion[s3] datafusion[gcp]

[matthewmturner]

@Jimexist i think thats a good idea. I will look into it.

[chitralverma]

+1 for this!

[schapala-hm]

+1 for this