Potential Enhancement: "configure" to intercept calls to read_csv, read_parquet, etc...

gage-russell / pandas-lineage

Apache License 2.0

13 stars 6 forks source link

Open alexcatanzaro opened 2 years ago

alexcatanzaro commented 2 years ago

As things stand, you must import read_csv, read_parquet, etc ... It could be beneficial to have an alternative usage via a configure function.

This would look something like :

from pandas-lineage import configure

configure()

At this point calls to pandas.to_csv, to_parquet, etc... would be intercepted and performed with the pandas-lineage counterpart.

gage-russell commented 2 years ago

Something to consider: https://modal.com/