search

narwhals-dev / narwhals

Lightweight and extensible compatibility layer between dataframe libraries!
https://narwhals-dev.github.io/narwhals/
MIT License
429 stars 76 forks source link

[Enh]: Add support for DataFrame.write_parquet #284

Closed Priyansh121096 closed 3 months ago

Priyansh121096 commented 3 months ago

We would like to learn about your use case. For example, if this feature is needed to adopt Narwhals in an open source project, could you please enter the link to it below?

No response

Please describe the purpose of the new feature or describe the problem to solve.

I have a library which lets users ingest a pandas dataframe to a datalake. The dataframe provided to the lib by the user is converted to a parquet file (with pyarrow) before putting it in an ETL pipeline. I'd like to let users ingest polars dataframes as well via the same library and would like to use narwhals for the implementation, if possible.

Suggest a solution if possible.

For polars, nw.DataFrame.write_parquet would be the same as pl.DataFrame.write_parquet. For pandas, nw.DataFrame.write_parquet would be pd.DataFrame.to_parquet.

If you have tried alternatives, please describe them below.

No response

Would you want to open a pull request?

no

Additional information that may help us understand your needs.

No response

MarcoGorelli commented 3 months ago

hey @Priyansh121096 !

Seems like a good request to me! I think that reading data is probably out-of-scope for Narwhals - but writing, that should be in-scope

MarcoGorelli commented 3 months ago

this should be available in the new version, pls let us know if it doesn't work for you, thanks!