search

mongodb-labs / mongo-arrow

MongoDB integrations for Apache Arrow. Export MongoDB documents to numpy array, parquet files, and pandas dataframes in one line of code.
https://mongo-arrow.readthedocs.io
Apache License 2.0
90 stars 14 forks source link

Add an optional bool flag to the `write` function to skip writing `null` fields #210

Open alessio-locatelli opened 6 months ago

alessio-locatelli commented 6 months ago

Function parameters example

def write(collection, tabular, *, exclude_none: bool = False):
    ...

Usage example

write(collection, df, exclude_none=True)

How

Replacing https://github.com/mongodb-labs/mongo-arrow/blob/main/bindings/python/pymongoarrow/api.py#L390 with

if exclude_none:
    yield {k:v for k, v in row.items() if v is not None}
else:
    yield row

did the job.

blink1073 commented 6 months ago

Hi @olk-m! I've opened https://jira.mongodb.org/browse/ARROW-233 to track the request.