$ POLARS_VERBOSE=1 python parse.py
polars version: 0.20.10
/private/tmp/polarsbug/parse.py:7: UserWarning: Polars found a filename. Ensure you pass a path to the file instead of a python file object when possible for best performance.
df = pl.read_ndjson(f)
thread '<unnamed>' panicked at /Users/runner/work/polars/polars/crates/polars-io/src/mmap.rs:80:37:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last):
File "/private/tmp/polarsbug/parse.py", line 7, in <module>
df = pl.read_ndjson(f)
File "/Users/llimllib/.local/share/asdf/installs/python/3.10.12/lib/python3.10/site-packages/polars/io/ndjson.py", line 49, in read_ndjson
return pl.DataFrame._read_ndjson(
File "/Users/llimllib/.local/share/asdf/installs/python/3.10.12/lib/python3.10/site-packages/polars/dataframe/frame.py", line 1066, in _read_ndjson
self._df = PyDataFrame.read_ndjson(
pyo3_runtime.PanicException: called `Option::unwrap()` on a `None` value
Issue description
The polars documentation suggests that passing a file object ought to be valid, so it seems to me like the ought to succeed, and at least to throw a useful error instead of panicking if it's not
It works as expected if you replace f with f.read()
the point for me though is that I would not like to read the file into memory - is there any way to read a gzip-compressed ndjson file without loading the whole thing into memory?
Expected behavior
I expect the script to work, and at the very least to fail with a useful error instead of panicking
Checks
Reproducible example
Log output
Issue description
f
withf.read()
Expected behavior
I expect the script to work, and at the very least to fail with a useful error instead of panicking
Installed versions