Open cmdlineluser opened 9 months ago
I just got bit by this too, pl.scan_parquet(f.name).with_columns(pl.col(pl.Utf8).str.strip_chars()).sink_parquet(f.name)
.
The seek_len function feels sketchy at best in an asynchronous context to determine the file size.
On https://github.com/rust-lang/rust/issues/59359 there's mention of using file.metadata().len()
to get the size of a file without requiring a mutable borrow, maybe that's a better fit here?
I found an older discussion of this, and it does make sense that you wouldn't be able to write to the same file you're streaming from - it's already open. I do think the error message could be improved here.
Checks
[X] I have checked that this issue has not already been reported.
[X] I have confirmed this bug exists on the latest version of Polars.
Reproducible example
Log output
No response
Issue description
Just noticed Polars lets you use the same filename here, but it ends up truncating the file.
Not sure if this is supposed to be allowed or not, but if not, it should raise and leave the input intact?
Expected behavior
"Work" or raise exception and leave input intact.
Installed versions