Open slobodan-ilic opened 1 year ago
Which version of pyarrow are you using? I'm using 13.0 in my macos and the file generated successfully...(Though it's a bit large and writing the file takes severo seconds)
I'm using pyarrow: 13.0.0
, on MacOS Monterey 12.6, MacBook Pro 2018.
I'll try it on couple of different machines and report back the results. Wanted to do that anyways, but you beat me to it @mapleFU .
Hope some detail infomation or reproduce method. I cannot help if not have enough info...
Which version of pyarrow are you using? I'm using 13.0 in my macos and the file generated successfully...(Though it's a bit large and writing the file takes severo seconds)
I tried it on a different machine and it's not producing a segfault
. Weird that I can only repro it on my personal machine. I'll try a couple more and get back to you. Can you let me know which OS you're using, and what's your python version (for which you tried the ☝️ example)?
My environment on which it failed was python 3.11.3, but also 3.10.6...
@slobodan-ilic I'm a little busy these days, sorry for late reply. when segfault happens, would you mind catch a stack or coredump file? It will helps a lot.
If still not find, I'll run a script and loop the write_table
for multiple times this weekend. Currently I'm a little busy so don't have enough bindwidth to reproduce the problem...
Describe the bug, including details regarding any error messages, version, and platform.
When trying to export a portion of real survey data from our system, we encountered a segfault when invoking
pq.write_table
. Granted that there might be problem with data, the function still shouldn't fail in such a way, but rather with a message. Here's the code snippet that reproduces the issue, together with 4 relevantnpy
files. I wasn't able to reproduce it in an explicit manner by generating data manually (this is a slice of real data that produced this error).Here's the code:
The relevant
npy
files are zipped in the archive here attached:data.zip
Lmk if you need any clarifications. I'll try generating a minimal example, but this is what I've got for now.
Component(s)
Parquet, Python