search

apache / arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics
https://arrow.apache.org/
Apache License 2.0
14.65k stars 3.55k forks source link

Error compiling cython file when installing pyarrow #11581

Open aniket-l opened 3 years ago

aniket-l commented 3 years ago

I am trying to install streamlit (which requires pyarrow as dependency) through Yocto and Bitbake. For Pyarrow installation, I am getting cython compilation errors

PFA the Pyarrow bitbake recipe as well as the error logs (log.do_compile) error_logs.txt python3-pyarrow_5.0.0.txt

I am new to Cython as well as Yocto and don't understand the source of the errors. Would like some help with debugging the same. Any suggestions are welcome.

Thanks, Aniket Limaye

bkmgit commented 3 years ago

Would version 6.0.0 work for you? https://pypi.org/project/pyarrow/ Does pip3 install pyarrow work on its own?

aniket-l commented 3 years ago

pip3 install pyarrow does seem to work but we want to enable pyarrow out of box on the target. We are using Yocto and bitbake to compile python packages for the target system. We want to fix this through the bitbake recipe itself but I don't know what the root cause of the errors is.

bkmgit commented 3 years ago

Ok. Found documentation for bitbake https://docs.yoctoproject.org/bitbake/index.html Do you have an example of a minimal failing recipe?

If you need to use Conda, perhaps try nightly build https://anaconda.org/arrow-nightlies/pyarrow

Streamlit has a repository https://github.com/streamlit/streamlit - does this work for you on commonly available hardware? It seems you are building on aarch 64 - there are some CI tests for this. So probably your setup may need to be adapted to work correctly.

aniket-l commented 3 years ago

The txt file attached in the original post here is the recipe for pyarrow that is failing (python3-pyarrow_5.0.0.txt).

westonpace commented 3 years ago

Have you tried updating the cython version? Do you know what version of cython you are using?

aniket-l commented 3 years ago

I am using version 0.29.14, which as I understand satisfies the requirement (>=0.29) Could be a problem with the cython recipe as well? Attaching for reference: cython_recipe.zip

jorisvandenbossche commented 2 years ago

@aniket-l sorry for the slow follow-up. Do you still have the issue, or were you able to fix it?