Closed benfred closed 1 year ago
Importing pyarrow on systems with a pagesize bigger than 16K causes an issue on importing pyarrow:
In [2]: import pyarrow <jemalloc>: Unsupported system page size
This is on a system with a pagesize of 64K.
# getconf -a | grep PAGESIZE PAGESIZE 65536
The jemalloc in arrow is compiled with --with-lg-page=14 in this feedstock, which corresponds to a maximum pagesize of 16K.
--with-lg-page=14
The 16K pagesize was added for support to M1 Macs by @xhochy, as detailed in this post: https://uwekorn.com/2021/01/11/apache-arrow-on-the-apple-m1.html , and corresponding PR https://github.com/conda-forge/arrow-cpp-feedstock/pull/253
Changing to --with-lg-page=16 here would allow systems with a 64K pagesize to use the arrow from this conda feedstock without error, and will still work correctly for systems with a smaller pagesize.
--with-lg-page=16
(collapsed some of the longer debug info sections with <details>...</details> to improve readability)
<details>...</details>
Could you open a PR?
Solution to issue cannot be found in the documentation.
Issue
Importing pyarrow on systems with a pagesize bigger than 16K causes an issue on importing pyarrow:
This is on a system with a pagesize of 64K.
The jemalloc in arrow is compiled with
--with-lg-page=14
in this feedstock, which corresponds to a maximum pagesize of 16K.The 16K pagesize was added for support to M1 Macs by @xhochy, as detailed in this post: https://uwekorn.com/2021/01/11/apache-arrow-on-the-apple-m1.html , and corresponding PR https://github.com/conda-forge/arrow-cpp-feedstock/pull/253
Changing to
--with-lg-page=16
here would allow systems with a 64K pagesize to use the arrow from this conda feedstock without error, and will still work correctly for systems with a smaller pagesize.Environment info