search

mongodb-labs / mongo-arrow

MongoDB integrations for Apache Arrow. Export MongoDB documents to numpy array, parquet files, and pandas dataframes in one line of code.
https://mongo-arrow.readthedocs.io
Apache License 2.0
88 stars 14 forks source link

undefined symbol: _ZN5arrow6StatusC1ENS_10StatusCodeERKSs with airflow 2.8.1 #207

Open alexisvannier opened 6 months ago

alexisvannier commented 6 months ago

I'm reproducing a bug in airflow with the docker-compose method to run airflow2.8.1 with python 3.11 ( https://airflow.apache.org/docs/apache-airflow/2.8.1/howto/docker-compose/index.html#fetching-docker-compose-yaml ).

I'm creating a requirements.txt with the following packages : 

pymongo==4.6.1
pymongo[srv]==4.6.1
pymongoarrow==1.2.0
pandas==2.1.4

After starting the airflow services, each tasks containing a pymongoarrow reference return the following error :

from pymongoarrow.monkey import patch_all
/home/airflow/.local/lib/python3.11/site-packages/pymongoarrow/__init__.py:27: UserWarning: Could not find compiled pymongoarrow.lib extension, please install from source or report the following traceback on the issue tracker:
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.11/site-packages/pymongoarrow/__init__.py", line 25, in <module>
    from pymongoarrow.lib import libbson_version
ImportError: /home/airflow/.local/lib/python3.11/site-packages/pymongoarrow/lib.cpython-311-aarch64-linux-gnu.so: undefined symbol: _ZN5arrow6StatusC1ENS_10StatusCodeERKSs

  warnings.warn(
blink1073 commented 6 months ago

Hi @alexisvannier, we require gcc 12+, I opened https://jira.mongodb.org/browse/ARROW-229 to track adding it to the documentation.