I install pyarrow ver 0.2.post because that's what I was able to get working on my machine.
[steven.simpson@ip-10-20-74-154 ~]$ sudo su
[root@ip-10-20-74-154 steven.simpson]# /mnt/var/lib/anaconda2/bin/conda install -c conda-forge pyarrow=0.2.post
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata .........
Solving package specifications: ..........
Package plan for installation in environment /mnt/var/lib/anaconda2:
The following packages will be downloaded:
package | build
---------------------------|-----------------
conda-env-2.6.0 | 0 1017 B conda-forge
jemalloc-4.4.0 | 0 2.3 MB conda-forge
parquet-cpp-1.1.0pre | 1 1.6 MB conda-forge
ruamel_yaml-0.11.14 | py27_0 420 KB conda-forge
arrow-cpp-0.3.pre | np111py27_1 1.3 MB conda-forge
conda-4.2.13 | py27_0 374 KB conda-forge
pyarrow-0.2.post | np111py27_0 3.3 MB conda-forge
------------------------------------------------------------
Total: 9.2 MB
The following NEW packages will be INSTALLED:
arrow-cpp: 0.3.pre-np111py27_1 conda-forge
jemalloc: 4.4.0-0 conda-forge
parquet-cpp: 1.1.0pre-1 conda-forge
pyarrow: 0.2.post-np111py27_0 conda-forge
The following packages will be UPDATED:
conda: 4.1.6-py27_0 --> 4.2.13-py27_0 conda-forge
conda-env: 2.5.1-py27_0 --> 2.6.0-0 conda-forge
ruamel_yaml: 0.11.7-py27_0 --> 0.11.14-py27_0 conda-forge
Proceed ([y]/n)? y
Fetching packages ...
conda-env-2.6. 100% |######################################################################| Time: 0:00:00 1.40 MB/s
jemalloc-4.4.0 100% |######################################################################| Time: 0:00:01 2.25 MB/s
parquet-cpp-1. 100% |######################################################################| Time: 0:00:00 1.97 MB/s
ruamel_yaml-0. 100% |######################################################################| Time: 0:00:00 911.23 kB/s
arrow-cpp-0.3. 100% |######################################################################| Time: 0:00:00 1.63 MB/s
conda-4.2.13-p 100% |######################################################################| Time: 0:00:00 659.74 kB/s
pyarrow-0.2.po 100% |######################################################################| Time: 0:00:00 3.72 MB/s
Extracting packages ...
[ COMPLETE ]|#########################################################################################| 100%
Unlinking packages ...
[ COMPLETE ]|#########################################################################################| 100%
Linking packages ...
[ COMPLETE ]|#########################################################################################| 100%
When I try to import pyarrow I get a libarrow_io.so error
[steven.simpson@ip-10-20-74-154 ~]$ python
Python 2.7.12 |Anaconda 4.1.1 (64-bit)| (default, Jul 2 2016, 17:42:40)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import pyarrow
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/mnt/var/lib/anaconda2/lib/python2.7/site-packages/pyarrow/__init__.py", line 28, in <module>
import pyarrow.config
ImportError: libarrow_io.so: cannot open shared object file: No such file or directory
>>> quit()
[steven.simpson@ip-10-20-74-154 ~]$ sudo su
[root@ip-10-20-74-154 steven.simpson]# /mnt/var/lib/anaconda2/bin/conda install -c conda-forge pyarrow
Fetching package metadata .........
Solving package specifications: ..........
Package plan for installation in environment /mnt/var/lib/anaconda2:
The following packages will be downloaded:
package | build
---------------------------|-----------------
pyarrow-0.3.pre | np111py27_0 2.3 MB conda-forge
The following packages will be UPDATED:
pyarrow: 0.2.post-np111py27_0 conda-forge --> 0.3.pre-np111py27_0 conda-forge
Proceed ([y]/n)? yes
Fetching packages ...
pyarrow-0.3.pr 100% |######################################################################| Time: 0:00:01 2.25 MB/s
Extracting packages ...
[ COMPLETE ]|#########################################################################################| 100%
Unlinking packages ...
[ COMPLETE ]|#########################################################################################| 100%
Linking packages ...
[ COMPLETE ]|#########################################################################################| 100%
[root@ip-10-20-74-154 steven.simpson]# exit
Then try to rerun import pyarrow in python interpreter and get a Numpy related error
[steven.simpson@ip-10-20-74-154 ~]$ python
Python 2.7.12 |Anaconda 4.1.1 (64-bit)| (default, Jul 2 2016, 17:42:40)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import pyarrow
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/mnt/var/lib/anaconda2/lib/python2.7/site-packages/pyarrow/__init__.py", line 28, in <module>
import pyarrow._config
ImportError: /mnt/var/lib/anaconda2/lib/python2.7/site-packages/pyarrow/_config.so: undefined symbol: _ZN5arrow2py9InitNumPyEv
What's weird and frustrating is that I can get pyarrow working on my local conda env:
Conda Environment on EC2
I install pyarrow ver 0.2.post because that's what I was able to get working on my machine.
When I try to
import pyarrow
I get alibarrow_io.so
errorSo I try to update pyarrow to
0.3.pre
, which I've had trouble with beforeThen try to rerun
import pyarrow
in python interpreter and get a Numpy related errorWhat's weird and frustrating is that I can get pyarrow working on my local conda env: