search

conda-forge / pyarrow-feedstock

A conda-smithy repository for pyarrow.
BSD 3-Clause "New" or "Revised" License
6 stars 26 forks source link

Upgrade pyarrow defaults to 0.16.0 #103

Closed damondd closed 4 years ago

damondd commented 4 years ago

Issue:

When install pyarrow in clean linux conda environment (2020.02):

conda install -c conda-forge pyarrow

The following packages will be downloaded:
    package                    |            build
    ---------------------------|-----------------
    arrow-cpp-0.11.1           |py37h0e61e49_1004         6.3 MB  conda-forge
    boost-cpp-1.68.0           |    h11c811c_1000        20.5 MB  conda-forge
    conda-4.8.3                |   py37hc8dfbb8_1         3.0 MB  conda-forge
    libprotobuf-3.6.1          |    hdbcaa40_1001         4.0 MB  conda-forge
    parquet-cpp-1.5.1          |                3           3 KB  conda-forge
    pyarrow-0.11.1             |py37hbbcf98d_1002         2.0 MB  conda-forge
    python_abi-3.7             |          1_cp37m           4 KB  conda-forge
    thrift-cpp-0.12.0          |    h0a07b25_1002         2.4 MB  conda-forge
    ------------------------------------------------------------
                                           Total:        38.2 MB

The default version is pyarrow-0.11.1, while conda repo actually has the latest version 0.16.0 ( https://anaconda.org/conda-forge/pyarrow ).

Specify the version does not help at all:

conda install -c conda-forge pyarrow=0.16.0

Workaround

I have to manually download below packages from conda then install them locally:

arrow-cpp-0.16.0-py37hb0edad2_0.tar.bz2
aws-sdk-cpp-1.7.164-h1f8afcc_0.tar.bz2
boost-cpp-1.70.0-h8e57a91_2.tar.bz2
brotli-1.0.7-he1b5a44_1000.tar.bz2
c-ares-1.15.0-h516909a_1001.tar.bz2
gflags-2.2.2-he1b5a44_1002.tar.bz2
glog-0.4.0-he1b5a44_1.tar.bz2
grpc-cpp-1.25.0-h213be95_2.tar.bz2
libprotobuf-3.11.3-h8b12597_0.tar.bz2
lz4-c-1.8.3-he1b5a44_1001.tar.bz2
parquet-cpp-1.5.1-1.tar.bz2
pyarrow-0.16.0-py37h8b68381_1.tar.bz2
re2-2020.01.01-he1b5a44_0.tar.bz2
snappy-1.1.8-he1b5a44_1.tar.bz2
thrift-cpp-0.12.0-hf3afdfd_1004.tar.bz2
zstd-1.4.4-h3b9ef0a_1.tar.bz2



Details about conda and system ( conda info ):

``` $ conda info active environment : base active env location : /home/dwang/anaconda3 shell level : 1 user config file : /home/dwang/.condarc populated config files : conda version : 4.8.2 conda-build version : 3.18.11 python version : 3.7.6.final.0 virtual packages : __glibc=2.27 base environment : /home/dwang/anaconda3 (writable) channel URLs : https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /home/dwang/anaconda3/pkgs /home/dwang/.conda/pkgs envs directories : /home/dwang/anaconda3/envs /home/dwang/.conda/envs platform : linux-64 user-agent : conda/4.8.2 requests/2.22.0 CPython/3.7.6 Linux/5.0.0-1033-gcp ubuntu/18.04.4 glibc/2.27 UID:GID : 1001:1002 netrc file : None offline mode : False ```
xhochy commented 4 years ago

Please add conda-forge to your channel list using conda config --add channels conda-forge. Your conda info suggest that all other packages are from anaconda/defaults. You cannot mix packages from there with ones from conda-forge. You need to recreate your environment purely with packages from conda-forge to make this work.

Feel free to reopen if other issues appear but this seems to stem from a mix-and-match of incompatible packages. Also please post your conda list in that case.