conda-forge / jax-feedstock

A conda-smithy repository for jax.
BSD 3-Clause "New" or "Revised" License
6 stars 20 forks source link

Version 0.3.0 causes crashes #76

Closed inducer closed 2 years ago

inducer commented 2 years ago

Do the following:

$ curl -L https://raw.githubusercontent.com/pyrometheus/pyrometheus/1a9b3dc9e20b23215728b5627d36504302ca0fbb/.test-conda-env-py3.yml
$ conda env create -n test-jax -f=.test-conda-env-py3.yml  # resulting in env listed below
$ conda activate test-jax

and observe:

$ python
Python 3.10.2 | packaged by conda-forge | (main, Feb  1 2022, 19:29:00) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import jax
/home/andreas/pack/mambaforge/envs/test-pyro/lib/python3.10/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses
  import imp
[libprotobuf ERROR google/protobuf/descriptor_database.cc:641] File already exists in database: tensorflow/compiler/xla/xla_data.proto
[libprotobuf FATAL google/protobuf/descriptor.cc:2021] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size): 
terminate called after throwing an instance of 'google::protobuf::FatalException'
  what():  CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size): 
[1]    172375 IOT instruction (core dumped)  python


Environment (conda list):

``` $ conda list # packages in environment at /home/andreas/pack/mambaforge/envs/test-pyro: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 1_gnu conda-forge abseil-cpp 20211102.0 h9c3ff4c_0 conda-forge absl-py 1.0.0 pyhd8ed1ab_0 conda-forge bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.18.1 h7f98852_0 conda-forge ca-certificates 2021.10.8 ha878542_0 conda-forge cached-property 1.5.2 hd8ed1ab_1 conda-forge cached_property 1.5.2 pyha770c72_1 conda-forge cantera 2.5.1 py310h93d520e_3 conda-forge fmt 8.1.1 h4bd325d_0 conda-forge gmp 6.2.1 h58526e2_0 conda-forge grpc-cpp 1.44.0 h1021d7f_0 conda-forge h5py 3.6.0 nompi_py310he751f51_100 conda-forge hdf5 1.12.1 nompi_h7f166f4_103 conda-forge jax 0.3.0 pyhd8ed1ab_0 conda-forge jaxlib 0.3.0 py310h0e91589_2 conda-forge krb5 1.19.2 h48eae69_3 conda-forge ld_impl_linux-64 2.36.1 hea4e1c9_2 conda-forge libblas 3.9.0 13_linux64_openblas conda-forge libcantera 2.5.1 haf93dff_3 conda-forge libcblas 3.9.0 13_linux64_openblas conda-forge libcurl 7.81.0 h494985f_0 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libffi 3.4.2 h7f98852_5 conda-forge libgcc-ng 11.2.0 h1d223b6_12 conda-forge libgfortran-ng 11.2.0 h69a702a_12 conda-forge libgfortran5 11.2.0 h5c6108e_12 conda-forge libgomp 11.2.0 h1d223b6_12 conda-forge liblapack 3.9.0 13_linux64_openblas conda-forge libnghttp2 1.46.0 ha19adfc_0 conda-forge libnsl 2.0.0 h7f98852_0 conda-forge libopenblas 0.3.18 pthreads_h8fe5266_0 conda-forge libprotobuf 3.19.4 h780b84a_0 conda-forge libssh2 1.10.0 ha35d2d1_2 conda-forge libstdcxx-ng 11.2.0 he4da1e4_12 conda-forge libuuid 2.32.1 h7f98852_1000 conda-forge libzlib 1.2.11 h36c2ea0_1013 conda-forge metis 5.1.0 h58526e2_1006 conda-forge mpfr 4.1.0 h9202a9a_1 conda-forge ncurses 6.3 h9c3ff4c_0 conda-forge numpy 1.22.2 py310h454958d_0 conda-forge openssl 3.0.0 h7f98852_2 conda-forge opt_einsum 3.3.0 pyhd8ed1ab_1 conda-forge pip 22.0.3 pyhd8ed1ab_0 conda-forge python 3.10.2 hc74c709_3_cpython conda-forge python-flatbuffers 2.0 pyhd8ed1ab_0 conda-forge python_abi 3.10 2_cp310 conda-forge re2 2022.02.01 h9c3ff4c_0 conda-forge readline 8.1 h46c0cb4_0 conda-forge ruamel_yaml 0.15.80 py310h6acc77f_1006 conda-forge scipy 1.8.0 py310hea5193d_1 conda-forge setuptools 60.9.3 py310hff52083_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge sqlite 3.37.0 h9cd32fc_0 conda-forge suitesparse 5.10.1 h9e50725_1 conda-forge sundials 5.7.0 h558c624_0 conda-forge tbb 2021.5.0 h4bd325d_0 conda-forge tk 8.6.12 h27826a3_0 conda-forge typing_extensions 4.1.1 pyha770c72_0 conda-forge tzdata 2021e he74cb21_0 conda-forge wheel 0.37.1 pyhd8ed1ab_0 conda-forge xz 5.2.5 h516909a_1 conda-forge yaml 0.2.5 h7f98852_2 conda-forge yaml-cpp 0.6.3 he1b5a44_4 conda-forge zlib 1.2.11 h36c2ea0_1013 conda-forge ```


Details about conda and system ( conda info ):

$ conda info

     active environment : test-pyro
    active env location : /home/andreas/pack/mambaforge/envs/test-pyro
            shell level : 2
       user config file : /home/andreas/.condarc
 populated config files : /home/andreas/pack/mambaforge/.condarc
                          /home/andreas/.condarc
          conda version : 4.11.0
    conda-build version : not installed
         python version : 3.9.7.final.0
       virtual packages : __linux=5.16.0=0
                          __glibc=2.33=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/andreas/pack/mambaforge  (writable)
      conda av data dir : /home/andreas/pack/mambaforge/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/andreas/pack/mambaforge/pkgs
                          /home/andreas/.conda/pkgs
       envs directories : /home/andreas/pack/mambaforge/envs
                          /home/andreas/.conda/envs
               platform : linux-64
             user-agent : conda/4.11.0 requests/2.26.0 CPython/3.9.7 Linux/5.16.0-1-amd64 debian/testing glibc/2.33
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False
inducer commented 2 years ago

Only pinning to very specific past versions results in no crashes for me:

https://github.com/pyrometheus/pyrometheus/pull/40/commits/0b23396dd9f9a10cd741248d468374461bac99ef

mattwthompson commented 2 years ago

I'm getting cryptic crashes, though not the same ones you are. The problem may be in jaxlib, probably this one: https://github.com/conda-forge/jaxlib-feedstock/issues/89

xhochy commented 2 years ago

Should be fixed now, can you try updating jaxlib and see whether the crash still exists?

mattwthompson commented 2 years ago

Looks fixed to me - I checked my macOS logs to make sure version 0.3.0 or newer is pulled down and it's pulled down from conda.

inducer commented 2 years ago

Just to confirm, it's this PR that would have contained the fix, correct? https://github.com/conda-forge/jaxlib-feedstock/pull/90

xhochy commented 2 years ago

Just to confirm, it's this PR that would have contained the fix, correct? conda-forge/jaxlib-feedstock#90

Yes!

xhochy commented 2 years ago

Also https://github.com/conda-forge/admin-requests/pull/401 marked the previous builds as broken.

ngam commented 2 years ago

This seems to have been resolved, so closing. Please feel free to reopen or open a new issue. Thanks for reporting :)