search

bayesflow-org / bayesflow

A Python library for amortized Bayesian workflows using generative neural networks.
https://bayesflow.org/
MIT License
396 stars 53 forks source link

InvertibleNetwork error (Input 0 of layer "dense" is incompatible with the layer) #183

Closed lucasgautheron closed 3 months ago

lucasgautheron commented 3 months ago

Hi,

InvertibleNetwork does not seem to be working for me regardless of the input parameters (for instance, I cannot reproduce any of the examples because of this issue.)

Here is a minimal example exhibiting the issue:

import bayesflow.networks
bayesflow.networks.InvertibleNetwork(num_params=4)

Here is the error I get with this example:

Python 3.12.4 | packaged by conda-forge | (main, Jun 17 2024, 10:11:10) [Clang 16.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import bayesflow.networks
2024-08-06 10:29:28.404900: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.

Failed to import TF-Keras. Please note that TF-Keras is not installed by default when you install TensorFlow Probability. This is so that JAX-only users do not have to install TensorFlow or TF-Keras. To use TensorFlow Probability with TensorFlow, please install the tf-keras or tf-keras-nightly package.
This can be be done through installing the tensorflow-probability[tf] extra.

WARNING:root:Some dependencies failed to load. BayesFlow modules may not work properly!
WARNING:root:No module named 'tf_keras'
>>> bayesflow.networks.InvertibleNetwork(num_params=4)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/anaconda3/envs/stan/lib/python3.12/site-packages/bayesflow/inference_networks.py", line 139, in __init__
    self.coupling_layers = self._create_coupling_layers(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/stan/lib/python3.12/site-packages/bayesflow/inference_networks.py", line 257, in _create_coupling_layers
    CouplingLayer(coupling_design=design, coupling_settings=_coupling_settings, **settings)
  File "/opt/anaconda3/envs/stan/lib/python3.12/site-packages/bayesflow/coupling_networks.py", line 563, in __init__
    self.net1 = coupling_type(self.dim_out1, coupling_settings)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/stan/lib/python3.12/site-packages/bayesflow/coupling_networks.py", line 83, in __init__
    self.scale = DenseCouplingNet(s_settings, dim_out)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/stan/lib/python3.12/site-packages/bayesflow/helper_networks.py", line 93, in __init__
    self.fc.build(input_shape=())
  File "/opt/anaconda3/envs/stan/lib/python3.12/site-packages/keras/src/layers/layer.py", line 225, in build_wrapper
    original_build_method(*args, **kwargs)
  File "/opt/anaconda3/envs/stan/lib/python3.12/site-packages/keras/src/models/sequential.py", line 183, in build
    x = layer(x)
        ^^^^^^^^
  File "/opt/anaconda3/envs/stan/lib/python3.12/site-packages/keras/src/utils/traceback_utils.py", line 122, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/opt/anaconda3/envs/stan/lib/python3.12/site-packages/keras/src/layers/input_spec.py", line 202, in assert_input_compatibility
    raise ValueError(
ValueError: Input 0 of layer "dense" is incompatible with the layer: expected min_ndim=2, found ndim=0. Full shape received: ()

I installed the version 1.1.4. of the package as recommended (pip install bayesflow). It automatically installed tensorflow==2.16.2

Thank you for your help!

LarsKue commented 3 months ago

Hi @lucasgautheron, sorry for the late reply. I cannot reproduce the issue in a fresh environment. Did you follow the install instructions correctly? What version of Python are you using?

It also looks like the warnings in your output are hinting at another (possibly unrelated) issue:

Failed to import TF-Keras. Some dependencies failed to load. No module named 'tf_keras'

Does running pip install tf-keras=2.16 fix the issue? If not, can you please set up a fresh environment using

conda create bf-test python=3.11
conda activate bf-test

and then install the following dependency versions:

pip install bayesflow==1.1.6 tensorflow==2.15.1 tensorflow-probability==0.23.0

If all else fails, here is an environment-lock.yaml file that runs for me without problems:

name: bf-test
channels:
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_gnu
  - bzip2=1.0.8=h4bc722e_7
  - ca-certificates=2024.7.4=hbcca054_0
  - ld_impl_linux-64=2.40=hf3520f5_7
  - libexpat=2.6.2=h59595ed_0
  - libffi=3.4.2=h7f98852_5
  - libgcc-ng=14.1.0=h77fa898_0
  - libgomp=14.1.0=h77fa898_0
  - libnsl=2.0.1=hd590300_0
  - libsqlite=3.46.0=hde9e2c9_0
  - libuuid=2.38.1=h0b41bf4_0
  - libxcrypt=4.4.36=hd590300_1
  - libzlib=1.3.1=h4ab18f5_1
  - ncurses=6.5=h59595ed_0
  - openssl=3.3.1=h4bc722e_2
  - pip=24.2=pyhd8ed1ab_0
  - python=3.11.9=hb806964_0_cpython
  - readline=8.2=h8228510_1
  - setuptools=72.1.0=pyhd8ed1ab_0
  - tk=8.6.13=noxft_h4845f30_101
  - wheel=0.44.0=pyhd8ed1ab_0
  - xz=5.2.6=h166bdaf_0
  - pip:
      - absl-py==2.1.0
      - aesara==2.9.3
      - astunparse==1.6.3
      - bayesflow==1.1.6
      - cachetools==5.4.0
      - certifi==2024.7.4
      - charset-normalizer==3.3.2
      - cloudpickle==3.0.0
      - cons==0.4.6
      - contourpy==1.2.1
      - cycler==0.12.1
      - decorator==5.1.1
      - dm-tree==0.1.8
      - etuples==0.3.9
      - filelock==3.15.4
      - flatbuffers==24.3.25
      - fonttools==4.53.1
      - gast==0.6.0
      - google-auth==2.33.0
      - google-auth-oauthlib==1.2.1
      - google-pasta==0.2.0
      - grpcio==1.65.4
      - h5py==3.11.0
      - idna==3.7
      - joblib==1.4.2
      - keras==2.15.0
      - kiwisolver==1.4.5
      - libclang==18.1.1
      - logical-unification==0.4.6
      - markdown==3.6
      - markupsafe==2.1.5
      - matplotlib==3.9.1.post1
      - minikanren==1.0.3
      - ml-dtypes==0.3.2
      - multipledispatch==1.0.0
      - numpy==1.26.4
      - oauthlib==3.2.2
      - opt-einsum==3.3.0
      - packaging==24.1
      - pandas==2.2.2
      - pillow==10.4.0
      - protobuf==4.25.4
      - pyasn1==0.6.0
      - pyasn1-modules==0.4.0
      - pyparsing==3.1.2
      - python-dateutil==2.9.0.post0
      - pytz==2024.1
      - requests==2.32.3
      - requests-oauthlib==2.0.0
      - rsa==4.9
      - scikit-learn==1.5.1
      - scipy==1.14.0
      - seaborn==0.13.2
      - six==1.16.0
      - tensorboard==2.15.2
      - tensorboard-data-server==0.7.2
      - tensorflow==2.15.1
      - tensorflow-estimator==2.15.0
      - tensorflow-io-gcs-filesystem==0.37.1
      - tensorflow-probability==0.23.0
      - termcolor==2.4.0
      - threadpoolctl==3.5.0
      - toolz==0.12.1
      - tqdm==4.66.5
      - typing-extensions==4.12.2
      - tzdata==2024.1
      - urllib3==2.2.2
      - werkzeug==3.0.3
      - wrapt==1.14.1