Issues using the custom neural network models in AI-Economist

[aslansd]

Hello,

I am developing AI-Economist and I am trying to run my own modified version of AI-Economist with your provided custom neural network models: KerasConvLSTM and RandomAction.

Here is my system properties:

Model Name: MacBook Pro Model Identifier: MacBookPro17,1 Chip: Apple M1 Total Number of Cores: 8 (4 performance and 4 efficiency) Memory: 16 GB

Here is the list of modules in my modified ai-economist environment:

Name Version Build Channel

absl-py 1.2.0 pypi_0 pypi aiosignal 1.2.0 pypi_0 pypi appdirs 1.4.4 pypi_0 pypi appnope 0.1.2 py310hca03da5_1001 anaconda argon2-cffi 20.1.0 pypi_0 pypi astroid 2.5.6 pypi_0 pypi asttokens 2.0.5 pyhd3eb1b0_0 anaconda astunparse 1.6.3 pypi_0 pypi async-generator 1.10 pypi_0 pypi attrs 21.2.0 pypi_0 pypi backcall 0.2.0 pyhd3eb1b0_0 anaconda beautifulsoup4 4.9.3 pypi_0 pypi black 22.8.0 pypi_0 pypi bleach 3.3.0 pypi_0 pypi bs4 0.0.1 pypi_0 pypi bzip2 1.0.8 h620ffc9_4 anaconda c-ares 1.18.1 h3422bc3_0 conda-forge ca-certificates 2022.07.19 hca03da5_0
cached-property 1.5.2 hd8ed1ab_1 conda-forge cached_property 1.5.2 pyha770c72_1 conda-forge cachetools 5.2.0 pypi_0 pypi certifi 2020.12.5 pypi_0 pypi cffi 1.14.5 pypi_0 pypi chardet 4.0.0 pypi_0 pypi click 8.0.1 pypi_0 pypi cloudpickle 2.1.0 pypi_0 pypi cycler 0.10.0 pypi_0 pypi debugpy 1.5.1 py310hc377ac9_0 anaconda decorator 5.0.9 pypi_0 pypi defusedxml 0.7.1 pypi_0 pypi distlib 0.3.6 pypi_0 pypi dm-tree 0.1.7 pypi_0 pypi entrypoints 0.3 pypi_0 pypi et-xmlfile 1.1.0 pypi_0 pypi executing 0.8.3 pyhd3eb1b0_0 anaconda filelock 3.8.0 pypi_0 pypi flake8 3.9.2 pypi_0 pypi flatbuffers 1.12 pypi_0 pypi fonttools 4.37.1 pypi_0 pypi frozenlist 1.3.1 pypi_0 pypi gast 0.4.0 pypi_0 pypi google-auth 2.11.0 pypi_0 pypi google-auth-oauthlib 0.4.6 pypi_0 pypi google-pasta 0.2.0 pypi_0 pypi gputil 1.4.0 pypi_0 pypi grpcio 1.43.0 py310h00ca444_0 conda-forge gym 0.21.0 pypi_0 pypi h5py 3.6.0 nompi_py310hb8bbf05_100 conda-forge hdf5 1.12.1 nompi_h829dc4f_103 conda-forge idna 2.10 pypi_0 pypi imageio 2.21.2 pypi_0 pypi iniconfig 1.1.1 pypi_0 pypi ipykernel 5.5.5 pypi_0 pypi ipython 7.31.1 pypi_0 pypi ipython-genutils 0.2.0 pypi_0 pypi ipywidgets 7.6.3 pypi_0 pypi isort 5.8.0 pypi_0 pypi jedi 0.18.0 pypi_0 pypi jinja2 3.0.1 pypi_0 pypi jsonschema 3.2.0 pypi_0 pypi jupyter 1.0.0 pypi_0 pypi jupyter-client 6.1.12 pypi_0 pypi jupyter-console 6.4.0 pypi_0 pypi jupyter-core 4.7.1 pypi_0 pypi jupyter_client 7.2.2 py310hca03da5_0 anaconda jupyter_core 4.10.0 py310hca03da5_0 anaconda jupyterlab-pygments 0.1.2 pypi_0 pypi jupyterlab-widgets 1.0.0 pypi_0 pypi keras 2.9.0 pypi_0 pypi keras-preprocessing 1.1.2 pypi_0 pypi kiwisolver 1.3.1 pypi_0 pypi krb5 1.19.2 hfff258c_3 conda-forge lazy-object-proxy 1.6.0 pypi_0 pypi libblas 3.9.0 16_osxarm64_openblas conda-forge libcblas 3.9.0 16_osxarm64_openblas conda-forge libclang 14.0.6 pypi_0 pypi libcurl 7.81.0 h8d07098_0 conda-forge libcxx 12.0.0 hf6beb65_1 anaconda libedit 3.1.20191231 hc8eb9b7_2 conda-forge libev 4.33 h642e427_1 conda-forge libffi 3.4.2 hc377ac9_4 anaconda libgfortran 5.0.0 11_3_0_hd922786_25 conda-forge libgfortran5 11.3.0 hdaf2cc0_25 conda-forge liblapack 3.9.0 16_osxarm64_openblas conda-forge libnghttp2 1.46.0 h62a264d_0 conda-forge libopenblas 0.3.21 openmp_hc731615_3 conda-forge libsodium 1.0.18 h1a28f6b_0 anaconda libssh2 1.10.0 h7a5bd25_3 conda-forge libzlib 1.2.12 ha287fd2_2 conda-forge llvm-openmp 14.0.4 hd125106_0 conda-forge lz4 3.1.3 pypi_0 pypi markdown 3.4.1 pypi_0 pypi markupsafe 2.1.1 pypi_0 pypi matplotlib 3.5.3 pypi_0 pypi matplotlib-inline 0.1.6 pypi_0 pypi mccabe 0.6.1 pypi_0 pypi mistune 0.8.4 pypi_0 pypi modified-ai-economist 1.1.0 dev_0 msgpack 1.0.4 pypi_0 pypi mypy-extensions 0.4.3 pypi_0 pypi nbclient 0.5.3 pypi_0 pypi nbconvert 6.0.7 pypi_0 pypi nbformat 5.1.3 pypi_0 pypi ncurses 6.3 h1a28f6b_2 anaconda nest-asyncio 1.5.1 pypi_0 pypi networkx 2.8.6 pypi_0 pypi notebook 6.4.1 pypi_0 pypi numpy 1.21.0 pypi_0 pypi oauthlib 3.2.0 pypi_0 pypi openpyxl 3.0.7 pypi_0 pypi openssl 3.0.5 h03a7124_2 conda-forge opt-einsum 3.3.0 pypi_0 pypi packaging 20.9 pypi_0 pypi pandas 1.2.4 pypi_0 pypi pandocfilters 1.4.3 pypi_0 pypi parso 0.8.2 pypi_0 pypi pathspec 0.10.1 pypi_0 pypi pexpect 4.8.0 pyhd3eb1b0_3 anaconda pickleshare 0.7.5 pyhd3eb1b0_1003 anaconda pillow 9.0.1 pypi_0 pypi pip 21.2.4 py310hca03da5_0 anaconda platformdirs 2.5.2 pypi_0 pypi pluggy 0.13.1 pypi_0 pypi prometheus-client 0.10.1 pypi_0 pypi prompt-toolkit 3.0.18 pypi_0 pypi protobuf 3.19.4 pypi_0 pypi ptyprocess 0.7.0 pyhd3eb1b0_2 anaconda pure_eval 0.2.2 pyhd3eb1b0_0 anaconda py 1.10.0 pypi_0 pypi pyasn1 0.4.8 pypi_0 pypi pyasn1-modules 0.2.8 pypi_0 pypi pycodestyle 2.7.0 pypi_0 pypi pycparser 2.20 pypi_0 pypi pycryptodome 3.10.1 pypi_0 pypi pyflakes 2.3.1 pypi_0 pypi pygments 2.9.0 pypi_0 pypi pylint 2.8.2 pypi_0 pypi pyparsing 2.4.7 pypi_0 pypi pyrsistent 0.17.3 pypi_0 pypi pytest 6.2.4 pypi_0 pypi python 3.10.5 h4eee789_0_cpython conda-forge python-dateutil 2.8.1 pypi_0 pypi python_abi 3.10 2_cp310 conda-forge pytz 2021.1 pypi_0 pypi pywavelets 1.3.0 pypi_0 pypi pyyaml 5.4.1 pypi_0 pypi pyzmq 22.0.3 pypi_0 pypi qtconsole 5.1.0 pypi_0 pypi qtpy 1.9.0 pypi_0 pypi ray 2.0.0 pypi_0 pypi readline 8.1.2 h1a28f6b_1 anaconda regex 2021.4.4 pypi_0 pypi requests 2.25.1 pypi_0 pypi requests-oauthlib 1.3.1 pypi_0 pypi rsa 4.9 pypi_0 pypi scikit-image 0.19.3 pypi_0 pypi scipy 1.9.1 pypi_0 pypi seaborn 0.12.0 pypi_0 pypi send2trash 1.5.0 pypi_0 pypi setuptools 61.2.0 py310hca03da5_0 anaconda six 1.16.0 pyh6c4a22f_0 conda-forge soupsieve 2.2.1 pypi_0 pypi sqlite 3.38.5 h1058600_0 anaconda stack_data 0.2.0 pyhd3eb1b0_0 anaconda tabulate 0.8.10 pypi_0 pypi tensorboard 2.9.1 pypi_0 pypi tensorboard-data-server 0.6.1 pypi_0 pypi tensorboard-plugin-wit 1.8.1 pypi_0 pypi tensorboardx 2.5.1 pypi_0 pypi tensorflow-deps 2.8.0 0 apple tensorflow-estimator 2.9.0 pypi_0 pypi tensorflow-macos 2.9.2 pypi_0 pypi tensorflow-metal 0.5.1 pypi_0 pypi termcolor 1.1.0 pypi_0 pypi terminado 0.10.0 pypi_0 pypi testpath 0.5.0 pypi_0 pypi tifffile 2022.8.12 pypi_0 pypi tk 8.6.12 hb8d0fd4_0 anaconda toml 0.10.2 pypi_0 pypi tomli 2.0.1 pypi_0 pypi tornado 6.1 py310h1a28f6b_0 anaconda tqdm 4.60.0 pypi_0 pypi traitlets 5.0.5 pypi_0 pypi typing-extensions 3.10.0.0 pypi_0 pypi tzdata 2022a hda174b7_0 anaconda urllib3 1.26.5 pypi_0 pypi virtualenv 20.16.4 pypi_0 pypi wcwidth 0.2.5 pyhd3eb1b0_0 anaconda webencodings 0.5.1 pypi_0 pypi werkzeug 2.2.2 pypi_0 pypi wheel 0.37.1 pyhd3eb1b0_0 anaconda widgetsnbextension 3.5.1 pypi_0 pypi wrapt 1.12.1 pypi_0 pypi xz 5.2.5 h1a28f6b_1 anaconda zeromq 4.3.4 hc377ac9_0 anaconda zlib 1.2.12 h5a0b063_2 anaconda

As it is clear from the above list, there are some extra modules beside the required modules of the original AI-Economist and also some modules have different versions than what are mentioned in the original AI-Economist. For example, the ray version here is 2.0.0. I think with this version of ray, the parent class of any tensorflow recurrent custom model should be set RecurrentNetwork instead of RecurrentTFModelV2, so I made modifications in tf_models as required.

Then, I ran the following commands similar to what are brought in your provided notebook: multi_agent_training_with_rllib

from rllib.env_wrapper import RLlibEnvWrapper env_obj = RLlibEnvWrapper({"env_config_dict": env_config_dict}, verbose=True)

import ray from ray.rllib.agents.ppo import PPOTrainer

from ray.rllib.models.catalog import ModelCatalog from rllib.tf_models import KerasConvLSTM, RandomAction

ModelCatalog.register_custom_model(KerasConvLSTM.custom_name, KerasConvLSTM) ModelCatalog.register_custom_model(RandomAction.custom_name, RandomAction)

policies = { "a": ( None, # uses default policy env_obj.observation_space, env_obj.action_space, {'clip_param': 0.3, 'entropy_coeff': 0.025, 'entropy_coeff_schedule': None, 'gamma': 0.998, 'grad_clip': 10.0, 'kl_coeff': 0.0, 'kl_target': 0.01, 'lambda': 0.98, 'lr': 0.0003, 'lr_schedule': None, 'model': {'custom_model': 'keras_conv_lstm', 'custom_model_config': {'fc_dim': 128, 'idx_emb_dim': 4, 'input_emb_vocab': 100, 'lstm_cell_size': 128, 'num_conv': 2, 'num_fc': 2}, 'max_seq_len': 25}, 'use_gae': True, 'vf_clip_param': 50.0, 'vf_loss_coeff': 0.05, 'vf_share_layers': False} # define a custom agent policy configuration. ), "p": ( None, # uses default policy env_obj.observation_space_pl, env_obj.action_space_pl, {'clip_param': 0.3, 'entropy_coeff': 0.125, 'entropy_coeff_schedule': [[0, 2.0], [50000000, 0.125]], 'gamma': 0.998, 'grad_clip': 10.0, 'kl_coeff': 0.0, 'kl_target': 0.01, 'lambda': 0.98, 'lr': 0.0001, 'lr_schedule': None, 'model': {'custom_model': 'keras_conv_lstm', 'custom_model_config': {'fc_dim': 256, 'idx_emb_dim': 4, 'input_emb_vocab': 100, 'lstm_cell_size': 256, 'num_conv': 2, 'num_fc': 2}, 'max_seq_len': 25}, 'use_gae': True, 'vf_clip_param': 50.0, 'vf_loss_coeff': 0.05, 'vf_share_layers': False} # define a custom planner policy configuration. ) }

policy_mapping_fun = lambda i: "a" if str(i).isdigit() else "p"

policies_to_train = ["a", "p"]

trainer_config = { "multiagent": { "policies": policies, "policies_to_train": policies_to_train, "policy_mapping_fn": policy_mapping_fun, } }

trainer_config.update( { "num_workers": 5, "num_envs_per_worker": 1,

Other training parameters

    "train_batch_size": 4000,
    "sgd_minibatch_size": 4000,
    "num_sgd_iter": 1
}

)

env_config = { "env_config_dict": env_config_dict, "num_envs_per_worker": trainer_config.get('num_envs_per_worker'),
}

trainer_config.update( { "env_config": env_config
} )

ray.init()

trainer = PPOTrainer(env=RLlibEnvWrapper, config=trainer_config )

However, the following error is generated:

RayActorError: The actor died because of an error raised in its creation task, ray::RolloutWorker.init() (pid=1353, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x16a373790>) File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/evaluation/rollout_worker.py", line 613, in init self._build_policy_map( File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/evaluation/rollout_worker.py", line 1784, in _build_policy_map self.policy_map.create_policy( File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/policy/policy_map.py", line 123, in create_policy self[policy_id] = create_policy_for_framework( File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/utils/policy.py", line 71, in create_policy_for_framework return policy_class( File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/algorithms/ppo/ppo_tf_policy.py", line 83, in init base.init( File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 81, in init self.model = self.make_model() File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 221, in make_model return ModelCatalog.get_model_v2( File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/models/catalog.py", line 587, in get_model_v2 raise ValueError( ValueError: It looks like you are still using <rllib.tf_models.KerasConvLSTM object at 0x16a6f2ec0>.register_variables() to register your model's weights. This is no longer required, but if you are still calling this method at least once, you must make sure to register all created variables properly. The missing variables are {<Reference wrapping <tf.Variable 'a_wk4/conv2D_2_pol/kernel:0' shape=(3, 3, 16, 32) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/value/bias:0' shape=(1,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_1_pol/bias:0' shape=(16,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/layer_norm_pol/beta:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_1_pol/kernel:0' shape=(3, 3, 26, 16) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_pol/lstm_cell/bias:0' shape=(512,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/bias:0' shape=(512,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/value/kernel:0' shape=(128, 1) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense2_pol/bias:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/logits/bias:0' shape=(80,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_2_val/kernel:0' shape=(3, 3, 16, 32) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/layer_norm_val/beta:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_pol/lstm_cell/recurrent_kernel:0' shape=(128, 512) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/logits/kernel:0' shape=(128, 80) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense1_val/kernel:0' shape=(234, 128) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense1_val/bias:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/embedding_pol/embeddings:0' shape=(100, 4) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/layer_norm_pol/gamma:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_2_val/bias:0' shape=(32,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_2_pol/bias:0' shape=(32,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_1_val/bias:0' shape=(16,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/recurrent_kernel:0' shape=(128, 512) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense2_pol/kernel:0' shape=(128, 128) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_pol/lstm_cell/kernel:0' shape=(128, 512) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_1_val/kernel:0' shape=(3, 3, 26, 16) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense2_val/kernel:0' shape=(128, 128) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense2_val/bias:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense1_pol/kernel:0' shape=(234, 128) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense1_pol/bias:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/embedding_val/embeddings:0' shape=(100, 4) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/layer_norm_val/gamma:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/kernel:0' shape=(128, 512) dtype=float32>>}, and you only registered {<tf.Variable 'a_wk4/conv2D_2_pol/kernel:0' shape=(3, 3, 16, 32) dtype=float32>, <tf.Variable 'a_wk4/conv2D_1_pol/bias:0' shape=(16,) dtype=float32>, <tf.Variable 'a_wk4/value/bias:0' shape=(1,) dtype=float32>, <tf.Variable 'a_wk4/conv2D_1_pol/kernel:0' shape=(3, 3, 26, 16) dtype=float32>, <tf.Variable 'a_wk4/layer_norm_pol/beta:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/lstm_pol/lstm_cell/bias:0' shape=(512,) dtype=float32>, <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/bias:0' shape=(512,) dtype=float32>, <tf.Variable 'a_wk4/value/kernel:0' shape=(128, 1) dtype=float32>, <tf.Variable 'a_wk4/dense2_pol/bias:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/conv2D_2_val/kernel:0' shape=(3, 3, 16, 32) dtype=float32>, <tf.Variable 'a_wk4/logits/bias:0' shape=(80,) dtype=float32>, <tf.Variable 'a_wk4/layer_norm_val/beta:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/lstm_pol/lstm_cell/recurrent_kernel:0' shape=(128, 512) dtype=float32>, <tf.Variable 'a_wk4/logits/kernel:0' shape=(128, 80) dtype=float32>, <tf.Variable 'a_wk4/dense1_val/kernel:0' shape=(234, 128) dtype=float32>, <tf.Variable 'a_wk4/dense1_val/bias:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/embedding_pol/embeddings:0' shape=(100, 4) dtype=float32>, <tf.Variable 'a_wk4/layer_norm_pol/gamma:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/conv2D_2_val/bias:0' shape=(32,) dtype=float32>, <tf.Variable 'a_wk4/conv2D_2_pol/bias:0' shape=(32,) dtype=float32>, <tf.Variable 'a_wk4/conv2D_1_val/bias:0' shape=(16,) dtype=float32>, <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/recurrent_kernel:0' shape=(128, 512) dtype=float32>, <tf.Variable 'a_wk4/dense2_pol/kernel:0' shape=(128, 128) dtype=float32>, <tf.Variable 'a_wk4/lstm_pol/lstm_cell/kernel:0' shape=(128, 512) dtype=float32>, <tf.Variable 'a_wk4/conv2D_1_val/kernel:0' shape=(3, 3, 26, 16) dtype=float32>, <tf.Variable 'a_wk4/dense2_val/kernel:0' shape=(128, 128) dtype=float32>, <tf.Variable 'a_wk4/dense2_val/bias:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/dense1_pol/kernel:0' shape=(234, 128) dtype=float32>, <tf.Variable 'a_wk4/dense1_pol/bias:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/embedding_val/embeddings:0' shape=(100, 4) dtype=float32>, <tf.Variable 'a_wk4/layer_norm_val/gamma:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/kernel:0' shape=(128, 512) dtype=float32>}. Did you forget to call register_variables() on some of the variables in question?

This happen even if I use RandomAction network. Also, in your provided notebook, you have not used these custom modelsI. Moreover, I think this is most probably not related to the changes that I made in the AI-Economist but it might be related to the compatibility issues of different versions of the modules that I have in my environment. I greatly appreciate any comments.

By the way, please let me know if you need further information. If it is required I would be happy to share the GitHub repository of my own version of AI-Economist with you. Many thanks in advance!

[sa1g]

Hi there, as you noticed there are many compatibility issues in AI-Economist... To fix your problem you just have to comment register_variables(). Note that many other issues are ahead of this one. I think there are compatibility issues between rllib, tensorflow + non full retro-compatibility when you start upgrading stuff.

[aslansd]

Hi @sa1g, thanks, I already solved this problem. However, I have now other issue which I posted here:

https://discuss.ray.io/t/using-custom-neural-network-in-rllib/8094

I was wondering if you could be helpful in this case too! Many thanks in advance!