search

facebookresearch / ReAgent

A platform for Reasoning systems (Reinforcement Learning, Contextual Bandits, etc.)
https://reagent.ai
BSD 3-Clause "New" or "Revised" License
3.56k stars 516 forks source link

sac test failing #91

Closed MrCartoonology closed 5 years ago

MrCartoonology commented 5 years ago

I just installed Horizon and ran the tests, however there is an error:

======================================================================
ERROR: test_sac_trainer (ml.rl.test.gridworld.test_gridworld_sac.TestGridworldSAC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/david/Code/Horizon/ml/rl/test/gridworld/test_gridworld_sac.py", line 191, in test_sac_trainer
    self._test_sac_trainer()
  File "/Users/david/Code/Horizon/ml/rl/test/gridworld/test_gridworld_sac.py", line 167, in _test_sac_trainer
    actor_predictor = self.get_actor_predictor(trainer, environment)
  File "/Users/david/Code/Horizon/ml/rl/test/gridworld/test_gridworld_sac.py", line 145, in get_actor_predictor
    trainer.actor_network, feature_extractor, output_transformer
  File "/Users/david/Code/Horizon/ml/rl/training/rl_exporter.py", line 59, in export
    output_transformer=self.output_transformer,
  File "/Users/david/Code/Horizon/ml/rl/models/base.py", line 152, in get_predictor_export_meta_and_workspace
    c2_model, input_blobs, output_blobs = self.get_caffe2_model()
  File "/Users/david/Code/Horizon/ml/rl/models/base.py", line 129, in get_caffe2_model
    caffe2.python.onnx.backend.prepare(model_protobuf),
  File "/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/caffe2/python/onnx/backend.py", line 689, in prepare
    super(Caffe2Backend, cls).prepare(model, device, **kwargs)
  File "/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/onnx/backend/base.py", line 74, in prepare
    onnx.checker.check_model(model)
  File "/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/onnx/checker.py", line 86, in check_model
    C.check_model(model.SerializeToString())
onnx.onnx_cpp2py_export.checker.ValidationError: No Op or Function registered for ConstantFill with domain_version of 9

==> Context: Bad node spec: input: "25" output: "26" op_type: "ConstantFill" attribute { name: "dtype" i: 1 type: INT } attribute { name: "input_as_shape" i: 1 type: INT } attribute { name: "value" f: 0 type: FLOAT } doc_string: "/Users/david/Code/Horizon/ml/rl/models/actor.py(137): forward\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py(477): _slow_forward\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py(487): __call__\n/Users/david/Code/Horizon/ml/rl/models/base.py(337): forward\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py(477): _slow_forward\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py(487): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/jit/__init__.py(253): forward\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py(489): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/jit/__init__.py(198): get_trace_graph\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/onnx/utils.py(192): _trace_and_get_graph_from_model\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/onnx/utils.py(224): _model_to_graph\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/onnx/utils.py(281): _export\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/onnx/utils.py(104): export\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/onnx/__init__.py(27): export\n/Users/david/Code/Horizon/ml/rl/models/base.py(120): export_to_buffer\n/Users/david/Code/Horizon/ml/rl/models/base.py(125): get_caffe2_model\n/Users/david/Code/Horizon/ml/rl/models/base.py(152): get_predictor_export_meta_and_workspace\n/Users/david/Code/Horizon/ml/rl/training/rl_exporter.py(59): export\n/Users/david/Code/Horizon/ml/rl/test/gridworld/test_gridworld_sac.py(145): get_actor_predictor\n/Users/david/Code/Horizon/ml/rl/test/gridworld/test_gridworld_sac.py(167): _test_sac_trainer\n/Users/david/Code/Horizon/ml/rl/test/gridworld/test_gridworld_sac.py(191): test_sac_trainer\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/case.py(605): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/case.py(653): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(122): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(84): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(122): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(84): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(122): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(84): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(122): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(84): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(122): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(84): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(122): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(84): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(122): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(84): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/runner.py(176): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/main.py(256): runTests\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/main.py(95): __init__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/setuptools/command/test.py(250): run_tests\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/setuptools/command/test.py(228): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/distutils/dist.py(974): run_command\n/Users/david/anaconda/envs/pytorch/lib/python3.6/distutils/dist.py(955): run_commands\n/Users/david/anaconda/envs/pytorch/lib/python3.6/distutils/core.py(148): setup\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/setuptools/__init__.py(143): setup\nsetup.py(27): <module>\n"

======================================================================
ERROR: test_sac_trainer_model_propensity (ml.rl.test.gridworld.test_gridworld_sac.TestGridworldSAC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/david/Code/Horizon/ml/rl/test/gridworld/test_gridworld_sac.py", line 205, in test_sac_trainer_model_propensity
    self._test_sac_trainer(logged_action_uniform_prior=True)
  File "/Users/david/Code/Horizon/ml/rl/test/gridworld/test_gridworld_sac.py", line 167, in _test_sac_trainer
    actor_predictor = self.get_actor_predictor(trainer, environment)
  File "/Users/david/Code/Horizon/ml/rl/test/gridworld/test_gridworld_sac.py", line 145, in get_actor_predictor
    trainer.actor_network, feature_extractor, output_transformer
  File "/Users/david/Code/Horizon/ml/rl/training/rl_exporter.py", line 59, in export
    output_transformer=self.output_transformer,
  File "/Users/david/Code/Horizon/ml/rl/models/base.py", line 152, in get_predictor_export_meta_and_workspace
    c2_model, input_blobs, output_blobs = self.get_caffe2_model()
  File "/Users/david/Code/Horizon/ml/rl/models/base.py", line 129, in get_caffe2_model
    caffe2.python.onnx.backend.prepare(model_protobuf),
  File "/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/caffe2/python/onnx/backend.py", line 689, in prepare
    super(Caffe2Backend, cls).prepare(model, device, **kwargs)
  File "/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/onnx/backend/base.py", line 74, in prepare
    onnx.checker.check_model(model)
  File "/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/onnx/checker.py", line 86, in check_model
    C.check_model(model.SerializeToString())
onnx.onnx_cpp2py_export.checker.ValidationError: No Op or Function registered for ConstantFill with domain_version of 9

==> Context: Bad node spec: input: "25" output: "26" op_type: "ConstantFill" attribute { name: "dtype" i: 1 type: INT } attribute { name: "input_as_shape" i: 1 type: INT } attribute { name: "value" f: 0 type: FLOAT } doc_string: "/Users/david/Code/Horizon/ml/rl/models/actor.py(137): forward\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py(477): _slow_forward\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py(487): __call__\n/Users/david/Code/Horizon/ml/rl/models/base.py(337): forward\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py(477): _slow_forward\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py(487): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/jit/__init__.py(253): forward\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py(489): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/jit/__init__.py(198): get_trace_graph\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/onnx/utils.py(192): _trace_and_get_graph_from_model\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/onnx/utils.py(224): _model_to_graph\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/onnx/utils.py(281): _export\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/onnx/utils.py(104): export\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/torch/onnx/__init__.py(27): export\n/Users/david/Code/Horizon/ml/rl/models/base.py(120): export_to_buffer\n/Users/david/Code/Horizon/ml/rl/models/base.py(125): get_caffe2_model\n/Users/david/Code/Horizon/ml/rl/models/base.py(152): get_predictor_export_meta_and_workspace\n/Users/david/Code/Horizon/ml/rl/training/rl_exporter.py(59): export\n/Users/david/Code/Horizon/ml/rl/test/gridworld/test_gridworld_sac.py(145): get_actor_predictor\n/Users/david/Code/Horizon/ml/rl/test/gridworld/test_gridworld_sac.py(167): _test_sac_trainer\n/Users/david/Code/Horizon/ml/rl/test/gridworld/test_gridworld_sac.py(205): test_sac_trainer_model_propensity\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/case.py(605): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/case.py(653): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(122): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(84): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(122): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(84): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(122): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(84): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(122): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(84): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(122): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(84): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(122): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(84): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(122): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/suite.py(84): __call__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/runner.py(176): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/main.py(256): runTests\n/Users/david/anaconda/envs/pytorch/lib/python3.6/unittest/main.py(95): __init__\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/setuptools/command/test.py(250): run_tests\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/setuptools/command/test.py(228): run\n/Users/david/anaconda/envs/pytorch/lib/python3.6/distutils/dist.py(974): run_command\n/Users/david/anaconda/envs/pytorch/lib/python3.6/distutils/dist.py(955): run_commands\n/Users/david/anaconda/envs/pytorch/lib/python3.6/distutils/core.py(148): setup\n/Users/david/anaconda/envs/pytorch/lib/python3.6/site-packages/setuptools/__init__.py(143): setup\nsetup.py(27): <module>\n"

For the installation, I have a separate conda environment I call pytorch. I had already installed pytorch before installing Horizon. The requirements seem to install a new pytorch from the nightly build. When I look at my conda environment below, I see both pytorch'es in there.

Also I installed spark via pip install - it got 2.4.0 not the 2.3.1 mentioned in the installation.md

Here is my conda environment, I am on Mac OS/X

(pytorch) ~/code/Horizon$ conda list
# packages in environment at /Users/david/anaconda/envs/pytorch:
#
_tflow_select             2.2.0                     eigen  
absl-py                   0.6.1                    py36_0  
apipkg                    1.5                        py_0    conda-forge
appnope                   0.1.0            py36hf537a9a_0  
astor                     0.7.1                    py36_0  
atari-py                  0.1.7                     <pip>
atomicwrites              1.2.1                      py_0    conda-forge
attrs                     18.2.0                     py_0    conda-forge
backcall                  0.1.0                    py36_0  
blas                      1.1                    openblas    conda-forge
bleach                    3.0.2                    py36_0  
box2d-py                  2.3.8                     <pip>
c-ares                    1.15.0               h1de35cc_1  
ca-certificates           2018.03.07                    0  
certifi                   2018.11.29               py36_0  
cffi                      1.11.5           py36h6174b99_1  
chardet                   3.0.4                     <pip>
cloudpickle               0.6.1                    py36_0  
cycler                    0.10.0           py36hfc81398_0  
cytoolz                   0.9.0.1          py36h1de35cc_1  
dask-core                 1.0.0                    py36_0  
dbus                      1.13.2               h760590f_1  
decorator                 4.3.0                    py36_0  
entrypoints               0.2.3                    py36_2  
execnet                   1.5.0                      py_0    conda-forge
expat                     2.2.6                h0a44026_0  
freetype                  2.9.1                hb4e5f40_0  
future                    0.17.1                    <pip>
gast                      0.2.0                    py36_0  
gettext                   0.19.8.1             h15daf44_3  
glib                      2.56.2               hd9629dc_0  
grpcio                    1.16.0          py36h9011c5e_1000    conda-forge
gym                       0.10.9                    <pip>
h5py                      2.9.0            py36h3134771_0  
hdf5                      1.10.4               hfa1e0ec_0  
icu                       58.2                 h4b95b61_1  
idna                      2.8                       <pip>
imageio                   2.4.1                    py36_0  
intel-openmp              2019.1                      144  
ipykernel                 5.1.0            py36h39e3cac_0  
ipython                   7.2.0            py36h39e3cac_0  
ipython_genutils          0.2.0            py36h241746c_0  
ipywidgets                7.4.2                    py36_0  
jedi                      0.13.2                   py36_0  
jinja2                    2.10                     py36_0  
jpeg                      9b                   he5867d9_2  
jsonschema                2.6.0            py36hb385e00_0  
jupyter                   1.0.0                    py36_7  
jupyter_client            5.2.4                    py36_0  
jupyter_console           6.0.0                    py36_0  
jupyter_core              4.4.0                    py36_0  
keras-applications        1.0.6                    py36_0  
keras-preprocessing       1.0.5                    py36_0  
kiwisolver                1.0.1            py36h0a44026_0  
libcxx                    4.0.1                hcfea43d_1  
libcxxabi                 4.0.1                hcfea43d_1  
libedit                   3.1.20170329         hb402a30_2  
libffi                    3.2.1                h475c297_4  
libgfortran               3.0.1                h93005f0_2  
libiconv                  1.15                 hdd342a3_7  
libopenblas               0.3.3                hdc02c5d_3  
libpng                    1.6.35               ha441bb4_0  
libprotobuf               3.6.1                hd9629dc_0  
libsodium                 1.0.16               h3efe00b_0  
libtiff                   4.0.9                hcb84e12_2  
markdown                  3.0.1                    py36_0  
markupsafe                1.1.0            py36h1de35cc_0  
matplotlib                3.0.2            py36h54f8f79_0  
maven                     3.5.0                         0    conda-forge
mistune                   0.8.4            py36h1de35cc_0  
mkl                       2019.1                      144  
mkl_fft                   1.0.10                   py36_0    conda-forge
mkl_random                1.0.2                    py36_0    conda-forge
more-itertools            4.3.0                 py36_1000    conda-forge
nbconvert                 5.3.1                    py36_0  
nbformat                  4.4.0            py36h827af21_0  
ncurses                   6.1                  h0a44026_1  
networkx                  2.2                      py36_1  
ninja                     1.8.2            py36h04f5b5a_1  
notebook                  5.7.4                    py36_0  
numpy                     1.15.1          py36_blas_openblashd3ea46f_1  [blas_openblas]  conda-forge
numpy-base                1.15.4           py36ha711998_0  
olefile                   0.46                     py36_0  
onnx                      1.4.1                     <pip>
openblas                  0.2.20                        8    conda-forge
openjdk                   8.0.144          zulu8.23.0.3_2    conda-forge
openssl                   1.0.2p            h1de35cc_1002    conda-forge
pandas                    0.23.4          py36h1702cab_1000    conda-forge
pandoc                    2.2.3.2                       0  
pandocfilters             1.4.2                    py36_1  
parso                     0.3.1                    py36_0  
pcre                      8.42                 h378b8a2_0  
pexpect                   4.6.0                    py36_0  
pickleshare               0.7.5                    py36_0  
pillow                    5.3.0            py36hb68e598_0  
pip                       18.1                     py36_0  
pluggy                    0.8.1                      py_0    conda-forge
prometheus_client         0.5.0                    py36_0  
prompt_toolkit            2.0.7                    py36_0  
protobuf                  3.6.1           py36h0a44026_1001    conda-forge
ptyprocess                0.6.0                    py36_0  
py                        1.7.0                      py_0    conda-forge
py4j                      0.10.7                    <pip>
pycparser                 2.19                     py36_0  
pyglet                    1.3.2                     <pip>
pygments                  2.3.1                    py36_0  
PyOpenGL                  3.1.0                     <pip>
pyparsing                 2.3.0                    py36_0  
pyqt                      5.9.2            py36h655552a_2  
pyspark                   2.4.0                     <pip>
pytest                    4.1.1                 py36_1000    conda-forge
pytest-forked             1.0.1                      py_0    conda-forge
pytest-xdist              1.24.0                     py_0    conda-forge
python                    3.6.6             h4a56312_1003    conda-forge
python-dateutil           2.7.5                    py36_0  
pytorch                   1.0.0                   py3.6_1    pytorch
pytorch-nightly           1.0.0.dev20190113         py3.6_0    pytorch
pytz                      2018.7                   py36_0  
pywavelets                1.0.1            py36h1d22016_0  
pyzmq                     17.1.2           py36h1de35cc_0  
qt                        5.9.7                h468cd18_1  
qtconsole                 4.4.3                    py36_0  
readline                  7.0                  h1de35cc_5  
requests                  2.21.0                    <pip>
scikit-image              0.14.1           py36h0a44026_0  
scipy                     1.1.0           py36_blas_openblash7943236_201  [blas_openblas]  conda-forge
send2trash                1.5.0                    py36_0  
setuptools                40.6.3                   py36_0  
sip                       4.19.8           py36h0a44026_0  
six                       1.12.0                   py36_0  
sqlite                    3.26.0               ha441bb4_0  
tensorboard               1.9.0                    py36_0    conda-forge
tensorboardx              1.4                        py_0    conda-forge
tensorflow                1.9.0                    py36_0    conda-forge
tensorflow-base           1.12.0          eigen_py36h4f0eeca_0  
termcolor                 1.1.0                    py36_1  
terminado                 0.8.1                    py36_1  
testpath                  0.4.2                    py36_0  
thrift                    0.10.0                   py36_0    conda-forge
thrift-cpp                0.10.0                        1    conda-forge
tk                        8.6.8                ha441bb4_0  
toolz                     0.9.0                    py36_0  
torchvision               0.2.1                      py_2    pytorch
tornado                   5.1.1            py36h1de35cc_0  
traitlets                 4.3.2            py36h65bd3ce_0  
typing                    3.6.6                     <pip>
typing-extensions         3.7.2                     <pip>
urllib3                   1.24.1                    <pip>
wcwidth                   0.1.7            py36h8c6ec74_0  
webencodings              0.5.1                    py36_1  
werkzeug                  0.14.1                   py36_0  
wheel                     0.32.3                   py36_0  
widgetsnbextension        3.4.2                    py36_0  
xz                        5.2.4                h1de35cc_4  
zeromq                    4.2.5                h0a44026_1  
zlib                      1.2.11               h1de35cc_3  
(pytorch) ~/code/Horizon$
kittipatv commented 5 years ago

https://github.com/facebookresearch/Horizon/pull/99 should fix this issue. Can you please confirm and close?

kittipatv commented 5 years ago

Fix is landed. Please reopen if you still experience the issue.