search

DeepX-inc / machina

Control section: Deep Reinforcement Learning framework
MIT License
279 stars 45 forks source link

[WIP] add test #148

Closed takerfume closed 5 years ago

takerfume commented 5 years ago

I finished writing test code of bc, GAIL and AIRL. @jinbeizame007 , Could you please write test code of MPC in this branch?

rarilurelo commented 5 years ago

@takerfume Please check the test details.

jinbeizame007 commented 5 years ago

It's ok. I write the test code of MPC in this branch.

takerfume commented 5 years ago

@takerfume Please check the test details.

@rarilurelo what do you mean? If you mean I should seek the reason why traivis-ci's build fail, I think this cannot be resolve. This is because it arise following error massage, Traceback (most recent call last): File "/home/travis/build/DeepX-inc/machina/tests/test_algos.py", line 578, in test_learning with open(os.path.join('../data/expert_epis', 'Pendulum-v0_100epis.pkl'), 'rb') as f: FileNotFoundError: [Errno 2] No such file or directory: '../data/expert_epis/Pendulum-v0_100epis.pkl'

However, pickle file should be downloaded from google drive instead of git repository.

jinbeizame007 commented 5 years ago

@takerfume I wrote test code of MPC and checked result of the test. An error has occurred in Travis CI, but this is an error due to test code of AIRL.

takerfume commented 5 years ago

@takerfume I wrote test code of MPC and checked result of the test. An error has occurred in Travis CI, but this is an error due to test code of AIRL.

Thank you very much! Error in Travis CI is inevitable unless we store pickle file of expert trajectory in this repository, but that is not desirable.

takerfume commented 5 years ago

Test has passed in my part.

takerfume commented 5 years ago

@jinbeizame007 in turn, there is an error in TestMPC, could you please check it?

jinbeizame007 commented 5 years ago

@takerfume In the MPC paper, we use a hacky reward function specific to each task (Swimmer, Half-Cheetah, Hopper, Ant). I used a reward function specific to Half-Cheetah task. I will implement a reward function specific to pendulum task.

takerfume commented 5 years ago

@takerfume In the MPC paper, we use a hacky reward function specific to each task (Swimmer, Half-Cheetah, Hopper, Ant). I used a reward function specific to Half-Cheetah task. I will implement a reward function specific to pendulum task.

I see. There is a history that installing pybullet was removed. Therefore, I removed pybullet from your code. @rarilurelo, which is desirable, to change code for pendulum or install pybullet.

rarilurelo commented 5 years ago

Sometimes pybullet with travis faild. So pendulum version is preferable. If installation of pybullet with .travis.yml is not faild, it's also ok.

jinbeizame007 commented 5 years ago

I implemented a reward function specific to the pendulum task. But, the models have not been successfully learned yet. Please wait a little more.

takerfume commented 5 years ago

I fixed conflict of example/make_expert_epis.py by running this command

git pull --rebase origin master
git push -f origin add_test
takerfume commented 5 years ago

Again I fixed conflict caused by merged pull request #135

takerfume commented 5 years ago

I fixed conflict caused by merging PR of #138

jinbeizame007 commented 5 years ago

@takerfume I fixed a reward function and confirmed that the models successfully learn.

takerfume commented 5 years ago

@jinbeizame007 In Travis, the following error still exists. Could you check this?


nose.plugins.cover: ERROR: Coverage not available: unable to import coverage module
/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/gym/envs/registration.py:14: PkgResourcesDeprecationWarning: Parameters to load are deprecated.  Call .resolve and .require separately.
  result = entry_point.load(False)
....EException ignored in: <bound method EpiSampler.__del__ of <machina.samplers.epi_sampler.EpiSampler object at 0x7f862a0d8ac8>>
Traceback (most recent call last):
  File "/home/travis/build/DeepX-inc/machina/machina/samplers/epi_sampler.py", line 175, in __del__
    for p in self.processes:
AttributeError: 'EpiSampler' object has no attribute 'processes'
======================================================================
ERROR: test_learning (test_algos.TestMPC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/DeepX-inc/machina/tests/test_algos.py", line 699, in test_learning
    self.env, mpc_pol, num_parallel=1)
  File "/home/travis/build/DeepX-inc/machina/machina/samplers/epi_sampler.py", line 149, in __init__
    self.pol = copy.deepcopy(pol)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 247, in _deepcopy_method
    return type(x)(x.__func__, deepcopy(x.__self__, memo))
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/copy.py", line 161, in deepcopy
    y = copier(memo)
TypeError: cannot deepcopy this pattern object```
jinbeizame007 commented 5 years ago

It's ok. I check this error.

jinbeizame007 commented 5 years ago

I could not reproduce this error. In TravisCI, what is the version of pytorch?

takerfume commented 5 years ago

@jinbeizame007 as you can see in #159 , in Travis CI, the folloing versions of packages are used. This is a result of pip freeze in Travis

attrs==17.3.0
cached-property==1.5.1
certifi==2018.11.29
chardet==3.0.4
future==0.17.1
gym==0.12.0
idna==2.8
joblib==0.13.2
-e git+git@github.com:DeepX-inc/machina.git@d24886fffbc62a6dc9efa7b25a8cdd6a7327630f#egg=machina
mock==2.0.0
nose==1.3.7
numpy==1.13.3
pandas==0.24.1
pbr==3.1.1
pluggy==0.6.0
py==1.5.2
pyglet==1.3.2
pytest==3.3.0
python-dateutil==2.8.0
pytz==2018.9
requests==2.21.0
scipy==1.2.1
six==1.11.0
terminaltables==3.1.0
torch==1.0.1.post2
urllib3==1.24.1
takerfume commented 5 years ago

I rebased to fix version of gym==0.10.5 same as #158

takerfume commented 5 years ago

@jinbeizame007 I ran the command nosetests -x tests in my local, and I got the same error as above.

jinbeizame007 commented 5 years ago

@takerfume Thank you very much for teaching me kindly! I try to reproduce this error.

jinbeizame007 commented 5 years ago

@takerfume I fixed the bug about deepcopy of MPCPol.

takerfume commented 5 years ago

@jinbeizame007 Thank you for your working hard! @rarilurelo I cannot merge this pull request without your approving review? Could you check this pr?

rarilurelo commented 5 years ago

We can't merge PR before travis-ci 's build passing. And if master is updated from the time this branch is branched, we have to merge commit of master (just push update branch botton).