RatInABox-Lab / RatInABox

A python package for modelling locomotion in complex environments and spatially/velocity selective cell activity.
MIT License
175 stars 31 forks source link

Task env errors #52

Closed TomGeorge1234 closed 1 year ago

TomGeorge1234 commented 1 year ago

@SynapticSage I'm getting some errors and several hundred identical warning on the recent tests. sorry if its something I did, the last lines of the stack trace are:

tests/test_taskenv.py::test_parallel_api[10-Env0-Ag0-2-possible_goal_positions1-noninteract]
[315](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:316)
  /opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/_pytest/python.py:198: PytestReturnNotNoneWarning: Expected None, but tests/test_taskenv.py::test_parallel_api[10-Env0-Ag0-2-possible_goal_positions1-noninteract] returned True, which will be an error in a future version of pytest.  Did you mean to use `assert` instead of `return`?
[316](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:317)
    warnings.warn(
[317](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:318)

[318](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:319)
tests/test_taskenv.py::test_parallel_api[10-Env0-Ag0-2-random_2-interact]
[319](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:320)
  /opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/_pytest/python.py:198: PytestReturnNotNoneWarning: Expected None, but tests/test_taskenv.py::test_parallel_api[10-Env0-Ag0-2-random_2-interact] returned True, which will be an error in a future version of pytest.  Did you mean to use `assert` instead of `return`?
[320](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:321)
    warnings.warn(
[321](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:322)

[322](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:323)
tests/test_taskenv.py::test_parallel_api[10-Env0-Ag0-2-random_2-noninteract]
[323](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:324)
  /opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/_pytest/python.py:198: PytestReturnNotNoneWarning: Expected None, but tests/test_taskenv.py::test_parallel_api[10-Env0-Ag0-2-random_2-noninteract] returned True, which will be an error in a future version of pytest.  Did you mean to use `assert` instead of `return`?
[324](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:325)
    warnings.warn(
[325](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:326)

[326](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:327)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
[327](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:328)
=========================== short test summary info ============================
[328](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:329)
FAILED test_taskenv.py::test_parallel_api[1-Env0-Ag0-2-random_2-interact] - AssertionError: ['agent_0'] != set()
[329](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:330)
assert {'agent_0'} == set()
[330](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:331)
  Extra items in the left set:
[331](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:332)
  'agent_0'
[332](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:333)
  Full diff:
[333](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:334)
  - set()
[334](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:335)
  + {'agent_0'}
[335](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:336)
============ 1 failed, 188 passed, 483 warnings in 87.53s (0:01:27) ============
[336](https://github.com/TomGeorge1234/RatInABox/actions/runs/5193543340/jobs/9364221102#step:6:337)
Error: Process completed with exit code 1.
SynapticSage commented 1 year ago

I'll take a look --- it seems to be pettingzoo's parallel_api_test.

TomGeorge1234 commented 1 year ago

Weird. Still breaking on GitHub but not for me locally. Yeah seems to be within the pettingzoo test. Wonder if its a bug and we can ignore the test or something we should worry about...

TomGeorge1234 commented 1 year ago

Hm, update is that it seems to be stochastically breaking, for now I've commented out the test but we should dig into it.

TomGeorge1234 commented 1 year ago

I'll leave it open for that reason

SynapticSage commented 1 year ago

Ya, noticing the stochasticity as well. I may not be handling an edge or corner case somewhere. I'll look at it when I catch some downtime.

The test in question uses an external call to is_terminated and is_truncated to manually track when an agent should/should not be "live" (i.e. active/un-terminated). I haven't seen anything in the agent videos ... but maybe the first or last frame of an episode.