numenta / nupic.embodied

GNU Affero General Public License v3.0
6 stars 6 forks source link

Support for policy hooks for data visualization (single and composite), env updates in trainer, saving/restoring eval env state #52

Closed lucasosouza closed 2 years ago

lucasosouza commented 2 years ago

Far from clean. Tests works in simple hooks. The threads are killed once the collection is done, so the only way to retrieve it is returning in the rollout function call inside the sampler. Attributes of the hook manager will not maintain state once the thread is killed. Had to resort to composition to handle multiple hooks, should work fine, but there is probably a cleaner way.

This includes:

lucasosouza commented 2 years ago

@abhi-iyer your plots were integrated. I had to do a few changes to fix bug and adapt to the requirements in this case, but didn't change the logic behinds the plots (at least not intentionally). But it doesn't seem to be showing anything super relevant - we can discuss this and do the changes in another PR, should be more straightforward now.

Will run a full test with the main config. Please review it meanwhile so I can merge as soon as results are out.

lucasosouza commented 2 years ago

test run: https://wandb.ai/nupic-research/multitask/runs/cdyg338