search

IntelLabs / coach

Reinforcement Learning Coach by Intel AI Lab enables easy experimentation with state of the art Reinforcement Learning algorithms
https://intellabs.github.io/coach/
Apache License 2.0
2.32k stars 460 forks source link

Using billiard instead of multiprocessing package on rl_coach #429

Closed gpanouts closed 4 years ago

gpanouts commented 4 years ago

Hello, I'm using rl_coach in combination with Celery (a distributed task queue) in order to be able to spawn long-running RL tasks from within the flask framework. However, there is a compatibility issue. Celery does not support python package multiprocessing, but instead uses billiard (which is forked from Python 2.7 multiprocessing package).

I've changed the parts of rl_coach code which use multiprocessing with billiard, and with a few experiments it seems that things work as expected. But is there a way to test this change more extensively?

Thanks in advance.

I'm using:

rl-coach==1.0.1 billiard==3.6.1.0 gym==0.12.5

galnov commented 4 years ago

The full suite of Coach tests can be found here - they should all be passing. If you would like to focus on multi-workers testing, it would be good to make sure convergence after your changes is similar to the graphs under the benchmarks folder, as in the A3C example here.

gpanouts commented 4 years ago

@galnov Thanks for the feedback!