Open Phirefly9 opened 1 year ago
I can reproduce but we don't support creating algorithms within algorithms. That's a very funky pattern. What's the use case here?
We implemented league play using this pattern in an older version of ray/rllib. But we will adjust if that is not supported.
Thanks
What happened + What you expected to happen
My team has a tune trainable that creates an rllib algorithm dynamically during
step()
, however tune is locking up during creation of this algorithm and has required us to not use tune and instead just manually create the tune loop ourselves.I've attached a script that recreates the issue. you will see the output
(TestCartPole pid=243845) Building ALGORITHM
but you will never seeTraining ALGORITHM
even though tune thinks it is still runningVersions / Dependencies
ray 2.7 python 3.10 pytorch 2.0.0
Reproduction script
Issue Severity
Medium: It is a significant difficulty but I can work around it.