Open ajlangley opened 1 month ago
@ajlangley Thanks for raising this issue. The PolicyClient
and PolicyServer
is definitely on our list to have, but before the Summit 2024 there is no chance to get to this nor guide you in it as the list is already quite full for us. We will come back to this after the Summit.
Description
My understanding is that
PolicyServerInput
andPolicyClient
don't work with the new API stack, which appears to be due to their use of the olderRolloutWorker
instead ofEnvRunner
s.The ability to hook in non-gym environments using this interface has been very valuable to me so far, and I'm interested in making an attempt to update these modules so that we can use the latest algorithms with them.
Currently, I have identified that mainly
PolicyServer
itself will need some updating, in addition toIOContext
(it currently only takes aRolloutWorker
in its__init__
method).My main question about this is, could I get some general guidance on how to update the modules in question without making breaking changes to other parts of the project? My main concern is that
IOContext
could touch other code, although I'm relatively sure thatPolicyServerInput
andPolicyClient
can be updated without causing issues elsewhere.Thank you in advance for considering my request!
Use case
The PolicyServerInput feature is really useful for cases in which wrapping the environment in a gym interface is difficult. It would be nice to make sure it gets updated to the new API stack so everyone can continue using it.