search

alex-petrenko / sample-factory

High throughput synchronous and asynchronous reinforcement learning
https://samplefactory.dev
MIT License
811 stars 109 forks source link

Fix actor critic separate weights PackedSequence #290

Closed BartekCupial closed 8 months ago

BartekCupial commented 8 months ago

Quite self-explanatory. For some reason ActorCriticSeparateWeights wasn't updated to support PackedSequence. As I've explained in the comment.

We cannot chunk PackedSequence directly, we first have to to unpack it, chunk, 
then pack chunks again to be able to process then through the cores. 
Finally we have to return concatenated outputs so we repeat the proces, 
but this time using concatenation - unpack, cat and pack.