fabioseel
commented
1 week ago - [ ] Check whether implementing an own Learner would be sufficient
- potentially could allow to use completely custom model
- yes and no. Actor critic is used in several other places, but its not completely outrageous...
- more importantly: the RNN handling might be sth we want, so it might be worth keeping it
- potentially could allow better initialization than write etc
- seems hard without changing major parts of sample-factory
- [ ] if so, find way to inject it or if we have to use our own cloned repo
- with our own repo, we'd at least only have to maintain the Learner and can just rebase on the 'true' repo every now and then
This is probably the biggest TODO. I think there won't be a way around modifying the library to our needs, I am unsure whether weird side-effects could arise. Might be worth implementing some safety checks / tests first and approach this test-driven.
Definitely will affect the Learner class / backward step.