luksfarris
commented
3 years ago After a lot of time spent on tf_agents, I decided to abandon it. Their implementation is very problematic, in terms of software engineering. Every new project I adopt from them is a new wrapper. A case in point is the concept of a reinforcement learning environment. It requires me to create a tf_agents wrapper, of a ml_fairness_gym wrapper, of a recsim wrapper, of a gym wrapper. Will resort to implementing REINFORCE myself
Time is really limited right now, so we figured we'd test the Dueling implementation against REINFORCE and a soft actor critic agent. This way we get a q-value based model, a policy gradient, and an actor-critic. I had them implemented somewhere but I figured I'd try and use tf-agents for now. It may give me the possibility to easily test other agents, as well as include some monitoring tools.
The biggest challenges I think are dealing with slates, and the tensorflow dependency hell