Question: How to implement simple agents?

[philwinder]

I want to use some "simple" agents to act as a baseline. For example, random, SARSA and tabular Q-learning.

There aren't any in rl_coach by default, so I attempted to create my own. But the Agent class is so laced with neural networks I am finding it hard to figure out what is minimally required. The documented examples show how to take a DQN-like algorithm and improve upon it, but not how to create a simpler agent.

Do you know of a really simple agent example? One that doesn't need any DL? For example how would you implement a random agent?

Thanks again!

[shadiendrawis]

Hello Phil,

It's true that the Agent class is quite interlaced with neural networks and we currently don't have any examples of how to implement very simple agents.

However, I don't think you need to concern yourself with all the machinery required to make other algorithms work in order to implement a very simple agent. For example, if you want to implement a random agent, you can inherit from the Agent class and implement a learn_from_batch that does nothing (it's implemented like that by default), and implement choose_action which just picks a random action according to the action space size.

Note: You can override Agent.train() to do nothing if you want none of the training code to run in the background which would irrelevant to something like a random agent.

Let me know if that works for you!

Shadi