search

IntelLabs / coach

Reinforcement Learning Coach by Intel AI Lab enables easy experimentation with state of the art Reinforcement Learning algorithms
https://intellabs.github.io/coach/
Apache License 2.0
2.32k stars 460 forks source link

Batch RL Example #346

Closed bbalaji-ucsd closed 5 years ago

bbalaji-ucsd commented 5 years ago

Thanks for adding the support for Batch RL!

I was looking for example code / documentation to try it out, but could not find them. Can you please point me to an example I can use?

gal-leibovich commented 5 years ago

Hi @bbalaji-ucsd,

We do not have a notebook, yet, showcasing the feature, nor do we have an example showing the usage of loading a dataset from a CSV file, just yet. What we do have, at the moment, are two presets showcasing Batch RL on CartPole (data collection, either by a random policy or by first training an agent, and then training). There are several supported algorithms, among them is a variant of BCQ (Batch Constrained Q Learning), which seems to work much better than just plain DQN.

The presets are - 1.CartPole_DQN_BatchRL.

  1. CartPole_DQN_BatchRL_BCQ.
bbalaji-ucsd commented 5 years ago

Thanks @gal-leibovich this is great! Agree on BCQ being better than vanilla DQN.

Look forward to the csv example. It looks like you have the APIs already in place, just need to have the csv in the right format. Would like to know what this csv file looks like.

gal-leibovich commented 5 years ago

Hi @bbalaji-ucsd,

I have added a tutorial which explains in detail the usage of Batch RL with Coach, incl. the CSV usage scheme.

It's now merged to master.

bbalaji-ucsd commented 5 years ago

Thank you @gal-leibovich !