search

keiohta / tf2rl

TensorFlow2 Reinforcement Learning
MIT License
465 stars 103 forks source link

pixel obs #93

Closed haoyu-x closed 4 years ago

haoyu-x commented 4 years ago

Hi @keiohta , thanks for sharing such a great project!

I'm wondering if I want to set observation to be image-based using tf2rl, do you have any advice?

keiohta commented 4 years ago

I'm not sure what you want to do but I guess Atari example with DQN might help you. https://github.com/keiohta/tf2rl/blob/master/examples/run_dqn_atari.py

haoyu-x commented 4 years ago

hi @keiohta, sorry for the late reply.

basically, I'd like to use GAIfO+SAC on image-based dm_control and robosuite tasks. What I'm trying to say is that does tf2rl have any support of image-based RL/IL. or let's say can I pull a request, for example, SAC+AE working on the image-based dm_conttrol suite. https://sites.google.com/view/sac-ae/home. also something like CURL. https://arxiv.org/abs/2004.04136 example of IL, the experiment of the original GAIfO paper is also based on image-based tasks, it uses CNN policy and CNN discriminator. also, VAIL explored video imitation in their experiments.

if there could be a CNN-GAIfO or VAE-GAIfO-SAC. that would be great! thanks!

Haoyu

keiohta commented 4 years ago

Hi @haoyu-x , sorry I completely forgot to answer your question...

Current tf2rl does not officially support image-based RL/IL (I mean I don't provide no example of such scripts under examples), but we've already implemented the codes to do that. For example, I think you can solve an easy image-based RL (let's say Pendulum-v0) using SAC + VAE, or do IRL in similar way.

I strongly suggest you use VAE instead of directly inputting image to your policy. Refs are:

Of course it would be awesome if you can send the PR that supports CURL or other image-based RL/IL!

haoyu-x commented 4 years ago

Thank you！

Kei Ohta notifications@github.com于2020年8月12日 周三21:30写道：

Hi @haoyu-x https://github.com/haoyu-x , sorry I completely forgot to answer your question...

Current tf2rl does not officially support image-based RL/IL (I mean I don't provide no example of such scripts under examples https://github.com/keiohta/tf2rl/tree/master/examples), but we've already implemented the codes to do that. For example, I think you can solve an easy image-based RL (let's say Pendulum-v0) using SAC https://github.com/keiohta/tf2rl/blob/master/tf2rl/algos/sac.py + VAE https://github.com/keiohta/tf2rl/blob/master/tf2rl/tools/vae.py, or do IRL in similar way.

I strongly suggest you use VAE instead of directly inputting image to your policy. Refs are:

Of course it would be awesome if you can send the PR that supports CURL or other image-based RL/IL!

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/keiohta/tf2rl/issues/93#issuecomment-672871430, or unsubscribe https://github.com/notifications/unsubscribe-auth/APACPZSDJ4VEWI5IABFFTYTSAKKOHANCNFSM4PQGTADQ .

keiohta commented 4 years ago

I close this issue since it seems to be resolved. Please reopen this issue if you find other problem related to this.