search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.29k stars 305 forks source link

[Feature Request] Image Augmentation Is All You Need benchmark #32

Open vmoens opened 2 years ago

vmoens commented 2 years ago

Nice benchmark to have as it would interact with other torch libraries

http://arxiv.org/abs/2004.13649

yemaedahrav commented 11 months ago

Hi, I would be interested in taking up this issue. Should I assign myself on this?

vmoens commented 11 months ago

That would be lovely! From what I understood the main technical difficulty is to do transforms in a consistent manner (eg, have an action mapping operator that follows the image/state transform etc). Not sure what form these will take but happy to help or provide some guidance if needed. If you want to discuss this not on gh, feel free to join our slack channel or discord (but gh is also perfectly fine).

yemaedahrav commented 11 months ago

I did go through the /benchmarks directory (including the commit history) and take a look at the code there and had read the paper as well. I feel I would still require more clarity on the goals this issue attempts to accomplish. Is it to get a working implementation integrated into the TorchRL library or am I missing something?

vmoens commented 11 months ago

I would say that the primary goal is that if anyone wants to build on top of this work a new research paper or an application (eg apply it to a different domain or else) it should be doable. The secondary goals are:

This hierarchy of values is important to keep in mind while implementing all this: we don't want to write a new benchmark for the sake of the script, we want the benchmark to show how to use what that paper proposes using torchrl.

I hope that makes sense!

neo-alex commented 11 months ago

I am looking forward to this DrQ implementation!

Just wondering if it wouldn't be better to rather reproduce DrQv2 directly? It should be the same amount of work but the performance is vastly superior, as shown in the first illustration of the paper: https://arxiv.org/abs/2107.09645 (the image augmentation trick is the same but the v2 made a couple of changes wrt. hyperparameters, replacing SAC with DDPG...).

By the way, you can find the official implementation of DrQv2 here as an inspiration: https://github.com/facebookresearch/drqv2

...and I believe that the core image augmentation trick is implemented here: https://github.com/facebookresearch/drqv2/blob/main/drqv2.py#L14

vmoens commented 11 months ago

That makes sense yes!

Denis (the author of the repo) isn't at meta anymore but we could still reach out if we need help!

yemaedahrav commented 11 months ago

Sounds good, is there any other such benchmark we have in this torchrl library that I can refer to?

vmoens commented 11 months ago

Not sure no. You can always check our examples in the examples folder.

PaulBeuran commented 2 days ago

Is this issue still open ? I'd like to contribute

yemaedahrav commented 1 day ago

Yes, I haven't really gotten around to completing this, however I did read up and make some progress. Maybe you could be a co-assignee on this issue and we can work together?

PaulBeuran commented 1 day ago

That would be great, thanks. Could we exchange through DIscord on the subject ? DM @pbeuran_71003 if that's the case.