Minimal example for building a TorchRL environment

[thomashirtz]

Motivation

I would like to use this library to do some very efficient training. I was therefore was planning to do write my environment using the TorchRL structure EnvBase. However I can't seem to find good minimal working example to show all the properties to set etc.

Solution

Add example in the documentation on how to create a TorchRL environment, or give the source of a repository that has one... Or even better, include an example in the repository's code.

Checklist

• [x] I have checked that there is no similar issue in the repo (required)

[vmoens]

Thanks for the suggestion! This is indeed something we should work on. Do you think more of a tutorial or simply some doc? Are you more interested in building your env by defining every operation, or building it on top of an existing simulator (we should write instructions for both but we'll prioritize the use case that suits your needs)?

[thomashirtz]

I think the fix with the minimal effort that would help the most the people would be the implementation of a very small environment (then simply add a section creating your own TorchRL environment in the environment page of the doc with a link towards the code). I remember that the environment fast from brax helped me a lot to design JAX envs as it was very very simple yet all the needed properties and attributes needed were shown

In my case it would be from scratch (every operation). I am trying to create an environment that either could take advantage of accelerators using torch (not sure to what extent it is possible, e.g. run batches of env at the same time), or more simply an environment that runs fast. And I felt that creating it from scratch would allow to troubleshoot better the bottlenecks, in contrario to a case where it is wrapped