Accounting for Episodes with Execution Failures

[peterdavidfagan]

In robotics it is common to have rollouts that fail for one reason or another. When using envlogger for collecting robotics datasets with scripted behaviours there is currently no fallback mechanism to prevent envlogger for erroring due to failures in writing episode data for the failed episodes.

A workaround is to define a dummy action/step in the environment which denotes a failure case, this however is a bit messy and requires further post processing scripts to parse out failed episodes.

I have started looking at the source code and didn't see any functionality which enables discarding an episode during data collection. I am opening this issue to start a discussion around this topic and the potential for adding fallbacks for errored/failed episodes that occur during data collection.

Example: In a scripted pick/place sim environment, object placement relies on sampling a collision free placement pose. There are currently a fixed number of iterations for this placement pose generation, if it fails during data collection it raises and error and resets the environment , envlogger errors on subsequent episode due to write errors (I will look to include an example trace when I regenerate this error). For context pick/place data collection can be found here.

In a similar vain, using envlogger in real-world environment data collection also encounters the same issue.

New Feature A useful feature would be to have a method which clears/resets episode history if/when an episode fails.

[peterdavidfagan]

Closing as it probably makes more sense to handle exceptions outside of envlogger for now.