Eclectic-Sheep / sheeprl

Distributed Reinforcement Learning accelerated by Lightning Fabric
https://eclecticsheep.ai
Apache License 2.0
305 stars 31 forks source link

Fix/bernoulli #186

Closed belerico closed 8 months ago

belerico commented 8 months ago

Summary

This PR possibly fixes #147 by introducing a safe Bernoulli distribution, in which the mode is computed in a TF-like fashion (https://github.com/tensorflow/tensorflow/blob/e9552f222b3237aafa4773e3f96a8cefb3ebaf95/tensorflow/python/ops/distributions/bernoulli.py#L160)

Type of Change

Please select the one relevant option below:

Checklist

Please confirm that the following tasks have been completed:

Screenshots or Visuals (Optional)

If applicable, please provide screenshots, diagrams, graphs, or videos of the changes, features or the error.

To prove this i've run a Walker Walk experiment with the following command:

MUJOCO_GL=egl python sheeprl.py exp=dreamer_v3_dmc_walker_walk.yaml fabric.accelerator=gpu fabric.precision=bf16-mixed env.num_envs=4 env.max_episode_stesp=-1

on a Pop-OS laptop with Ubuntu 22.04, an Intel i5-13500H and a NVIDIA RTX 4060 8GB

image image

The experiment has run for more than 8h before being manually stopped.

Thank you for your contribution! Once you have filled out this template, please ensure that you have assigned the appropriate reviewers and that all tests have passed.