Eclectic-Sheep / sheeprl

Distributed Reinforcement Learning accelerated by Lightning Fabric
https://eclecticsheep.ai
Apache License 2.0
300 stars 29 forks source link

Save memory mapped buffers to specified folder #61

Closed belerico closed 1 year ago

belerico commented 1 year ago

When buffers (i.e. the TensorDict) are memory mapped, those are by default mapped to tempfiles, which are cleared once everything is done (https://github.com/pytorch-labs/tensordict/blob/main/tensordict/memmap.py#L560): If we are going to resume a training from a checkpoint (for Dreamer's algos), the buffer is lost. We need to specify where to save the memory mapped buffers upon buffer creation.