In this PR we have introduced our new buffers, that handle data with standard dictionaries of numpy arrays instead of TensorDicts. This change has been introduced to keep algorithms implementation and interaction as simple as possible, while also gaining in terms of speed performance. In particular:
All the buffers have been converted to work with standard dictionaries of numpy arrays instead of TensorDicts
We have introduced the class sheeprl.utils.memmap.MemmapArray, which is a simple container to work with numpy memory-mapped arrays. Those arrays can be easily shared between processes that share the same underlying file-system and once shared they maintain the ownership of the array being shared
The EpisodeBuffer api has been made compliant to the ReplayBuffer api:
the add method takes in input one or more steps (possibly containing multiple episodes) instead of a single episode
The CheckpointCallback has been adapted to our new buffers
Type of Change
Please select the one relevant option below:
New feature (non-breaking change that adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Checklist
Please confirm that the following tasks have been completed:
[x] I have tested my changes locally and they work as expected. (Please describe the tests you performed.)
[x] I have added unit tests for my changes, or updated existing tests if necessary.
[x] I have updated the documentation, if applicable.
[x] I have installed pre-commit and run locally for my code changes.
Thank you for your contribution! Once you have filled out this template, please ensure that you have assigned the appropriate reviewers and that all tests have passed.
Summary
In this PR we have introduced our new buffers, that handle data with standard dictionaries of numpy arrays instead of TensorDicts. This change has been introduced to keep algorithms implementation and interaction as simple as possible, while also gaining in terms of speed performance. In particular:
sheeprl.utils.memmap.MemmapArray, which is a simple container to work with numpy memory-mapped arrays. Those arrays can be easily shared between processes that share the same underlying file-system and once shared they maintain the ownership of the array being sharedEpisodeBufferapi has been made compliant to theReplayBufferapi:addmethod takes in input one or more steps (possibly containing multiple episodes) instead of a single episodeCheckpointCallbackhas been adapted to our new buffersType of Change
Please select the one relevant option below:
Checklist
Please confirm that the following tasks have been completed:
Thank you for your contribution! Once you have filled out this template, please ensure that you have assigned the appropriate reviewers and that all tests have passed.