search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.19k stars 289 forks source link

[BugFix] Expose MARL modules #2321

Closed vmoens closed 1 month ago

pytorch-bot[bot] commented 1 month ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2321

Note: Links to docs will display an error until the docs builds have been completed.

:x: 2 New Failures, 1 Pending, 2 Unrelated Failures

As of commit 1420edcfda189582497e327d7c57f7aa36c9dbba with merge base c6ef08017bf0f4e756dabd730530a3f3e564ed5f (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2321#27918913884) ([gh](https://github.com/pytorch/rl/actions/runs/10096425037/job/27918913884)) `RuntimeError: Command docker exec -t 745875fec0947d426d14ec857dd11f243a4e661364f402a5a81f75a2faa48d79 /exec failed with exit code 139` * [Unit-tests on Linux / tests-cpu (3.11) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2321#27918959868) ([gh](https://github.com/pytorch/rl/actions/runs/10096425056/job/27918959868)) `test/test_transforms.py::TestVecNorm::test_vecnorm_parallel_auto[5]`

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2321#27918920294) ([gh](https://github.com/pytorch/rl/actions/runs/10096425042/job/27918920294)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/c6ef08017bf0f4e756dabd730530a3f3e564ed5f#27879553577)) `AttributeError: module 'torch' has no attribute 'compiler'` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2321#27918913153) ([gh](https://github.com/pytorch/rl/actions/runs/10096425045/job/27918913153)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/c6ef08017bf0f4e756dabd730530a3f3e564ed5f#27879545997)) `test/test_transforms.py::TestActionDiscretizer::test_trans_parallel_env_check[False]`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}9$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 58.9278ms | 57.8061ms | 17.2992 Ops/s | 16.9481 Ops/s | $\color{#35bf28}+2.07\\%$ | | test_sync | 49.5898ms | 34.2150ms | 29.2270 Ops/s | 31.4207 Ops/s | $\textbf{\color{#d91a1a}-6.98\\%}$ | | test_async | 54.3993ms | 30.0475ms | 33.2806 Ops/s | 33.3922 Ops/s | $\color{#d91a1a}-0.33\\%$ | | test_simple | 0.5024s | 0.4149s | 2.4104 Ops/s | 2.4264 Ops/s | $\color{#d91a1a}-0.66\\%$ | | test_transformed | 0.6220s | 0.5681s | 1.7603 Ops/s | 1.7963 Ops/s | $\color{#d91a1a}-2.00\\%$ | | test_serial | 1.3146s | 1.2535s | 0.7977 Ops/s | 0.7870 Ops/s | $\color{#35bf28}+1.37\\%$ | | test_parallel | 1.1741s | 1.1182s | 0.8943 Ops/s | 0.9004 Ops/s | $\color{#d91a1a}-0.68\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1944ms | 25.7577μs | 38.8233 KOps/s | 39.2489 KOps/s | $\color{#d91a1a}-1.08\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 62.7670μs | 14.7677μs | 67.7154 KOps/s | 68.0905 KOps/s | $\color{#d91a1a}-0.55\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 74.8600μs | 14.7192μs | 67.9384 KOps/s | 68.4790 KOps/s | $\color{#d91a1a}-0.79\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 35.5660μs | 8.5173μs | 117.4087 KOps/s | 117.4823 KOps/s | $\color{#d91a1a}-0.06\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 95.8990μs | 27.1590μs | 36.8202 KOps/s | 36.9508 KOps/s | $\color{#d91a1a}-0.35\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 57.2380μs | 16.5384μs | 60.4655 KOps/s | 61.3710 KOps/s | $\color{#d91a1a}-1.48\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 71.1430μs | 16.3732μs | 61.0754 KOps/s | 60.2508 KOps/s | $\color{#35bf28}+1.37\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 37.4900μs | 10.1541μs | 98.4820 KOps/s | 98.4174 KOps/s | $\color{#35bf28}+0.07\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 81.6430μs | 28.9143μs | 34.5849 KOps/s | 34.4268 KOps/s | $\color{#35bf28}+0.46\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 71.0930μs | 18.2671μs | 54.7433 KOps/s | 55.8977 KOps/s | $\color{#d91a1a}-2.07\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 60.7140μs | 16.4130μs | 60.9275 KOps/s | 61.1263 KOps/s | $\color{#d91a1a}-0.33\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 55.4950μs | 10.1368μs | 98.6503 KOps/s | 98.9840 KOps/s | $\color{#d91a1a}-0.34\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 57.6080μs | 30.4272μs | 32.8653 KOps/s | 33.1382 KOps/s | $\color{#d91a1a}-0.82\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 69.8610μs | 19.6486μs | 50.8943 KOps/s | 51.2944 KOps/s | $\color{#d91a1a}-0.78\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 50.3340μs | 17.8224μs | 56.1090 KOps/s | 56.5580 KOps/s | $\color{#d91a1a}-0.79\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 70.6020μs | 11.7233μs | 85.3004 KOps/s | 85.8412 KOps/s | $\color{#d91a1a}-0.63\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 84.3880μs | 28.7841μs | 34.7414 KOps/s | 34.6425 KOps/s | $\color{#35bf28}+0.29\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 45.1640μs | 18.1976μs | 54.9524 KOps/s | 55.3029 KOps/s | $\color{#d91a1a}-0.63\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 72.5660μs | 18.9477μs | 52.7769 KOps/s | 53.1101 KOps/s | $\color{#d91a1a}-0.63\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 27.0410μs | 11.4365μs | 87.4395 KOps/s | 87.5546 KOps/s | $\color{#d91a1a}-0.13\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 83.9470μs | 30.4318μs | 32.8604 KOps/s | 33.2405 KOps/s | $\color{#d91a1a}-1.14\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 72.6660μs | 19.5906μs | 51.0448 KOps/s | 51.4463 KOps/s | $\color{#d91a1a}-0.78\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 48.5810μs | 20.3084μs | 49.2408 KOps/s | 49.4929 KOps/s | $\color{#d91a1a}-0.51\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 59.3010μs | 12.8260μs | 77.9668 KOps/s | 78.0795 KOps/s | $\color{#d91a1a}-0.14\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.3941ms | 32.7587μs | 30.5262 KOps/s | 31.2280 KOps/s | $\color{#d91a1a}-2.25\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 48.6510μs | 21.2448μs | 47.0703 KOps/s | 47.8359 KOps/s | $\color{#d91a1a}-1.60\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 53.4600μs | 20.2376μs | 49.4129 KOps/s | 48.4621 KOps/s | $\color{#35bf28}+1.96\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 39.4340μs | 12.8634μs | 77.7397 KOps/s | 78.1650 KOps/s | $\color{#d91a1a}-0.54\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 87.0130μs | 33.4323μs | 29.9112 KOps/s | 30.1205 KOps/s | $\color{#d91a1a}-0.69\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 46.8480μs | 22.5448μs | 44.3561 KOps/s | 44.3145 KOps/s | $\color{#35bf28}+0.09\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 74.8400μs | 21.5366μs | 46.4326 KOps/s | 46.5562 KOps/s | $\color{#d91a1a}-0.27\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 63.2180μs | 14.3441μs | 69.7152 KOps/s | 69.9999 KOps/s | $\color{#d91a1a}-0.41\\%$ | | test_values[generalized_advantage_estimate-True-True] | 10.8955ms | 9.7021ms | 103.0706 Ops/s | 105.2550 Ops/s | $\color{#d91a1a}-2.08\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 37.7187ms | 35.4316ms | 28.2234 Ops/s | 29.8790 Ops/s | $\textbf{\color{#d91a1a}-5.54\\%}$ | | test_values[td0_return_estimate-False-False] | 0.2466ms | 0.1744ms | 5.7334 KOps/s | 5.9124 KOps/s | $\color{#d91a1a}-3.03\\%$ | | test_values[td1_return_estimate-False-False] | 27.6493ms | 23.8539ms | 41.9219 Ops/s | 41.7875 Ops/s | $\color{#35bf28}+0.32\\%$ | | test_values[vec_td1_return_estimate-False-False] | 37.5231ms | 35.5243ms | 28.1498 Ops/s | 29.8727 Ops/s | $\textbf{\color{#d91a1a}-5.77\\%}$ | | test_values[td_lambda_return_estimate-True-False] | 38.0400ms | 34.2744ms | 29.1763 Ops/s | 28.8884 Ops/s | $\color{#35bf28}+1.00\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 37.7077ms | 35.5774ms | 28.1077 Ops/s | 29.9729 Ops/s | $\textbf{\color{#d91a1a}-6.22\\%}$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.5868ms | 8.4114ms | 118.8864 Ops/s | 118.8178 Ops/s | $\color{#35bf28}+0.06\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.3408ms | 1.9943ms | 501.4299 Ops/s | 503.0225 Ops/s | $\color{#d91a1a}-0.32\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4247ms | 0.3483ms | 2.8714 KOps/s | 2.7878 KOps/s | $\color{#35bf28}+3.00\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 48.2681ms | 45.5851ms | 21.9370 Ops/s | 21.6377 Ops/s | $\color{#35bf28}+1.38\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.9557ms | 3.0422ms | 328.7121 Ops/s | 330.0656 Ops/s | $\color{#d91a1a}-0.41\\%$ | | test_dqn_speed | 1.7220ms | 1.3600ms | 735.2795 Ops/s | 715.6047 Ops/s | $\color{#35bf28}+2.75\\%$ | | test_ddpg_speed | 3.2992ms | 2.8887ms | 346.1795 Ops/s | 339.8206 Ops/s | $\color{#35bf28}+1.87\\%$ | | test_sac_speed | 10.1468ms | 8.3852ms | 119.2580 Ops/s | 118.6032 Ops/s | $\color{#35bf28}+0.55\\%$ | | test_redq_speed | 15.6527ms | 13.7746ms | 72.5974 Ops/s | 74.7139 Ops/s | $\color{#d91a1a}-2.83\\%$ | | test_redq_deprec_speed | 18.0453ms | 13.5826ms | 73.6237 Ops/s | 73.9535 Ops/s | $\color{#d91a1a}-0.45\\%$ | | test_td3_speed | 9.0115ms | 8.3271ms | 120.0894 Ops/s | 119.7499 Ops/s | $\color{#35bf28}+0.28\\%$ | | test_cql_speed | 37.9257ms | 36.5756ms | 27.3406 Ops/s | 27.4145 Ops/s | $\color{#d91a1a}-0.27\\%$ | | test_a2c_speed | 9.0838ms | 7.5596ms | 132.2819 Ops/s | 134.1747 Ops/s | $\color{#d91a1a}-1.41\\%$ | | test_ppo_speed | 8.9879ms | 7.8664ms | 127.1226 Ops/s | 128.8012 Ops/s | $\color{#d91a1a}-1.30\\%$ | | test_reinforce_speed | 8.1108ms | 6.6227ms | 150.9966 Ops/s | 151.3628 Ops/s | $\color{#d91a1a}-0.24\\%$ | | test_iql_speed | 34.4745ms | 32.4720ms | 30.7957 Ops/s | 30.6680 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 7.6039ms | 4.9334ms | 202.6985 Ops/s | 202.5885 Ops/s | $\color{#35bf28}+0.05\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.8906ms | 0.4803ms | 2.0822 KOps/s | 2.0849 KOps/s | $\color{#d91a1a}-0.13\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6566ms | 0.4554ms | 2.1960 KOps/s | 2.1959 KOps/s | $+0.01\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.5259ms | 4.7852ms | 208.9778 Ops/s | 205.2560 Ops/s | $\color{#35bf28}+1.81\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.5734ms | 0.4711ms | 2.1227 KOps/s | 2.1134 KOps/s | $\color{#35bf28}+0.44\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6362ms | 0.4508ms | 2.2182 KOps/s | 2.1831 KOps/s | $\color{#35bf28}+1.61\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.0180ms | 1.6844ms | 593.6737 Ops/s | 585.1833 Ops/s | $\color{#35bf28}+1.45\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2047ms | 1.5944ms | 627.1962 Ops/s | 623.3717 Ops/s | $\color{#35bf28}+0.61\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.4021ms | 5.1053ms | 195.8757 Ops/s | 199.2995 Ops/s | $\color{#d91a1a}-1.72\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.7676ms | 0.6186ms | 1.6166 KOps/s | 1.6271 KOps/s | $\color{#d91a1a}-0.64\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7370ms | 0.5888ms | 1.6984 KOps/s | 1.6988 KOps/s | $\color{#d91a1a}-0.02\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.2371ms | 4.8863ms | 204.6551 Ops/s | 203.7907 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6958ms | 0.4856ms | 2.0593 KOps/s | 2.0628 KOps/s | $\color{#d91a1a}-0.17\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 7.2257ms | 0.4656ms | 2.1476 KOps/s | 2.1863 KOps/s | $\color{#d91a1a}-1.77\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.2684ms | 4.8540ms | 206.0140 Ops/s | 206.7135 Ops/s | $\color{#d91a1a}-0.34\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8664ms | 0.4793ms | 2.0864 KOps/s | 2.0994 KOps/s | $\color{#d91a1a}-0.62\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7666ms | 0.4553ms | 2.1963 KOps/s | 2.1720 KOps/s | $\color{#35bf28}+1.12\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.9198ms | 4.9904ms | 200.3850 Ops/s | 200.8018 Ops/s | $\color{#d91a1a}-0.21\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.1129s | 0.7728ms | 1.2940 KOps/s | 1.5980 KOps/s | $\textbf{\color{#d91a1a}-19.02\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7652ms | 0.5886ms | 1.6989 KOps/s | 1.6787 KOps/s | $\color{#35bf28}+1.20\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1141s | 5.9747ms | 167.3713 Ops/s | 119.6717 Ops/s | $\textbf{\color{#35bf28}+39.86\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 16.7836ms | 12.7898ms | 78.1871 Ops/s | 77.0414 Ops/s | $\color{#35bf28}+1.49\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 5.2388ms | 1.2025ms | 831.5768 Ops/s | 914.7567 Ops/s | $\textbf{\color{#d91a1a}-9.09\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1070s | 5.8031ms | 172.3207 Ops/s | 168.6356 Ops/s | $\color{#35bf28}+2.19\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 16.9603ms | 12.7563ms | 78.3928 Ops/s | 75.3186 Ops/s | $\color{#35bf28}+4.08\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 5.0206ms | 1.1901ms | 840.2551 Ops/s | 885.5925 Ops/s | $\textbf{\color{#d91a1a}-5.12\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1136s | 8.2189ms | 121.6703 Ops/s | 164.5820 Ops/s | $\textbf{\color{#d91a1a}-26.07\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 17.1747ms | 12.9211ms | 77.3929 Ops/s | 75.6691 Ops/s | $\color{#35bf28}+2.28\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 4.9979ms | 1.3383ms | 747.2251 Ops/s | 803.5026 Ops/s | $\textbf{\color{#d91a1a}-7.00\\%}$ |
github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1085s | 0.1083s | 9.2337 Ops/s | 9.3238 Ops/s | $\color{#d91a1a}-0.97\\%$ | | test_sync | 96.1498ms | 95.1794ms | 10.5065 Ops/s | 10.4798 Ops/s | $\color{#35bf28}+0.25\\%$ | | test_async | 0.1794s | 90.4379ms | 11.0573 Ops/s | 11.0329 Ops/s | $\color{#35bf28}+0.22\\%$ | | test_single_pixels | 0.1182s | 0.1179s | 8.4789 Ops/s | 8.4873 Ops/s | $\color{#d91a1a}-0.10\\%$ | | test_sync_pixels | 76.8460ms | 74.5811ms | 13.4082 Ops/s | 13.4233 Ops/s | $\color{#d91a1a}-0.11\\%$ | | test_async_pixels | 0.1395s | 69.1012ms | 14.4715 Ops/s | 14.1013 Ops/s | $\color{#35bf28}+2.63\\%$ | | test_simple | 0.8743s | 0.7986s | 1.2521 Ops/s | 1.2366 Ops/s | $\color{#35bf28}+1.26\\%$ | | test_transformed | 1.1053s | 1.0336s | 0.9674 Ops/s | 1.0001 Ops/s | $\color{#d91a1a}-3.26\\%$ | | test_serial | 2.3211s | 2.2510s | 0.4443 Ops/s | 0.4425 Ops/s | $\color{#35bf28}+0.40\\%$ | | test_parallel | 2.0418s | 1.9785s | 0.5054 Ops/s | 0.5012 Ops/s | $\color{#35bf28}+0.83\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1405ms | 37.0958μs | 26.9572 KOps/s | 27.5130 KOps/s | $\color{#d91a1a}-2.02\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 0.1820ms | 21.2908μs | 46.9686 KOps/s | 48.8479 KOps/s | $\color{#d91a1a}-3.85\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 0.2102ms | 21.3320μs | 46.8780 KOps/s | 48.7620 KOps/s | $\color{#d91a1a}-3.86\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 27.6200μs | 12.0442μs | 83.0276 KOps/s | 85.5272 KOps/s | $\color{#d91a1a}-2.92\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 57.5610μs | 39.6455μs | 25.2235 KOps/s | 25.9305 KOps/s | $\color{#d91a1a}-2.73\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 0.2193ms | 23.3659μs | 42.7974 KOps/s | 44.4144 KOps/s | $\color{#d91a1a}-3.64\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 41.1110μs | 23.1852μs | 43.1310 KOps/s | 44.0709 KOps/s | $\color{#d91a1a}-2.13\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 0.1251ms | 14.1278μs | 70.7823 KOps/s | 71.7281 KOps/s | $\color{#d91a1a}-1.32\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 66.3910μs | 41.8011μs | 23.9228 KOps/s | 24.9453 KOps/s | $\color{#d91a1a}-4.10\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 45.5810μs | 25.5993μs | 39.0635 KOps/s | 40.0440 KOps/s | $\color{#d91a1a}-2.45\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 40.1010μs | 22.7180μs | 44.0180 KOps/s | 44.3313 KOps/s | $\color{#d91a1a}-0.71\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 31.5510μs | 14.1000μs | 70.9220 KOps/s | 72.7164 KOps/s | $\color{#d91a1a}-2.47\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 62.2220μs | 42.9041μs | 23.3078 KOps/s | 23.2761 KOps/s | $\color{#35bf28}+0.14\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 46.0510μs | 27.8352μs | 35.9257 KOps/s | 36.6415 KOps/s | $\color{#d91a1a}-1.95\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 45.2410μs | 25.0504μs | 39.9194 KOps/s | 40.1405 KOps/s | $\color{#d91a1a}-0.55\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 37.8510μs | 16.1591μs | 61.8847 KOps/s | 62.6790 KOps/s | $\color{#d91a1a}-1.27\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 70.3220μs | 41.5141μs | 24.0882 KOps/s | 24.7934 KOps/s | $\color{#d91a1a}-2.84\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 0.1301ms | 25.4152μs | 39.3465 KOps/s | 40.2057 KOps/s | $\color{#d91a1a}-2.14\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 64.7720μs | 27.6405μs | 36.1788 KOps/s | 37.4175 KOps/s | $\color{#d91a1a}-3.31\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 0.1767ms | 16.2737μs | 61.4490 KOps/s | 62.9768 KOps/s | $\color{#d91a1a}-2.43\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 0.2342ms | 43.6739μs | 22.8970 KOps/s | 23.2989 KOps/s | $\color{#d91a1a}-1.72\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 90.7320μs | 27.7617μs | 36.0209 KOps/s | 36.8832 KOps/s | $\color{#d91a1a}-2.34\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 49.6910μs | 29.3976μs | 34.0163 KOps/s | 34.6518 KOps/s | $\color{#d91a1a}-1.83\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 0.2125ms | 18.5283μs | 53.9714 KOps/s | 55.2290 KOps/s | $\color{#d91a1a}-2.28\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.9132ms | 46.5919μs | 21.4630 KOps/s | 22.1537 KOps/s | $\color{#d91a1a}-3.12\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 59.9820μs | 30.2052μs | 33.1069 KOps/s | 34.0757 KOps/s | $\color{#d91a1a}-2.84\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 0.1742ms | 29.3902μs | 34.0250 KOps/s | 34.3493 KOps/s | $\color{#d91a1a}-0.94\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 0.1115ms | 18.3915μs | 54.3729 KOps/s | 54.0061 KOps/s | $\color{#35bf28}+0.68\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 79.0320μs | 47.3963μs | 21.0987 KOps/s | 21.7921 KOps/s | $\color{#d91a1a}-3.18\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 53.4110μs | 32.0818μs | 31.1704 KOps/s | 32.1542 KOps/s | $\color{#d91a1a}-3.06\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 60.0810μs | 31.5176μs | 31.7283 KOps/s | 32.5059 KOps/s | $\color{#d91a1a}-2.39\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 37.4810μs | 20.2299μs | 49.4317 KOps/s | 50.0368 KOps/s | $\color{#d91a1a}-1.21\\%$ | | test_values[generalized_advantage_estimate-True-True] | 25.5451ms | 24.4371ms | 40.9213 Ops/s | 41.4756 Ops/s | $\color{#d91a1a}-1.34\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 96.6744ms | 2.8377ms | 352.4016 Ops/s | 353.6507 Ops/s | $\color{#d91a1a}-0.35\\%$ | | test_values[td0_return_estimate-False-False] | 93.2720μs | 67.5823μs | 14.7968 KOps/s | 15.2973 KOps/s | $\color{#d91a1a}-3.27\\%$ | | test_values[td1_return_estimate-False-False] | 54.9462ms | 54.5225ms | 18.3410 Ops/s | 18.3130 Ops/s | $\color{#35bf28}+0.15\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.5243ms | 1.0905ms | 917.0223 Ops/s | 922.8242 Ops/s | $\color{#d91a1a}-0.63\\%$ | | test_values[td_lambda_return_estimate-True-False] | 87.5307ms | 86.9039ms | 11.5070 Ops/s | 11.4760 Ops/s | $\color{#35bf28}+0.27\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.2475ms | 1.0794ms | 926.4754 Ops/s | 924.1566 Ops/s | $\color{#35bf28}+0.25\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 24.9305ms | 24.5761ms | 40.6900 Ops/s | 41.1428 Ops/s | $\color{#d91a1a}-1.10\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9583ms | 0.7231ms | 1.3830 KOps/s | 1.3426 KOps/s | $\color{#35bf28}+3.01\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.8154ms | 0.6688ms | 1.4951 KOps/s | 1.4906 KOps/s | $\color{#35bf28}+0.31\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.6123ms | 1.4698ms | 680.3703 Ops/s | 679.6469 Ops/s | $\color{#35bf28}+0.11\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.8391ms | 0.6859ms | 1.4580 KOps/s | 1.4642 KOps/s | $\color{#d91a1a}-0.42\\%$ | | test_dqn_speed | 7.3264ms | 1.4767ms | 677.1993 Ops/s | 698.2179 Ops/s | $\color{#d91a1a}-3.01\\%$ | | test_ddpg_speed | 3.3476ms | 2.9997ms | 333.3636 Ops/s | 340.0717 Ops/s | $\color{#d91a1a}-1.97\\%$ | | test_sac_speed | 0.1015s | 9.2721ms | 107.8505 Ops/s | 120.0286 Ops/s | $\textbf{\color{#d91a1a}-10.15\\%}$ | | test_redq_speed | 11.6812ms | 10.9639ms | 91.2081 Ops/s | 91.9535 Ops/s | $\color{#d91a1a}-0.81\\%$ | | test_redq_deprec_speed | 12.2096ms | 11.6216ms | 86.0469 Ops/s | 88.1535 Ops/s | $\color{#d91a1a}-2.39\\%$ | | test_td3_speed | 9.3279ms | 8.4638ms | 118.1502 Ops/s | 120.5186 Ops/s | $\color{#d91a1a}-1.97\\%$ | | test_cql_speed | 27.2289ms | 26.4924ms | 37.7466 Ops/s | 34.9408 Ops/s | $\textbf{\color{#35bf28}+8.03\\%}$ | | test_a2c_speed | 6.1245ms | 5.8224ms | 171.7515 Ops/s | 180.0292 Ops/s | $\color{#d91a1a}-4.60\\%$ | | test_ppo_speed | 6.3865ms | 6.1717ms | 162.0293 Ops/s | 169.8192 Ops/s | $\color{#d91a1a}-4.59\\%$ | | test_reinforce_speed | 4.9910ms | 4.6583ms | 214.6715 Ops/s | 221.1294 Ops/s | $\color{#d91a1a}-2.92\\%$ | | test_iql_speed | 20.8481ms | 20.1504ms | 49.6267 Ops/s | 50.9632 Ops/s | $\color{#d91a1a}-2.62\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.7903ms | 6.6030ms | 151.4472 Ops/s | 151.6862 Ops/s | $\color{#d91a1a}-0.16\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.1098s | 0.5917ms | 1.6900 KOps/s | 1.9286 KOps/s | $\textbf{\color{#d91a1a}-12.37\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7463ms | 0.4937ms | 2.0255 KOps/s | 2.0306 KOps/s | $\color{#d91a1a}-0.25\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.8894ms | 6.5429ms | 152.8376 Ops/s | 154.1314 Ops/s | $\color{#d91a1a}-0.84\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.5560ms | 0.5111ms | 1.9567 KOps/s | 1.9692 KOps/s | $\color{#d91a1a}-0.64\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7203ms | 0.4891ms | 2.0447 KOps/s | 2.0580 KOps/s | $\color{#d91a1a}-0.65\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.1735ms | 1.9712ms | 507.3112 Ops/s | 509.1050 Ops/s | $\color{#d91a1a}-0.35\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.1128ms | 1.8784ms | 532.3766 Ops/s | 531.3304 Ops/s | $\color{#35bf28}+0.20\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.9806ms | 6.7382ms | 148.4075 Ops/s | 148.5027 Ops/s | $\color{#d91a1a}-0.06\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.1872ms | 0.6663ms | 1.5008 KOps/s | 1.5101 KOps/s | $\color{#d91a1a}-0.62\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8556ms | 0.6439ms | 1.5530 KOps/s | 1.5701 KOps/s | $\color{#d91a1a}-1.09\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.8067ms | 6.6229ms | 150.9915 Ops/s | 152.4648 Ops/s | $\color{#d91a1a}-0.97\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.4676ms | 0.5154ms | 1.9402 KOps/s | 1.9368 KOps/s | $\color{#35bf28}+0.18\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6735ms | 0.4944ms | 2.0228 KOps/s | 2.0284 KOps/s | $\color{#d91a1a}-0.27\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.9669ms | 6.5448ms | 152.7927 Ops/s | 153.2142 Ops/s | $\color{#d91a1a}-0.28\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7257ms | 0.5114ms | 1.9553 KOps/s | 1.9635 KOps/s | $\color{#d91a1a}-0.42\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 4.1760ms | 0.4962ms | 2.0155 KOps/s | 2.0191 KOps/s | $\color{#d91a1a}-0.18\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.9355ms | 6.7830ms | 147.4275 Ops/s | 148.2753 Ops/s | $\color{#d91a1a}-0.57\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.1720ms | 0.6707ms | 1.4911 KOps/s | 1.4939 KOps/s | $\color{#d91a1a}-0.19\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7888ms | 0.6485ms | 1.5419 KOps/s | 1.5513 KOps/s | $\color{#d91a1a}-0.60\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1444s | 7.9296ms | 126.1103 Ops/s | 126.5520 Ops/s | $\color{#d91a1a}-0.35\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 20.4348ms | 16.0599ms | 62.2670 Ops/s | 63.1001 Ops/s | $\color{#d91a1a}-1.32\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.4940ms | 1.2770ms | 783.0567 Ops/s | 788.7407 Ops/s | $\color{#d91a1a}-0.72\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1268s | 7.6088ms | 131.4272 Ops/s | 131.8763 Ops/s | $\color{#d91a1a}-0.34\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1362s | 18.2717ms | 54.7296 Ops/s | 63.3803 Ops/s | $\textbf{\color{#d91a1a}-13.65\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.5269ms | 1.2734ms | 785.2687 Ops/s | 794.2537 Ops/s | $\color{#d91a1a}-1.13\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1252s | 7.7271ms | 129.4149 Ops/s | 129.2417 Ops/s | $\color{#35bf28}+0.13\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.5694ms | 15.9860ms | 62.5546 Ops/s | 62.5929 Ops/s | $\color{#d91a1a}-0.06\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.3865ms | 1.4086ms | 709.9319 Ops/s | 697.9760 Ops/s | $\color{#35bf28}+1.71\\%$ |