search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.24k stars 296 forks source link

[Versioning] v0.5 bump #2267

Closed vmoens closed 2 months ago

vmoens commented 3 months ago

Depends on https://github.com/pytorch/tensordict/pull/848

pytorch-bot[bot] commented 3 months ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2267

Note: Links to docs will display an error until the docs builds have been completed.

:x: 3 New Failures, 1 Pending, 3 Unrelated Failures

As of commit 4315f0c19add8660bec0949810ecb6336839bda1 with merge base 28acf6159d81c59c313c2626d8cd8a72a8212ee4 (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2267#27264484382) ([gh](https://github.com/pytorch/rl/actions/runs/9873052085/job/27264484382)) `RuntimeError: Command docker exec -t 45f464e1022bdeec8de7f316fcdd8ae162a6d0bb3b1def5621ca34ebf2646630 /exec failed with exit code 139` * [Unit-tests on Linux / tests-cpu (3.11) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2267#27264512065) ([gh](https://github.com/pytorch/rl/actions/runs/9873052089/job/27264512065)) `RuntimeError: Command docker exec -t 43d0829d0985e2c8fca4f66a65b4144c44ce016dde13ab74961fa7344b22a74c /exec failed with exit code 1` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2267#27264515467) ([gh](https://github.com/pytorch/rl/actions/runs/9873052089/job/27264515467)) `RuntimeError: Command docker exec -t 89fb276714541493ae7ac674c863e1c53984c63ee166390df75a858ccde08da4 /exec failed with exit code 1`

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Libs Tests on Linux / unittests-robohive (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2267#27264518175) ([gh](https://github.com/pytorch/rl/actions/runs/9873052075/job/27264518175)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/28acf6159d81c59c313c2626d8cd8a72a8212ee4#27263825644)) `test/test_libs.py::TestRoboHive::test_robohive[franka_slide_random-v3-True-True]` * [Libs Tests on Linux / unittests-vmas (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2267#27264519791) ([gh](https://github.com/pytorch/rl/actions/runs/9873052075/job/27264519791)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/28acf6159d81c59c313c2626d8cd8a72a8212ee4#27263827155)) `test/test_libs.py::TestVmas::test_vmas_repr[road_traffic-1-20]` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2267#27264490000) ([gh](https://github.com/pytorch/rl/actions/runs/9873052052/job/27264490000)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/28acf6159d81c59c313c2626d8cd8a72a8212ee4#27263815657)) `test/test_transforms.py::TestActionDiscretizer::test_trans_parallel_env_check[False]`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 3 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ------------------------------------ | | test_single | 0.1649s | 0.1122s | 8.9112 Ops/s | 9.5264 Ops/s | $\textbf{\color{#d91a1a}-6.46\\%}$ | | test_sync | 94.5247ms | 93.3356ms | 10.7140 Ops/s | 10.9460 Ops/s | $\color{#d91a1a}-2.12\\%$ | | test_async | 0.1784s | 90.2347ms | 11.0822 Ops/s | 11.1726 Ops/s | $\color{#d91a1a}-0.81\\%$ | | test_single_pixels | 0.1186s | 0.1172s | 8.5316 Ops/s | 8.6351 Ops/s | $\color{#d91a1a}-1.20\\%$ | | test_sync_pixels | 77.2956ms | 75.4899ms | 13.2468 Ops/s | 14.4214 Ops/s | $\textbf{\color{#d91a1a}-8.14\\%}$ | | test_async_pixels | 0.1410s | 71.9670ms | 13.8953 Ops/s | 15.6238 Ops/s | $\textbf{\color{#d91a1a}-11.06\\%}$ | | test_simple | 0.7635s | 0.7629s | 1.3107 Ops/s | 1.3562 Ops/s | $\color{#d91a1a}-3.35\\%$ | | test_transformed | 1.0696s | 1.0079s | 0.9922 Ops/s | 1.0063 Ops/s | $\color{#d91a1a}-1.40\\%$ | | test_serial | 2.3193s | 2.2586s | 0.4428 Ops/s | 0.4540 Ops/s | $\color{#d91a1a}-2.48\\%$ | | test_parallel | 2.0366s | 1.9771s | 0.5058 Ops/s | 0.5335 Ops/s | $\textbf{\color{#d91a1a}-5.19\\%}$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1118ms | 34.2326μs | 29.2119 KOps/s | 28.9425 KOps/s | $\color{#35bf28}+0.93\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 47.8610μs | 19.8927μs | 50.2696 KOps/s | 50.8634 KOps/s | $\color{#d91a1a}-1.17\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 34.1600μs | 19.6216μs | 50.9642 KOps/s | 52.0940 KOps/s | $\color{#d91a1a}-2.17\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 26.5600μs | 11.3550μs | 88.0671 KOps/s | 89.3429 KOps/s | $\color{#d91a1a}-1.43\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 52.6500μs | 36.0947μs | 27.7049 KOps/s | 27.4360 KOps/s | $\color{#35bf28}+0.98\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 48.1710μs | 21.8811μs | 45.7014 KOps/s | 46.2625 KOps/s | $\color{#d91a1a}-1.21\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 38.2110μs | 21.6323μs | 46.2271 KOps/s | 47.6148 KOps/s | $\color{#d91a1a}-2.91\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 33.3010μs | 13.2044μs | 75.7321 KOps/s | 76.0557 KOps/s | $\color{#d91a1a}-0.43\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 62.1700μs | 38.3051μs | 26.1062 KOps/s | 25.7879 KOps/s | $\color{#35bf28}+1.23\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 40.7200μs | 24.0710μs | 41.5437 KOps/s | 42.8290 KOps/s | $\color{#d91a1a}-3.00\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 44.3210μs | 21.2020μs | 47.1654 KOps/s | 47.4806 KOps/s | $\color{#d91a1a}-0.66\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 35.6500μs | 13.2083μs | 75.7097 KOps/s | 75.0945 KOps/s | $\color{#35bf28}+0.82\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 63.4210μs | 39.9589μs | 25.0257 KOps/s | 25.0466 KOps/s | $\color{#d91a1a}-0.08\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 47.6700μs | 25.5002μs | 39.2154 KOps/s | 39.4824 KOps/s | $\color{#d91a1a}-0.68\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 39.1700μs | 22.8928μs | 43.6819 KOps/s | 43.2545 KOps/s | $\color{#35bf28}+0.99\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 36.9800μs | 14.9937μs | 66.6946 KOps/s | 67.1436 KOps/s | $\color{#d91a1a}-0.67\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 64.8410μs | 37.9601μs | 26.3435 KOps/s | 26.5515 KOps/s | $\color{#d91a1a}-0.78\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 46.8610μs | 23.9485μs | 41.7562 KOps/s | 42.6159 KOps/s | $\color{#d91a1a}-2.02\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 57.7510μs | 25.3469μs | 39.4525 KOps/s | 39.3811 KOps/s | $\color{#35bf28}+0.18\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 36.6800μs | 15.2112μs | 65.7411 KOps/s | 67.3411 KOps/s | $\color{#d91a1a}-2.38\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 56.7710μs | 39.3794μs | 25.3940 KOps/s | 24.9843 KOps/s | $\color{#35bf28}+1.64\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 49.7110μs | 25.5493μs | 39.1401 KOps/s | 38.9239 KOps/s | $\color{#35bf28}+0.56\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 47.2900μs | 27.0210μs | 37.0083 KOps/s | 36.4222 KOps/s | $\color{#35bf28}+1.61\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 39.7410μs | 16.9963μs | 58.8363 KOps/s | 59.0788 KOps/s | $\color{#d91a1a}-0.41\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 68.0710μs | 41.8362μs | 23.9028 KOps/s | 24.1449 KOps/s | $\color{#d91a1a}-1.00\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 51.7100μs | 27.5573μs | 36.2880 KOps/s | 37.0772 KOps/s | $\color{#d91a1a}-2.13\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 54.6610μs | 27.0272μs | 36.9997 KOps/s | 37.5693 KOps/s | $\color{#d91a1a}-1.52\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 36.8810μs | 17.1236μs | 58.3988 KOps/s | 60.4981 KOps/s | $\color{#d91a1a}-3.47\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 66.0900μs | 44.0781μs | 22.6870 KOps/s | 22.7792 KOps/s | $\color{#d91a1a}-0.40\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 50.0290μs | 29.4783μs | 33.9232 KOps/s | 34.0164 KOps/s | $\color{#d91a1a}-0.27\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 47.6410μs | 28.4873μs | 35.1034 KOps/s | 35.5614 KOps/s | $\color{#d91a1a}-1.29\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 34.9210μs | 18.7332μs | 53.3810 KOps/s | 54.7835 KOps/s | $\color{#d91a1a}-2.56\\%$ | | test_values[generalized_advantage_estimate-True-True] | 25.8981ms | 25.4923ms | 39.2276 Ops/s | 39.3534 Ops/s | $\color{#d91a1a}-0.32\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 89.7119ms | 2.6986ms | 370.5671 Ops/s | 92.3762 Ops/s | $\textbf{\color{#35bf28}+301.15\\%}$ | | test_values[td0_return_estimate-False-False] | 88.8410μs | 66.3651μs | 15.0682 KOps/s | 15.1694 KOps/s | $\color{#d91a1a}-0.67\\%$ | | test_values[td1_return_estimate-False-False] | 57.2822ms | 56.8454ms | 17.5916 Ops/s | 17.5073 Ops/s | $\color{#35bf28}+0.48\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.3531ms | 1.0914ms | 916.2220 Ops/s | 910.5352 Ops/s | $\color{#35bf28}+0.62\\%$ | | test_values[td_lambda_return_estimate-True-False] | 90.8045ms | 89.7188ms | 11.1459 Ops/s | 11.1158 Ops/s | $\color{#35bf28}+0.27\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.4198ms | 1.0881ms | 919.0277 Ops/s | 919.6351 Ops/s | $\color{#d91a1a}-0.07\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 26.8516ms | 26.5177ms | 37.7107 Ops/s | 38.2449 Ops/s | $\color{#d91a1a}-1.40\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9647ms | 0.7403ms | 1.3508 KOps/s | 1.3472 KOps/s | $\color{#35bf28}+0.26\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7520ms | 0.6731ms | 1.4857 KOps/s | 1.4792 KOps/s | $\color{#35bf28}+0.43\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5384ms | 1.4744ms | 678.2308 Ops/s | 678.4811 Ops/s | $\color{#d91a1a}-0.04\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7256ms | 0.6878ms | 1.4540 KOps/s | 1.4469 KOps/s | $\color{#35bf28}+0.49\\%$ | | test_dqn_speed | 1.6077ms | 1.4508ms | 689.2905 Ops/s | 665.4649 Ops/s | $\color{#35bf28}+3.58\\%$ | | test_ddpg_speed | 3.2879ms | 2.9636ms | 337.4326 Ops/s | 336.9094 Ops/s | $\color{#35bf28}+0.16\\%$ | | test_sac_speed | 8.7992ms | 8.4711ms | 118.0485 Ops/s | 116.2392 Ops/s | $\color{#35bf28}+1.56\\%$ | | test_redq_speed | 11.8987ms | 10.7029ms | 93.4322 Ops/s | 83.9871 Ops/s | $\textbf{\color{#35bf28}+11.25\\%}$ | | test_redq_deprec_speed | 0.1077s | 12.8355ms | 77.9088 Ops/s | 84.9920 Ops/s | $\textbf{\color{#d91a1a}-8.33\\%}$ | | test_td3_speed | 8.5523ms | 8.3586ms | 119.6370 Ops/s | 117.9980 Ops/s | $\color{#35bf28}+1.39\\%$ | | test_cql_speed | 26.6213ms | 25.8830ms | 38.6354 Ops/s | 38.2882 Ops/s | $\color{#35bf28}+0.91\\%$ | | test_a2c_speed | 6.0959ms | 5.7745ms | 173.1750 Ops/s | 172.9832 Ops/s | $\color{#35bf28}+0.11\\%$ | | test_ppo_speed | 6.2955ms | 6.0663ms | 164.8454 Ops/s | 162.3941 Ops/s | $\color{#35bf28}+1.51\\%$ | | test_reinforce_speed | 4.9955ms | 4.7390ms | 211.0154 Ops/s | 214.7722 Ops/s | $\color{#d91a1a}-1.75\\%$ | | test_iql_speed | 20.5361ms | 19.7464ms | 50.6423 Ops/s | 50.5402 Ops/s | $\color{#35bf28}+0.20\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.7657ms | 4.6798ms | 213.6841 Ops/s | 215.5351 Ops/s | $\color{#d91a1a}-0.86\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7285ms | 0.5359ms | 1.8661 KOps/s | 1.8642 KOps/s | $\color{#35bf28}+0.10\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 4.5295ms | 0.5186ms | 1.9282 KOps/s | 1.6869 KOps/s | $\textbf{\color{#35bf28}+14.31\\%}$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.8267ms | 4.6458ms | 215.2464 Ops/s | 216.0486 Ops/s | $\color{#d91a1a}-0.37\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7076ms | 0.5283ms | 1.8930 KOps/s | 1.8939 KOps/s | $\color{#d91a1a}-0.05\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 4.6375ms | 0.5101ms | 1.9602 KOps/s | 1.9613 KOps/s | $\color{#d91a1a}-0.05\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.1643ms | 1.9795ms | 505.1899 Ops/s | 506.8222 Ops/s | $\color{#d91a1a}-0.32\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 5.8010ms | 1.9159ms | 521.9615 Ops/s | 530.2792 Ops/s | $\color{#d91a1a}-1.57\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0440ms | 4.7591ms | 210.1234 Ops/s | 209.2867 Ops/s | $\color{#35bf28}+0.40\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8218ms | 0.6840ms | 1.4619 KOps/s | 1.4633 KOps/s | $\color{#d91a1a}-0.10\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.5232ms | 0.6660ms | 1.5015 KOps/s | 1.5041 KOps/s | $\color{#d91a1a}-0.17\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.7502ms | 4.6747ms | 213.9177 Ops/s | 213.4197 Ops/s | $\color{#35bf28}+0.23\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6539ms | 0.5359ms | 1.8661 KOps/s | 1.8715 KOps/s | $\color{#d91a1a}-0.29\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7019ms | 0.5139ms | 1.9460 KOps/s | 1.9395 KOps/s | $\color{#35bf28}+0.33\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.0479ms | 4.6201ms | 216.4453 Ops/s | 214.5959 Ops/s | $\color{#35bf28}+0.86\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.3712ms | 0.5312ms | 1.8825 KOps/s | 1.3802 KOps/s | $\textbf{\color{#35bf28}+36.39\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6617ms | 0.5116ms | 1.9548 KOps/s | 1.9721 KOps/s | $\color{#d91a1a}-0.88\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0142ms | 4.8073ms | 208.0168 Ops/s | 207.5506 Ops/s | $\color{#35bf28}+0.22\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.7788ms | 0.6769ms | 1.4774 KOps/s | 1.4472 KOps/s | $\color{#35bf28}+2.09\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7891ms | 0.6579ms | 1.5201 KOps/s | 1.4957 KOps/s | $\color{#35bf28}+1.63\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1268s | 7.3057ms | 136.8787 Ops/s | 133.9599 Ops/s | $\color{#35bf28}+2.18\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.5834ms | 15.7190ms | 63.6171 Ops/s | 63.7762 Ops/s | $\color{#d91a1a}-0.25\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.1375ms | 1.1148ms | 897.0208 Ops/s | 888.4387 Ops/s | $\color{#35bf28}+0.97\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1272s | 9.7352ms | 102.7204 Ops/s | 102.0820 Ops/s | $\color{#35bf28}+0.63\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 18.3253ms | 15.7750ms | 63.3914 Ops/s | 63.5517 Ops/s | $\color{#d91a1a}-0.25\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.3721ms | 1.1306ms | 884.4515 Ops/s | 894.1833 Ops/s | $\color{#d91a1a}-1.09\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1261s | 7.4145ms | 134.8712 Ops/s | 133.2966 Ops/s | $\color{#35bf28}+1.18\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.3763ms | 15.7247ms | 63.5942 Ops/s | 63.0877 Ops/s | $\color{#35bf28}+0.80\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 7.5567ms | 1.4086ms | 709.9254 Ops/s | 710.5209 Ops/s | $\color{#d91a1a}-0.08\\%$ |
github-actions[bot] commented 3 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1232s | 60.5213ms | 16.5231 Ops/s | 17.8072 Ops/s | $\textbf{\color{#d91a1a}-7.21\\%}$ | | test_sync | 41.5404ms | 31.1466ms | 32.1062 Ops/s | 32.0362 Ops/s | $\color{#35bf28}+0.22\\%$ | | test_async | 51.2258ms | 28.2293ms | 35.4242 Ops/s | 35.5282 Ops/s | $\color{#d91a1a}-0.29\\%$ | | test_simple | 0.3828s | 0.3795s | 2.6353 Ops/s | 2.5995 Ops/s | $\color{#35bf28}+1.38\\%$ | | test_transformed | 0.5496s | 0.5388s | 1.8558 Ops/s | 1.8118 Ops/s | $\color{#35bf28}+2.43\\%$ | | test_serial | 1.3249s | 1.2694s | 0.7878 Ops/s | 0.7801 Ops/s | $\color{#35bf28}+0.99\\%$ | | test_parallel | 1.1464s | 1.0928s | 0.9151 Ops/s | 0.9065 Ops/s | $\color{#35bf28}+0.94\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1369ms | 22.3819μs | 44.6789 KOps/s | 44.6528 KOps/s | $\color{#35bf28}+0.06\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 37.8700μs | 13.0977μs | 76.3496 KOps/s | 76.4305 KOps/s | $\color{#d91a1a}-0.11\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 31.7900μs | 13.0471μs | 76.6455 KOps/s | 77.2922 KOps/s | $\color{#d91a1a}-0.84\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 29.0940μs | 7.7119μs | 129.6690 KOps/s | 131.3947 KOps/s | $\color{#d91a1a}-1.31\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 59.7120μs | 23.8503μs | 41.9282 KOps/s | 42.1207 KOps/s | $\color{#d91a1a}-0.46\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 36.0880μs | 14.4937μs | 68.9955 KOps/s | 65.4431 KOps/s | $\textbf{\color{#35bf28}+5.43\\%}$ | | test_step_mdp_speed[True-True-False-False-True] | 39.2130μs | 14.3771μs | 69.5548 KOps/s | 71.0088 KOps/s | $\color{#d91a1a}-2.05\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 39.6440μs | 8.8974μs | 112.3921 KOps/s | 112.8174 KOps/s | $\color{#d91a1a}-0.38\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 60.1820μs | 25.0945μs | 39.8494 KOps/s | 40.4232 KOps/s | $\color{#d91a1a}-1.42\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 0.1228ms | 16.6059μs | 60.2194 KOps/s | 63.2662 KOps/s | $\color{#d91a1a}-4.82\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 37.5110μs | 14.2997μs | 69.9313 KOps/s | 70.8707 KOps/s | $\color{#d91a1a}-1.33\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 35.2360μs | 8.9562μs | 111.6546 KOps/s | 112.2225 KOps/s | $\color{#d91a1a}-0.51\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 62.1060μs | 26.2948μs | 38.0303 KOps/s | 38.2307 KOps/s | $\color{#d91a1a}-0.52\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 43.8020μs | 16.8979μs | 59.1788 KOps/s | 58.9993 KOps/s | $\color{#35bf28}+0.30\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 45.4350μs | 15.4134μs | 64.8788 KOps/s | 65.2834 KOps/s | $\color{#d91a1a}-0.62\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 46.4870μs | 10.0907μs | 99.1012 KOps/s | 99.4497 KOps/s | $\color{#d91a1a}-0.35\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 0.1340ms | 25.3579μs | 39.4355 KOps/s | 39.8006 KOps/s | $\color{#d91a1a}-0.92\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 0.1191ms | 15.7157μs | 63.6307 KOps/s | 62.9801 KOps/s | $\color{#35bf28}+1.03\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 43.3220μs | 16.4816μs | 60.6738 KOps/s | 60.4885 KOps/s | $\color{#35bf28}+0.31\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 36.5290μs | 9.9660μs | 100.3408 KOps/s | 99.3138 KOps/s | $\color{#35bf28}+1.03\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 58.2690μs | 26.2044μs | 38.1615 KOps/s | 38.0754 KOps/s | $\color{#35bf28}+0.23\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 49.5520μs | 16.9906μs | 58.8560 KOps/s | 59.0707 KOps/s | $\color{#d91a1a}-0.36\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 0.1000ms | 17.6889μs | 56.5326 KOps/s | 57.3275 KOps/s | $\color{#d91a1a}-1.39\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 36.8490μs | 11.2928μs | 88.5519 KOps/s | 88.2304 KOps/s | $\color{#35bf28}+0.36\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 76.7840μs | 27.7274μs | 36.0654 KOps/s | 36.4745 KOps/s | $\color{#d91a1a}-1.12\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 44.9150μs | 18.1365μs | 55.1374 KOps/s | 54.8291 KOps/s | $\color{#35bf28}+0.56\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 39.7740μs | 17.5824μs | 56.8751 KOps/s | 56.9461 KOps/s | $\color{#d91a1a}-0.12\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 41.9890μs | 11.2796μs | 88.6557 KOps/s | 88.5859 KOps/s | $\color{#35bf28}+0.08\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 41.6480μs | 28.8845μs | 34.6207 KOps/s | 34.8011 KOps/s | $\color{#d91a1a}-0.52\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 52.6380μs | 19.2168μs | 52.0378 KOps/s | 51.6394 KOps/s | $\color{#35bf28}+0.77\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 46.9280μs | 18.7322μs | 53.3839 KOps/s | 53.9021 KOps/s | $\color{#d91a1a}-0.96\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 39.3730μs | 12.3792μs | 80.7808 KOps/s | 81.0709 KOps/s | $\color{#d91a1a}-0.36\\%$ | | test_values[generalized_advantage_estimate-True-True] | 10.1800ms | 9.7401ms | 102.6688 Ops/s | 103.7970 Ops/s | $\color{#d91a1a}-1.09\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 50.2438ms | 34.1169ms | 29.3110 Ops/s | 29.6759 Ops/s | $\color{#d91a1a}-1.23\\%$ | | test_values[td0_return_estimate-False-False] | 0.2170ms | 0.1685ms | 5.9358 KOps/s | 6.0809 KOps/s | $\color{#d91a1a}-2.38\\%$ | | test_values[td1_return_estimate-False-False] | 27.0944ms | 24.2784ms | 41.1889 Ops/s | 41.1359 Ops/s | $\color{#35bf28}+0.13\\%$ | | test_values[vec_td1_return_estimate-False-False] | 34.9006ms | 33.6009ms | 29.7611 Ops/s | 29.5867 Ops/s | $\color{#35bf28}+0.59\\%$ | | test_values[td_lambda_return_estimate-True-False] | 37.1238ms | 34.6298ms | 28.8769 Ops/s | 28.4332 Ops/s | $\color{#35bf28}+1.56\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 34.8759ms | 33.5799ms | 29.7797 Ops/s | 29.8195 Ops/s | $\color{#d91a1a}-0.13\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 10.2083ms | 8.3568ms | 119.6637 Ops/s | 119.1710 Ops/s | $\color{#35bf28}+0.41\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.5894ms | 1.9696ms | 507.7213 Ops/s | 526.8464 Ops/s | $\color{#d91a1a}-3.63\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.5600ms | 0.3546ms | 2.8205 KOps/s | 2.7427 KOps/s | $\color{#35bf28}+2.83\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 48.9600ms | 44.9680ms | 22.2381 Ops/s | 22.8974 Ops/s | $\color{#d91a1a}-2.88\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.5600ms | 3.0365ms | 329.3271 Ops/s | 328.5671 Ops/s | $\color{#35bf28}+0.23\\%$ | | test_dqn_speed | 6.2607ms | 1.3638ms | 733.2240 Ops/s | 742.4333 Ops/s | $\color{#d91a1a}-1.24\\%$ | | test_ddpg_speed | 3.5746ms | 2.8518ms | 350.6616 Ops/s | 348.8318 Ops/s | $\color{#35bf28}+0.52\\%$ | | test_sac_speed | 9.7107ms | 8.6192ms | 116.0197 Ops/s | 117.2246 Ops/s | $\color{#d91a1a}-1.03\\%$ | | test_redq_speed | 15.3207ms | 13.9335ms | 71.7694 Ops/s | 73.3128 Ops/s | $\color{#d91a1a}-2.11\\%$ | | test_redq_deprec_speed | 16.1467ms | 13.8218ms | 72.3496 Ops/s | 74.5700 Ops/s | $\color{#d91a1a}-2.98\\%$ | | test_td3_speed | 18.3356ms | 8.5537ms | 116.9087 Ops/s | 116.4764 Ops/s | $\color{#35bf28}+0.37\\%$ | | test_cql_speed | 39.0063ms | 37.1092ms | 26.9475 Ops/s | 27.3324 Ops/s | $\color{#d91a1a}-1.41\\%$ | | test_a2c_speed | 8.5238ms | 7.6313ms | 131.0388 Ops/s | 133.9724 Ops/s | $\color{#d91a1a}-2.19\\%$ | | test_ppo_speed | 8.8643ms | 7.9036ms | 126.5246 Ops/s | 129.3168 Ops/s | $\color{#d91a1a}-2.16\\%$ | | test_reinforce_speed | 7.9786ms | 6.7513ms | 148.1207 Ops/s | 150.6990 Ops/s | $\color{#d91a1a}-1.71\\%$ | | test_iql_speed | 34.2423ms | 33.0540ms | 30.2536 Ops/s | 30.1842 Ops/s | $\color{#35bf28}+0.23\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.8616ms | 3.6432ms | 274.4815 Ops/s | 283.4109 Ops/s | $\color{#d91a1a}-3.15\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7682ms | 0.5094ms | 1.9629 KOps/s | 1.9877 KOps/s | $\color{#d91a1a}-1.25\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.8141ms | 0.4924ms | 2.0310 KOps/s | 2.0735 KOps/s | $\color{#d91a1a}-2.05\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.9611ms | 3.6400ms | 274.7246 Ops/s | 282.9140 Ops/s | $\color{#d91a1a}-2.89\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 3.0733ms | 0.5075ms | 1.9705 KOps/s | 1.9869 KOps/s | $\color{#d91a1a}-0.82\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6410ms | 0.4830ms | 2.0703 KOps/s | 2.0896 KOps/s | $\color{#d91a1a}-0.92\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.6122ms | 1.7492ms | 571.6805 Ops/s | 573.8801 Ops/s | $\color{#d91a1a}-0.38\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.5606ms | 1.7259ms | 579.4200 Ops/s | 603.4862 Ops/s | $\color{#d91a1a}-3.99\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.4755ms | 3.7333ms | 267.8601 Ops/s | 274.3788 Ops/s | $\color{#d91a1a}-2.38\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.0666ms | 0.6466ms | 1.5464 KOps/s | 1.3470 KOps/s | $\textbf{\color{#35bf28}+14.80\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9656ms | 0.6227ms | 1.6058 KOps/s | 1.6198 KOps/s | $\color{#d91a1a}-0.86\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.2925ms | 3.5793ms | 279.3864 Ops/s | 280.4715 Ops/s | $\color{#d91a1a}-0.39\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6281ms | 0.5040ms | 1.9841 KOps/s | 1.9751 KOps/s | $\color{#35bf28}+0.45\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.6172ms | 0.4889ms | 2.0454 KOps/s | 2.0624 KOps/s | $\color{#d91a1a}-0.82\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.8322ms | 3.5328ms | 283.0613 Ops/s | 283.8365 Ops/s | $\color{#d91a1a}-0.27\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8048ms | 0.4990ms | 2.0038 KOps/s | 2.0022 KOps/s | $\color{#35bf28}+0.08\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6398ms | 0.4745ms | 2.1074 KOps/s | 2.0539 KOps/s | $\color{#35bf28}+2.61\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.3167ms | 3.7693ms | 265.3021 Ops/s | 273.2753 Ops/s | $\color{#d91a1a}-2.92\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.2172ms | 0.6529ms | 1.5316 KOps/s | 1.5503 KOps/s | $\color{#d91a1a}-1.21\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8569ms | 0.6269ms | 1.5951 KOps/s | 1.5990 KOps/s | $\color{#d91a1a}-0.24\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1269s | 6.1182ms | 163.4471 Ops/s | 161.9870 Ops/s | $\color{#35bf28}+0.90\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 15.0210ms | 12.5674ms | 79.5710 Ops/s | 78.5392 Ops/s | $\color{#35bf28}+1.31\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 4.3999ms | 1.1385ms | 878.3872 Ops/s | 989.9169 Ops/s | $\textbf{\color{#d91a1a}-11.27\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1176s | 8.1037ms | 123.3999 Ops/s | 124.7474 Ops/s | $\color{#d91a1a}-1.08\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 15.0871ms | 12.4582ms | 80.2685 Ops/s | 78.7036 Ops/s | $\color{#35bf28}+1.99\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.7367ms | 1.0568ms | 946.2618 Ops/s | 911.4921 Ops/s | $\color{#35bf28}+3.81\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1144s | 6.0450ms | 165.4246 Ops/s | 167.6305 Ops/s | $\color{#d91a1a}-1.32\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 15.3299ms | 12.5697ms | 79.5561 Ops/s | 78.0233 Ops/s | $\color{#35bf28}+1.96\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.7399ms | 1.1895ms | 840.6851 Ops/s | 827.4450 Ops/s | $\color{#35bf28}+1.60\\%$ |