search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.27k stars 302 forks source link

[Doc] Add doc about vmap randomness #2316

Closed vmoens closed 2 months ago

vmoens commented 2 months ago

Stack from ghstack (oldest at bottom):

pytorch-bot[bot] commented 2 months ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2316

Note: Links to docs will display an error until the docs builds have been completed.

:x: 3 New Failures, 1 Pending, 1 Unrelated Failure

As of commit 888ea3101440f9d745fc0fd165bcb2a0a63bae4b with merge base f840a1a4364bbb0bd33fbff7c4554e75af3ee1db (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2316#27845440698) ([gh](https://github.com/pytorch/rl/actions/runs/10072775635/job/27845440698)) `RuntimeError: Command docker exec -t 887ffb42a7240e45b2c12cea91b63a9a7faf3356f73c9448fe3fbfd6d653807e /exec failed with exit code 139` * [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2316#27845504947) ([gh](https://github.com/pytorch/rl/actions/runs/10072775624/job/27845504947)) `AttributeError: module 'torch' has no attribute 'compiler'` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2316#27845408202) ([gh](https://github.com/pytorch/rl/actions/runs/10072775626/job/27845408202)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2316#27845487079) ([gh](https://github.com/pytorch/rl/actions/runs/10072775625/job/27845487079)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/f840a1a4364bbb0bd33fbff7c4554e75af3ee1db#27805197076)) `AttributeError: module 'torch' has no attribute 'compiler'`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 2 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 59.6283ms | 58.9336ms | 16.9683 Ops/s | 16.6482 Ops/s | $\color{#35bf28}+1.92\\%$ | | test_sync | 38.8306ms | 32.3806ms | 30.8827 Ops/s | 29.9948 Ops/s | $\color{#35bf28}+2.96\\%$ | | test_async | 81.6578ms | 31.1007ms | 32.1536 Ops/s | 32.6727 Ops/s | $\color{#d91a1a}-1.59\\%$ | | test_simple | 0.5164s | 0.4313s | 2.3183 Ops/s | 2.3678 Ops/s | $\color{#d91a1a}-2.09\\%$ | | test_transformed | 0.6692s | 0.5930s | 1.6863 Ops/s | 1.6969 Ops/s | $\color{#d91a1a}-0.63\\%$ | | test_serial | 1.3638s | 1.2946s | 0.7724 Ops/s | 0.7763 Ops/s | $\color{#d91a1a}-0.50\\%$ | | test_parallel | 1.2081s | 1.1433s | 0.8747 Ops/s | 0.8636 Ops/s | $\color{#35bf28}+1.28\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 97.5720μs | 25.3601μs | 39.4320 KOps/s | 38.9179 KOps/s | $\color{#35bf28}+1.32\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 65.9230μs | 14.5167μs | 68.8864 KOps/s | 66.9677 KOps/s | $\color{#35bf28}+2.87\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 44.1030μs | 14.6369μs | 68.3204 KOps/s | 67.2821 KOps/s | $\color{#35bf28}+1.54\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 55.3160μs | 8.4706μs | 118.0547 KOps/s | 109.4084 KOps/s | $\textbf{\color{#35bf28}+7.90\\%}$ | | test_step_mdp_speed[True-True-False-True-True] | 0.1034ms | 26.9517μs | 37.1035 KOps/s | 35.3602 KOps/s | $\color{#35bf28}+4.93\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 70.6320μs | 16.0863μs | 62.1646 KOps/s | 60.2638 KOps/s | $\color{#35bf28}+3.15\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 54.7920μs | 16.3153μs | 61.2922 KOps/s | 60.0861 KOps/s | $\color{#35bf28}+2.01\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 61.4250μs | 10.0067μs | 99.9334 KOps/s | 97.0488 KOps/s | $\color{#35bf28}+2.97\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 62.7080μs | 28.5668μs | 35.0057 KOps/s | 34.4737 KOps/s | $\color{#35bf28}+1.54\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 0.1398ms | 18.1813μs | 55.0017 KOps/s | 55.1456 KOps/s | $\color{#d91a1a}-0.26\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 0.1940ms | 16.9136μs | 59.1241 KOps/s | 61.0123 KOps/s | $\color{#d91a1a}-3.09\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 35.6670μs | 9.9695μs | 100.3057 KOps/s | 96.6274 KOps/s | $\color{#35bf28}+3.81\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 82.8550μs | 30.2725μs | 33.0333 KOps/s | 31.2538 KOps/s | $\textbf{\color{#35bf28}+5.69\\%}$ | | test_step_mdp_speed[True-False-False-True-False] | 59.8110μs | 19.2231μs | 52.0207 KOps/s | 50.8710 KOps/s | $\color{#35bf28}+2.26\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 85.5300μs | 17.7037μs | 56.4853 KOps/s | 55.7931 KOps/s | $\color{#35bf28}+1.24\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 52.0270μs | 11.4584μs | 87.2720 KOps/s | 85.3369 KOps/s | $\color{#35bf28}+2.27\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 91.0700μs | 28.6028μs | 34.9617 KOps/s | 34.7085 KOps/s | $\color{#35bf28}+0.73\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 52.8880μs | 17.7695μs | 56.2761 KOps/s | 54.8990 KOps/s | $\color{#35bf28}+2.51\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 45.2350μs | 18.5694μs | 53.8521 KOps/s | 52.5250 KOps/s | $\color{#35bf28}+2.53\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 58.0490μs | 11.2683μs | 88.7448 KOps/s | 86.7781 KOps/s | $\color{#35bf28}+2.27\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 71.3130μs | 29.9563μs | 33.3820 KOps/s | 32.7155 KOps/s | $\color{#35bf28}+2.04\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 61.5350μs | 19.3873μs | 51.5800 KOps/s | 50.4102 KOps/s | $\color{#35bf28}+2.32\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 54.9230μs | 20.3453μs | 49.1513 KOps/s | 48.5848 KOps/s | $\color{#35bf28}+1.17\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 65.7330μs | 12.7732μs | 78.2890 KOps/s | 76.8759 KOps/s | $\color{#35bf28}+1.84\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.6375ms | 32.2120μs | 31.0443 KOps/s | 30.7380 KOps/s | $\color{#35bf28}+1.00\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 88.7460μs | 20.9537μs | 47.7242 KOps/s | 47.2376 KOps/s | $\color{#35bf28}+1.03\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 92.6230μs | 20.5154μs | 48.7438 KOps/s | 47.6480 KOps/s | $\color{#35bf28}+2.30\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 52.9490μs | 12.6103μs | 79.3002 KOps/s | 77.0806 KOps/s | $\color{#35bf28}+2.88\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 92.6040μs | 33.0963μs | 30.2149 KOps/s | 29.6341 KOps/s | $\color{#35bf28}+1.96\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 64.4300μs | 22.3568μs | 44.7292 KOps/s | 44.0628 KOps/s | $\color{#35bf28}+1.51\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 85.3300μs | 21.4904μs | 46.5323 KOps/s | 46.0811 KOps/s | $\color{#35bf28}+0.98\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 67.3260μs | 14.0835μs | 71.0050 KOps/s | 70.3911 KOps/s | $\color{#35bf28}+0.87\\%$ | | test_values[generalized_advantage_estimate-True-True] | 13.3231ms | 9.7256ms | 102.8216 Ops/s | 98.9662 Ops/s | $\color{#35bf28}+3.90\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 48.7702ms | 38.5147ms | 25.9641 Ops/s | 27.9344 Ops/s | $\textbf{\color{#d91a1a}-7.05\\%}$ | | test_values[td0_return_estimate-False-False] | 0.2178ms | 0.1784ms | 5.6069 KOps/s | 5.7072 KOps/s | $\color{#d91a1a}-1.76\\%$ | | test_values[td1_return_estimate-False-False] | 27.3527ms | 23.9445ms | 41.7633 Ops/s | 40.4003 Ops/s | $\color{#35bf28}+3.37\\%$ | | test_values[vec_td1_return_estimate-False-False] | 35.9612ms | 33.5219ms | 29.8312 Ops/s | 27.6368 Ops/s | $\textbf{\color{#35bf28}+7.94\\%}$ | | test_values[td_lambda_return_estimate-True-False] | 38.1296ms | 34.7654ms | 28.7642 Ops/s | 28.2872 Ops/s | $\color{#35bf28}+1.69\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 35.5323ms | 33.5350ms | 29.8196 Ops/s | 27.5650 Ops/s | $\textbf{\color{#35bf28}+8.18\\%}$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.4719ms | 8.3188ms | 120.2095 Ops/s | 114.4425 Ops/s | $\textbf{\color{#35bf28}+5.04\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.2834ms | 1.8716ms | 534.3021 Ops/s | 544.8966 Ops/s | $\color{#d91a1a}-1.94\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4597ms | 0.3551ms | 2.8161 KOps/s | 2.8258 KOps/s | $\color{#d91a1a}-0.34\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 47.5121ms | 46.1522ms | 21.6675 Ops/s | 22.3573 Ops/s | $\color{#d91a1a}-3.09\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.9287ms | 3.0541ms | 327.4295 Ops/s | 322.5806 Ops/s | $\color{#35bf28}+1.50\\%$ | | test_dqn_speed | 2.0683ms | 1.3727ms | 728.4756 Ops/s | 717.4092 Ops/s | $\color{#35bf28}+1.54\\%$ | | test_ddpg_speed | 4.4084ms | 2.9247ms | 341.9174 Ops/s | 339.4576 Ops/s | $\color{#35bf28}+0.72\\%$ | | test_sac_speed | 15.1573ms | 8.6836ms | 115.1593 Ops/s | 114.9448 Ops/s | $\color{#35bf28}+0.19\\%$ | | test_redq_speed | 23.1059ms | 14.1044ms | 70.8999 Ops/s | 71.3634 Ops/s | $\color{#d91a1a}-0.65\\%$ | | test_redq_deprec_speed | 15.7875ms | 13.6309ms | 73.3625 Ops/s | 71.7948 Ops/s | $\color{#35bf28}+2.18\\%$ | | test_td3_speed | 10.2386ms | 8.6249ms | 115.9434 Ops/s | 115.3251 Ops/s | $\color{#35bf28}+0.54\\%$ | | test_cql_speed | 37.4767ms | 36.4118ms | 27.4637 Ops/s | 26.7993 Ops/s | $\color{#35bf28}+2.48\\%$ | | test_a2c_speed | 8.5273ms | 7.6659ms | 130.4479 Ops/s | 128.6186 Ops/s | $\color{#35bf28}+1.42\\%$ | | test_ppo_speed | 9.5137ms | 8.0536ms | 124.1684 Ops/s | 124.3998 Ops/s | $\color{#d91a1a}-0.19\\%$ | | test_reinforce_speed | 7.8771ms | 6.7031ms | 149.1838 Ops/s | 148.4937 Ops/s | $\color{#35bf28}+0.46\\%$ | | test_iql_speed | 34.6527ms | 32.9418ms | 30.3566 Ops/s | 29.0292 Ops/s | $\color{#35bf28}+4.57\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 7.7128ms | 5.1945ms | 192.5111 Ops/s | 193.8279 Ops/s | $\color{#d91a1a}-0.68\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.9130ms | 0.4967ms | 2.0135 KOps/s | 2.0022 KOps/s | $\color{#35bf28}+0.56\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7872ms | 0.4938ms | 2.0251 KOps/s | 2.1302 KOps/s | $\color{#d91a1a}-4.93\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 8.1811ms | 5.1707ms | 193.3961 Ops/s | 195.1497 Ops/s | $\color{#d91a1a}-0.90\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7040ms | 0.4859ms | 2.0579 KOps/s | 2.0301 KOps/s | $\color{#35bf28}+1.37\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7139ms | 0.4686ms | 2.1339 KOps/s | 2.1487 KOps/s | $\color{#d91a1a}-0.69\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 5.4717ms | 1.9133ms | 522.6505 Ops/s | 570.6909 Ops/s | $\textbf{\color{#d91a1a}-8.42\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 4.2826ms | 1.6888ms | 592.1274 Ops/s | 616.9482 Ops/s | $\color{#d91a1a}-4.02\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.0999ms | 5.2666ms | 189.8766 Ops/s | 192.6106 Ops/s | $\color{#d91a1a}-1.42\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.6741ms | 0.6352ms | 1.5743 KOps/s | 1.5338 KOps/s | $\color{#35bf28}+2.64\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8518ms | 0.5958ms | 1.6783 KOps/s | 1.6623 KOps/s | $\color{#35bf28}+0.96\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.8223ms | 5.1306ms | 194.9085 Ops/s | 194.9750 Ops/s | $\color{#d91a1a}-0.03\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.8997ms | 0.4887ms | 2.0461 KOps/s | 1.9981 KOps/s | $\color{#35bf28}+2.40\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6516ms | 0.4707ms | 2.1243 KOps/s | 2.0647 KOps/s | $\color{#35bf28}+2.89\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 7.4382ms | 5.0470ms | 198.1356 Ops/s | 196.9403 Ops/s | $\color{#35bf28}+0.61\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 2.0821ms | 0.4875ms | 2.0514 KOps/s | 1.5577 KOps/s | $\textbf{\color{#35bf28}+31.70\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6722ms | 0.4679ms | 2.1370 KOps/s | 2.1586 KOps/s | $\color{#d91a1a}-1.00\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 7.4759ms | 5.2397ms | 190.8504 Ops/s | 187.9929 Ops/s | $\color{#35bf28}+1.52\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.3797ms | 0.6293ms | 1.5892 KOps/s | 1.5827 KOps/s | $\color{#35bf28}+0.41\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 7.8668ms | 0.6168ms | 1.6212 KOps/s | 1.6452 KOps/s | $\color{#d91a1a}-1.46\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1296s | 8.5504ms | 116.9538 Ops/s | 153.1207 Ops/s | $\textbf{\color{#d91a1a}-23.62\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 17.1716ms | 12.9477ms | 77.2340 Ops/s | 76.6910 Ops/s | $\color{#35bf28}+0.71\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 5.6927ms | 1.1878ms | 841.8605 Ops/s | 816.8838 Ops/s | $\color{#35bf28}+3.06\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1139s | 6.0156ms | 166.2355 Ops/s | 117.8004 Ops/s | $\textbf{\color{#35bf28}+41.12\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 17.0262ms | 12.8229ms | 77.9855 Ops/s | 76.2783 Ops/s | $\color{#35bf28}+2.24\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.6317ms | 1.0603ms | 943.0942 Ops/s | 893.8233 Ops/s | $\textbf{\color{#35bf28}+5.51\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1211s | 8.5618ms | 116.7981 Ops/s | 158.6253 Ops/s | $\textbf{\color{#d91a1a}-26.37\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.2187ms | 13.2048ms | 75.7298 Ops/s | 75.9289 Ops/s | $\color{#d91a1a}-0.26\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 6.3221ms | 1.3740ms | 727.7874 Ops/s | 775.4786 Ops/s | $\textbf{\color{#d91a1a}-6.15\\%}$ |
github-actions[bot] commented 2 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1063s | 0.1057s | 9.4626 Ops/s | 8.5600 Ops/s | $\textbf{\color{#35bf28}+10.55\\%}$ | | test_sync | 95.7512ms | 95.2019ms | 10.5040 Ops/s | 10.5135 Ops/s | $\color{#d91a1a}-0.09\\%$ | | test_async | 0.1604s | 87.5344ms | 11.4241 Ops/s | 11.3252 Ops/s | $\color{#35bf28}+0.87\\%$ | | test_single_pixels | 0.1161s | 0.1159s | 8.6317 Ops/s | 8.5289 Ops/s | $\color{#35bf28}+1.21\\%$ | | test_sync_pixels | 76.2294ms | 74.2134ms | 13.4747 Ops/s | 13.3883 Ops/s | $\color{#35bf28}+0.65\\%$ | | test_async_pixels | 0.1439s | 69.7354ms | 14.3399 Ops/s | 14.3342 Ops/s | $\color{#35bf28}+0.04\\%$ | | test_simple | 0.8596s | 0.7839s | 1.2757 Ops/s | 1.2373 Ops/s | $\color{#35bf28}+3.11\\%$ | | test_transformed | 1.0774s | 1.0073s | 0.9927 Ops/s | 0.9874 Ops/s | $\color{#35bf28}+0.54\\%$ | | test_serial | 2.2761s | 2.2052s | 0.4535 Ops/s | 0.4464 Ops/s | $\color{#35bf28}+1.58\\%$ | | test_parallel | 2.0339s | 1.9749s | 0.5064 Ops/s | 0.5070 Ops/s | $\color{#d91a1a}-0.14\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1269ms | 37.1400μs | 26.9251 KOps/s | 26.3870 KOps/s | $\color{#35bf28}+2.04\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 50.2810μs | 21.4017μs | 46.7252 KOps/s | 46.5344 KOps/s | $\color{#35bf28}+0.41\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 47.3510μs | 20.9212μs | 47.7985 KOps/s | 46.9044 KOps/s | $\color{#35bf28}+1.91\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 28.9710μs | 12.3189μs | 81.1760 KOps/s | 81.6321 KOps/s | $\color{#d91a1a}-0.56\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 63.3320μs | 39.0917μs | 25.5809 KOps/s | 24.9221 KOps/s | $\color{#35bf28}+2.64\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 42.6400μs | 23.3197μs | 42.8823 KOps/s | 41.8947 KOps/s | $\color{#35bf28}+2.36\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 57.3010μs | 23.2891μs | 42.9386 KOps/s | 42.3310 KOps/s | $\color{#35bf28}+1.44\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 38.6510μs | 14.4553μs | 69.1789 KOps/s | 69.4844 KOps/s | $\color{#d91a1a}-0.44\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 64.8810μs | 42.0682μs | 23.7709 KOps/s | 23.8694 KOps/s | $\color{#d91a1a}-0.41\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 43.6210μs | 25.7924μs | 38.7711 KOps/s | 38.4107 KOps/s | $\color{#35bf28}+0.94\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 44.9410μs | 23.2511μs | 43.0087 KOps/s | 42.4704 KOps/s | $\color{#35bf28}+1.27\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 31.6900μs | 14.3428μs | 69.7213 KOps/s | 68.7815 KOps/s | $\color{#35bf28}+1.37\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 72.2410μs | 44.1924μs | 22.6283 KOps/s | 22.5094 KOps/s | $\color{#35bf28}+0.53\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 91.5310μs | 28.5199μs | 35.0633 KOps/s | 34.9955 KOps/s | $\color{#35bf28}+0.19\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 56.4210μs | 25.6937μs | 38.9201 KOps/s | 38.1098 KOps/s | $\color{#35bf28}+2.13\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 35.4200μs | 16.5519μs | 60.4161 KOps/s | 59.6168 KOps/s | $\color{#35bf28}+1.34\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 63.9010μs | 41.8822μs | 23.8765 KOps/s | 23.4372 KOps/s | $\color{#35bf28}+1.87\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 50.5710μs | 25.6950μs | 38.9180 KOps/s | 38.1442 KOps/s | $\color{#35bf28}+2.03\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 95.6410μs | 27.7167μs | 36.0793 KOps/s | 35.1379 KOps/s | $\color{#35bf28}+2.68\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 35.1010μs | 16.6864μs | 59.9290 KOps/s | 60.2725 KOps/s | $\color{#d91a1a}-0.57\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 76.1720μs | 44.0484μs | 22.7023 KOps/s | 22.4432 KOps/s | $\color{#35bf28}+1.15\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 49.5010μs | 28.2655μs | 35.3788 KOps/s | 35.1812 KOps/s | $\color{#35bf28}+0.56\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 48.8310μs | 29.5184μs | 33.8772 KOps/s | 32.5782 KOps/s | $\color{#35bf28}+3.99\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 43.5610μs | 18.6514μs | 53.6154 KOps/s | 53.0206 KOps/s | $\color{#35bf28}+1.12\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.8462ms | 46.3103μs | 21.5935 KOps/s | 20.9925 KOps/s | $\color{#35bf28}+2.86\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 49.3610μs | 30.4958μs | 32.7914 KOps/s | 32.0698 KOps/s | $\color{#35bf28}+2.25\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 55.1610μs | 29.9626μs | 33.3750 KOps/s | 32.6060 KOps/s | $\color{#35bf28}+2.36\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 43.7210μs | 18.7826μs | 53.2408 KOps/s | 53.0084 KOps/s | $\color{#35bf28}+0.44\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 73.3810μs | 48.1156μs | 20.7833 KOps/s | 20.6419 KOps/s | $\color{#35bf28}+0.69\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 98.0620μs | 32.6817μs | 30.5982 KOps/s | 30.2196 KOps/s | $\color{#35bf28}+1.25\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 54.1410μs | 31.6095μs | 31.6361 KOps/s | 31.0797 KOps/s | $\color{#35bf28}+1.79\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 0.2216ms | 20.7160μs | 48.2720 KOps/s | 47.2220 KOps/s | $\color{#35bf28}+2.22\\%$ | | test_values[generalized_advantage_estimate-True-True] | 25.2262ms | 24.8221ms | 40.2867 Ops/s | 40.2051 Ops/s | $\color{#35bf28}+0.20\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 96.9574ms | 2.8374ms | 352.4314 Ops/s | 366.5858 Ops/s | $\color{#d91a1a}-3.86\\%$ | | test_values[td0_return_estimate-False-False] | 93.2410μs | 67.0343μs | 14.9177 KOps/s | 14.8146 KOps/s | $\color{#35bf28}+0.70\\%$ | | test_values[td1_return_estimate-False-False] | 56.1779ms | 55.7953ms | 17.9226 Ops/s | 17.8355 Ops/s | $\color{#35bf28}+0.49\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.3076ms | 1.0847ms | 921.8938 Ops/s | 917.0370 Ops/s | $\color{#35bf28}+0.53\\%$ | | test_values[td_lambda_return_estimate-True-False] | 92.2128ms | 88.5669ms | 11.2909 Ops/s | 11.2987 Ops/s | $\color{#d91a1a}-0.07\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.3885ms | 1.0829ms | 923.4636 Ops/s | 918.0384 Ops/s | $\color{#35bf28}+0.59\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 25.0345ms | 24.7460ms | 40.4106 Ops/s | 40.3655 Ops/s | $\color{#35bf28}+0.11\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9661ms | 0.7198ms | 1.3893 KOps/s | 1.3778 KOps/s | $\color{#35bf28}+0.84\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7626ms | 0.6693ms | 1.4941 KOps/s | 1.4782 KOps/s | $\color{#35bf28}+1.08\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5063ms | 1.4626ms | 683.7317 Ops/s | 680.0435 Ops/s | $\color{#35bf28}+0.54\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7131ms | 0.6831ms | 1.4640 KOps/s | 1.4461 KOps/s | $\color{#35bf28}+1.24\\%$ | | test_dqn_speed | 7.1278ms | 1.4227ms | 702.9126 Ops/s | 688.2553 Ops/s | $\color{#35bf28}+2.13\\%$ | | test_ddpg_speed | 3.1736ms | 2.9276ms | 341.5766 Ops/s | 334.3482 Ops/s | $\color{#35bf28}+2.16\\%$ | | test_sac_speed | 0.1015s | 9.0117ms | 110.9667 Ops/s | 119.0747 Ops/s | $\textbf{\color{#d91a1a}-6.81\\%}$ | | test_redq_speed | 11.5212ms | 10.7240ms | 93.2485 Ops/s | 91.6791 Ops/s | $\color{#35bf28}+1.71\\%$ | | test_redq_deprec_speed | 11.8226ms | 11.0991ms | 90.0972 Ops/s | 87.4348 Ops/s | $\color{#35bf28}+3.05\\%$ | | test_td3_speed | 8.3847ms | 8.2146ms | 121.7341 Ops/s | 119.7836 Ops/s | $\color{#35bf28}+1.63\\%$ | | test_cql_speed | 26.7429ms | 25.5047ms | 39.2085 Ops/s | 35.4535 Ops/s | $\textbf{\color{#35bf28}+10.59\\%}$ | | test_a2c_speed | 5.8453ms | 5.5735ms | 179.4205 Ops/s | 177.6203 Ops/s | $\color{#35bf28}+1.01\\%$ | | test_ppo_speed | 6.2287ms | 5.8910ms | 169.7494 Ops/s | 168.3154 Ops/s | $\color{#35bf28}+0.85\\%$ | | test_reinforce_speed | 5.2571ms | 4.5494ms | 219.8114 Ops/s | 219.0489 Ops/s | $\color{#35bf28}+0.35\\%$ | | test_iql_speed | 19.9624ms | 19.5549ms | 51.1381 Ops/s | 51.1254 Ops/s | $\color{#35bf28}+0.02\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.7797ms | 6.5621ms | 152.3901 Ops/s | 149.0588 Ops/s | $\color{#35bf28}+2.23\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.1109s | 0.5849ms | 1.7096 KOps/s | 1.9593 KOps/s | $\textbf{\color{#d91a1a}-12.75\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6980ms | 0.4887ms | 2.0462 KOps/s | 2.0461 KOps/s | $+0.00\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.7587ms | 6.4592ms | 154.8171 Ops/s | 151.8909 Ops/s | $\color{#35bf28}+1.93\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.3705ms | 0.5059ms | 1.9767 KOps/s | 1.9827 KOps/s | $\color{#d91a1a}-0.30\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6615ms | 0.4839ms | 2.0667 KOps/s | 2.0600 KOps/s | $\color{#35bf28}+0.32\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.9164ms | 2.0040ms | 498.9906 Ops/s | 514.0703 Ops/s | $\color{#d91a1a}-2.93\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2259ms | 1.8521ms | 539.9246 Ops/s | 539.3426 Ops/s | $\color{#35bf28}+0.11\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.8453ms | 6.6760ms | 149.7899 Ops/s | 147.1905 Ops/s | $\color{#35bf28}+1.77\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9876ms | 0.6522ms | 1.5333 KOps/s | 1.5116 KOps/s | $\color{#35bf28}+1.44\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8111ms | 0.6326ms | 1.5808 KOps/s | 1.5154 KOps/s | $\color{#35bf28}+4.31\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.6881ms | 6.5277ms | 153.1939 Ops/s | 150.0509 Ops/s | $\color{#35bf28}+2.09\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.5129ms | 0.5070ms | 1.9723 KOps/s | 1.9498 KOps/s | $\color{#35bf28}+1.15\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7138ms | 0.4888ms | 2.0460 KOps/s | 1.9980 KOps/s | $\color{#35bf28}+2.40\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.6921ms | 6.4602ms | 154.7938 Ops/s | 151.3409 Ops/s | $\color{#35bf28}+2.28\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6054ms | 0.5036ms | 1.9857 KOps/s | 1.9776 KOps/s | $\color{#35bf28}+0.41\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.1410s | 0.6809ms | 1.4686 KOps/s | 2.0675 KOps/s | $\textbf{\color{#d91a1a}-28.96\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.8031ms | 6.6828ms | 149.6377 Ops/s | 147.2264 Ops/s | $\color{#35bf28}+1.64\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.1250ms | 0.6550ms | 1.5267 KOps/s | 1.4962 KOps/s | $\color{#35bf28}+2.04\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8331ms | 0.6400ms | 1.5624 KOps/s | 1.5621 KOps/s | $\color{#35bf28}+0.02\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1293s | 7.6252ms | 131.1437 Ops/s | 126.6216 Ops/s | $\color{#35bf28}+3.57\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 17.8293ms | 15.5894ms | 64.1460 Ops/s | 62.6377 Ops/s | $\color{#35bf28}+2.41\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.1777ms | 1.2044ms | 830.2574 Ops/s | 850.8854 Ops/s | $\color{#d91a1a}-2.42\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1266s | 9.9618ms | 100.3835 Ops/s | 132.7375 Ops/s | $\textbf{\color{#d91a1a}-24.37\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 17.7761ms | 15.6399ms | 63.9388 Ops/s | 62.8296 Ops/s | $\color{#35bf28}+1.77\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.1260ms | 1.1864ms | 842.8629 Ops/s | 866.2210 Ops/s | $\color{#d91a1a}-2.70\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1248s | 7.7104ms | 129.6948 Ops/s | 129.6152 Ops/s | $\color{#35bf28}+0.06\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 17.9699ms | 15.7923ms | 63.3220 Ops/s | 62.0935 Ops/s | $\color{#35bf28}+1.98\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.2901ms | 1.3494ms | 741.0468 Ops/s | 698.3636 Ops/s | $\textbf{\color{#35bf28}+6.11\\%}$ |