search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.01k stars 269 forks source link

[WIP] Correct typos #2263

Open vmoens opened 1 week ago

pytorch-bot[bot] commented 1 week ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2263

Note: Links to docs will display an error until the docs builds have been completed.

:x: 3 New Failures, 1 Unrelated Failure

As of commit 5e17fc95287c5845b8b52a61805d69d27bc2f729 with merge base 79fa8bfc908b9aff6d858b4c801e633841bc5f68 (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2263#26931706814) ([gh](https://github.com/pytorch/rl/actions/runs/9758022579/job/26931706814)) `RuntimeError: Command docker exec -t 795ec692d91a4640fdb3035c7c2298d031fca2a3cd388e73a264f8bcdcec221e /exec failed with exit code 139` * [RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2263#26931706813) ([gh](https://github.com/pytorch/rl/actions/runs/9758022580/job/26931706813)) `RuntimeError: Command docker exec -t c0106a8037ec7a4838ae62b560480b5d9fc5a968097ba06cb62c1894f2c53936 /exec failed with exit code 1` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2263#26931713175) ([gh](https://github.com/pytorch/rl/actions/runs/9758022578/job/26931713175)) `RuntimeError: Command docker exec -t 41d868af893f3be17fca65daae2c67f2da07056ea87976ed74b35325f15c0999 /exec failed with exit code 1`

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2263#26931705643) ([gh](https://github.com/pytorch/rl/actions/runs/9758022593/job/26931705643)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/79fa8bfc908b9aff6d858b4c801e633841bc5f68#26931314662)) `test/test_transforms.py::TestActionDiscretizer::test_trans_parallel_env_check[False]`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 week ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}10$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1281s | 62.3493ms | 16.0387 Ops/s | 17.6315 Ops/s | $\textbf{\color{#d91a1a}-9.03\\%}$ | | test_sync | 33.6298ms | 32.2535ms | 31.0043 Ops/s | 31.3190 Ops/s | $\color{#d91a1a}-1.00\\%$ | | test_async | 57.9784ms | 29.4984ms | 33.9002 Ops/s | 33.6003 Ops/s | $\color{#35bf28}+0.89\\%$ | | test_simple | 0.3954s | 0.3919s | 2.5518 Ops/s | 2.5621 Ops/s | $\color{#d91a1a}-0.40\\%$ | | test_transformed | 0.5635s | 0.5571s | 1.7951 Ops/s | 1.7914 Ops/s | $\color{#35bf28}+0.20\\%$ | | test_serial | 1.3866s | 1.3306s | 0.7515 Ops/s | 0.7713 Ops/s | $\color{#d91a1a}-2.57\\%$ | | test_parallel | 1.1572s | 1.1039s | 0.9059 Ops/s | 0.8953 Ops/s | $\color{#35bf28}+1.18\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1686ms | 23.1028μs | 43.2848 KOps/s | 44.5309 KOps/s | $\color{#d91a1a}-2.80\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 54.0310μs | 13.7111μs | 72.9333 KOps/s | 74.2461 KOps/s | $\color{#d91a1a}-1.77\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 53.9910μs | 13.5538μs | 73.7802 KOps/s | 75.8559 KOps/s | $\color{#d91a1a}-2.74\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 26.9010μs | 7.9290μs | 126.1192 KOps/s | 127.6900 KOps/s | $\color{#d91a1a}-1.23\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 74.9600μs | 24.5681μs | 40.7032 KOps/s | 41.5633 KOps/s | $\color{#d91a1a}-2.07\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 32.4310μs | 14.9183μs | 67.0318 KOps/s | 68.1449 KOps/s | $\color{#d91a1a}-1.63\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 46.9480μs | 14.8279μs | 67.4404 KOps/s | 69.0455 KOps/s | $\color{#d91a1a}-2.32\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 47.9590μs | 9.2493μs | 108.1160 KOps/s | 110.4064 KOps/s | $\color{#d91a1a}-2.07\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 76.9840μs | 26.1812μs | 38.1953 KOps/s | 38.9778 KOps/s | $\color{#d91a1a}-2.01\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 60.1320μs | 16.4234μs | 60.8886 KOps/s | 62.0085 KOps/s | $\color{#d91a1a}-1.81\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 49.5630μs | 14.8014μs | 67.5614 KOps/s | 69.4860 KOps/s | $\color{#d91a1a}-2.77\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 31.1480μs | 9.2668μs | 107.9124 KOps/s | 110.0266 KOps/s | $\color{#d91a1a}-1.92\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 70.7830μs | 27.0527μs | 36.9649 KOps/s | 37.6899 KOps/s | $\color{#d91a1a}-1.92\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 44.0820μs | 17.4789μs | 57.2119 KOps/s | 57.8832 KOps/s | $\color{#d91a1a}-1.16\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 69.0590μs | 16.0164μs | 62.4359 KOps/s | 64.1725 KOps/s | $\color{#d91a1a}-2.71\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 34.6340μs | 10.5556μs | 94.7365 KOps/s | 97.3687 KOps/s | $\color{#d91a1a}-2.70\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 78.0570μs | 26.0132μs | 38.4420 KOps/s | 39.3775 KOps/s | $\color{#d91a1a}-2.38\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 63.4190μs | 16.3153μs | 61.2921 KOps/s | 62.3747 KOps/s | $\color{#d91a1a}-1.74\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 38.9230μs | 17.3516μs | 57.6316 KOps/s | 59.6028 KOps/s | $\color{#d91a1a}-3.31\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 35.4570μs | 10.5323μs | 94.9460 KOps/s | 97.3112 KOps/s | $\color{#d91a1a}-2.43\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 52.6080μs | 27.1650μs | 36.8120 KOps/s | 37.2446 KOps/s | $\color{#d91a1a}-1.16\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 68.1180μs | 17.2697μs | 57.9048 KOps/s | 57.4730 KOps/s | $\color{#35bf28}+0.75\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 56.4860μs | 18.3856μs | 54.3905 KOps/s | 55.8674 KOps/s | $\color{#d91a1a}-2.64\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 52.6180μs | 11.6841μs | 85.5860 KOps/s | 87.2598 KOps/s | $\color{#d91a1a}-1.92\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 63.8090μs | 28.8173μs | 34.7014 KOps/s | 35.6688 KOps/s | $\color{#d91a1a}-2.71\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 63.3390μs | 18.7780μs | 53.2538 KOps/s | 53.5249 KOps/s | $\color{#d91a1a}-0.51\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 46.6280μs | 18.5044μs | 54.0412 KOps/s | 56.1455 KOps/s | $\color{#d91a1a}-3.75\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 52.8490μs | 11.7212μs | 85.3152 KOps/s | 86.4317 KOps/s | $\color{#d91a1a}-1.29\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 40.7660μs | 30.0863μs | 33.2378 KOps/s | 34.0479 KOps/s | $\color{#d91a1a}-2.38\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 50.5950μs | 19.9908μs | 50.0231 KOps/s | 50.4021 KOps/s | $\color{#d91a1a}-0.75\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 71.6340μs | 19.3614μs | 51.6491 KOps/s | 52.9851 KOps/s | $\color{#d91a1a}-2.52\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 94.5790μs | 12.9443μs | 77.2539 KOps/s | 79.4202 KOps/s | $\color{#d91a1a}-2.73\\%$ | | test_values[generalized_advantage_estimate-True-True] | 9.9968ms | 9.6134ms | 104.0213 Ops/s | 104.5786 Ops/s | $\color{#d91a1a}-0.53\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 37.3285ms | 33.5096ms | 29.8422 Ops/s | 28.3089 Ops/s | $\textbf{\color{#35bf28}+5.42\\%}$ | | test_values[td0_return_estimate-False-False] | 0.2239ms | 0.1707ms | 5.8572 KOps/s | 5.6856 KOps/s | $\color{#35bf28}+3.02\\%$ | | test_values[td1_return_estimate-False-False] | 27.0487ms | 24.2079ms | 41.3088 Ops/s | 41.0330 Ops/s | $\color{#35bf28}+0.67\\%$ | | test_values[vec_td1_return_estimate-False-False] | 35.6145ms | 33.6767ms | 29.6941 Ops/s | 27.9903 Ops/s | $\textbf{\color{#35bf28}+6.09\\%}$ | | test_values[td_lambda_return_estimate-True-False] | 37.6641ms | 34.7755ms | 28.7559 Ops/s | 29.1423 Ops/s | $\color{#d91a1a}-1.33\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 36.3649ms | 33.6482ms | 29.7193 Ops/s | 27.9577 Ops/s | $\textbf{\color{#35bf28}+6.30\\%}$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 11.8888ms | 8.5230ms | 117.3298 Ops/s | 119.9126 Ops/s | $\color{#d91a1a}-2.15\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.1230ms | 1.8521ms | 539.9294 Ops/s | 547.7161 Ops/s | $\color{#d91a1a}-1.42\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.8614ms | 0.3696ms | 2.7060 KOps/s | 2.7846 KOps/s | $\color{#d91a1a}-2.83\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 45.0717ms | 40.0122ms | 24.9924 Ops/s | 21.6405 Ops/s | $\textbf{\color{#35bf28}+15.49\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.7092ms | 3.0559ms | 327.2360 Ops/s | 326.3982 Ops/s | $\color{#35bf28}+0.26\\%$ | | test_dqn_speed | 1.8314ms | 1.3941ms | 717.3055 Ops/s | 718.0168 Ops/s | $\color{#d91a1a}-0.10\\%$ | | test_ddpg_speed | 3.7845ms | 2.9416ms | 339.9484 Ops/s | 342.0011 Ops/s | $\color{#d91a1a}-0.60\\%$ | | test_sac_speed | 9.5329ms | 8.8882ms | 112.5090 Ops/s | 112.2737 Ops/s | $\color{#35bf28}+0.21\\%$ | | test_redq_speed | 16.1568ms | 14.0166ms | 71.3442 Ops/s | 71.4226 Ops/s | $\color{#d91a1a}-0.11\\%$ | | test_redq_deprec_speed | 16.3639ms | 14.2857ms | 70.0002 Ops/s | 63.1632 Ops/s | $\textbf{\color{#35bf28}+10.82\\%}$ | | test_td3_speed | 18.2023ms | 9.0077ms | 111.0161 Ops/s | 113.0562 Ops/s | $\color{#d91a1a}-1.80\\%$ | | test_cql_speed | 38.8688ms | 37.5891ms | 26.6034 Ops/s | 26.7890 Ops/s | $\color{#d91a1a}-0.69\\%$ | | test_a2c_speed | 10.6682ms | 7.9751ms | 125.3904 Ops/s | 126.0740 Ops/s | $\color{#d91a1a}-0.54\\%$ | | test_ppo_speed | 8.9947ms | 8.0691ms | 123.9289 Ops/s | 121.9167 Ops/s | $\color{#35bf28}+1.65\\%$ | | test_reinforce_speed | 9.2191ms | 6.9921ms | 143.0179 Ops/s | 141.7098 Ops/s | $\color{#35bf28}+0.92\\%$ | | test_iql_speed | 34.8857ms | 33.5201ms | 29.8328 Ops/s | 29.6514 Ops/s | $\color{#35bf28}+0.61\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.5469ms | 3.7180ms | 268.9654 Ops/s | 259.7886 Ops/s | $\color{#35bf28}+3.53\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.8369ms | 0.5104ms | 1.9592 KOps/s | 1.6991 KOps/s | $\textbf{\color{#35bf28}+15.31\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.5315ms | 0.4852ms | 2.0612 KOps/s | 2.0100 KOps/s | $\color{#35bf28}+2.55\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.8149ms | 3.7298ms | 268.1139 Ops/s | 255.5009 Ops/s | $\color{#35bf28}+4.94\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8705ms | 0.5045ms | 1.9823 KOps/s | 1.9518 KOps/s | $\color{#35bf28}+1.56\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7718ms | 0.4872ms | 2.0525 KOps/s | 2.0070 KOps/s | $\color{#35bf28}+2.27\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.4688ms | 1.7827ms | 560.9475 Ops/s | 565.1267 Ops/s | $\color{#d91a1a}-0.74\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.3088ms | 1.6784ms | 595.8061 Ops/s | 592.3143 Ops/s | $\color{#35bf28}+0.59\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.6716ms | 3.8568ms | 259.2824 Ops/s | 252.3214 Ops/s | $\color{#35bf28}+2.76\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.3138ms | 0.6507ms | 1.5368 KOps/s | 1.5471 KOps/s | $\color{#d91a1a}-0.67\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 1.0962ms | 0.6314ms | 1.5837 KOps/s | 1.5864 KOps/s | $\color{#d91a1a}-0.17\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.4870ms | 3.7211ms | 268.7391 Ops/s | 263.2433 Ops/s | $\color{#35bf28}+2.09\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.9883ms | 0.5105ms | 1.9589 KOps/s | 1.9415 KOps/s | $\color{#35bf28}+0.90\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.8435ms | 0.4954ms | 2.0186 KOps/s | 2.0185 KOps/s | $+0.00\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.5069ms | 3.6900ms | 271.0028 Ops/s | 253.1233 Ops/s | $\textbf{\color{#35bf28}+7.06\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7846ms | 0.5077ms | 1.9698 KOps/s | 1.9375 KOps/s | $\color{#35bf28}+1.66\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6229ms | 0.4836ms | 2.0680 KOps/s | 2.0113 KOps/s | $\color{#35bf28}+2.82\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.6247ms | 3.8419ms | 260.2891 Ops/s | 241.3082 Ops/s | $\textbf{\color{#35bf28}+7.87\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.2029ms | 0.6537ms | 1.5297 KOps/s | 1.5019 KOps/s | $\color{#35bf28}+1.85\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9218ms | 0.6322ms | 1.5817 KOps/s | 1.5798 KOps/s | $\color{#35bf28}+0.12\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1340s | 6.3576ms | 157.2926 Ops/s | 152.1497 Ops/s | $\color{#35bf28}+3.38\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 0.1295s | 15.5502ms | 64.3080 Ops/s | 64.3267 Ops/s | $\color{#d91a1a}-0.03\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 3.9512ms | 1.1552ms | 865.6184 Ops/s | 942.6096 Ops/s | $\textbf{\color{#d91a1a}-8.17\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1204s | 6.0343ms | 165.7179 Ops/s | 158.8890 Ops/s | $\color{#35bf28}+4.30\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 15.8838ms | 13.1624ms | 75.9739 Ops/s | 71.7731 Ops/s | $\textbf{\color{#35bf28}+5.85\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 3.7711ms | 1.1553ms | 865.6130 Ops/s | 949.7330 Ops/s | $\textbf{\color{#d91a1a}-8.86\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1171s | 6.1184ms | 163.4407 Ops/s | 153.2888 Ops/s | $\textbf{\color{#35bf28}+6.62\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 16.5973ms | 13.4061ms | 74.5928 Ops/s | 75.1129 Ops/s | $\color{#d91a1a}-0.69\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.8546ms | 1.2722ms | 786.0606 Ops/s | 804.8417 Ops/s | $\color{#d91a1a}-2.33\\%$ |
github-actions[bot] commented 1 week ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1643s | 0.1090s | 9.1701 Ops/s | 9.5940 Ops/s | $\color{#d91a1a}-4.42\\%$ | | test_sync | 90.9577ms | 88.1566ms | 11.3434 Ops/s | 11.0080 Ops/s | $\color{#35bf28}+3.05\\%$ | | test_async | 0.1722s | 75.3215ms | 13.2764 Ops/s | 11.5550 Ops/s | $\textbf{\color{#35bf28}+14.90\\%}$ | | test_single_pixels | 0.1141s | 0.1139s | 8.7804 Ops/s | 8.8709 Ops/s | $\color{#d91a1a}-1.02\\%$ | | test_sync_pixels | 70.0081ms | 68.3745ms | 14.6253 Ops/s | 14.4292 Ops/s | $\color{#35bf28}+1.36\\%$ | | test_async_pixels | 0.1295s | 64.8896ms | 15.4108 Ops/s | 17.5299 Ops/s | $\textbf{\color{#d91a1a}-12.09\\%}$ | | test_simple | 0.7340s | 0.7323s | 1.3656 Ops/s | 1.3574 Ops/s | $\color{#35bf28}+0.60\\%$ | | test_transformed | 1.0466s | 0.9827s | 1.0177 Ops/s | 1.0351 Ops/s | $\color{#d91a1a}-1.69\\%$ | | test_serial | 2.2300s | 2.1742s | 0.4599 Ops/s | 0.4677 Ops/s | $\color{#d91a1a}-1.66\\%$ | | test_parallel | 1.9064s | 1.8543s | 0.5393 Ops/s | 0.5386 Ops/s | $\color{#35bf28}+0.12\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1081ms | 33.0250μs | 30.2801 KOps/s | 30.1590 KOps/s | $\color{#35bf28}+0.40\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 39.2000μs | 19.1089μs | 52.3317 KOps/s | 51.4299 KOps/s | $\color{#35bf28}+1.75\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 44.9800μs | 18.5886μs | 53.7963 KOps/s | 53.0307 KOps/s | $\color{#35bf28}+1.44\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 28.2510μs | 10.9249μs | 91.5342 KOps/s | 90.5122 KOps/s | $\color{#35bf28}+1.13\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 66.6810μs | 34.8237μs | 28.7161 KOps/s | 29.0362 KOps/s | $\color{#d91a1a}-1.10\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 0.1131ms | 20.9302μs | 47.7778 KOps/s | 47.3080 KOps/s | $\color{#35bf28}+0.99\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 38.1810μs | 20.3708μs | 49.0899 KOps/s | 48.7058 KOps/s | $\color{#35bf28}+0.79\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 31.8600μs | 12.7889μs | 78.1931 KOps/s | 77.7262 KOps/s | $\color{#35bf28}+0.60\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 66.1710μs | 36.6980μs | 27.2494 KOps/s | 27.4527 KOps/s | $\color{#d91a1a}-0.74\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 40.9000μs | 22.9679μs | 43.5390 KOps/s | 43.4190 KOps/s | $\color{#35bf28}+0.28\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 42.1900μs | 20.5552μs | 48.6494 KOps/s | 49.0175 KOps/s | $\color{#d91a1a}-0.75\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 31.8500μs | 12.7918μs | 78.1751 KOps/s | 77.8779 KOps/s | $\color{#35bf28}+0.38\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 0.1500ms | 38.3968μs | 26.0438 KOps/s | 26.0361 KOps/s | $\color{#35bf28}+0.03\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 50.2710μs | 24.6599μs | 40.5516 KOps/s | 40.5548 KOps/s | $-0.01\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 46.9600μs | 22.1326μs | 45.1823 KOps/s | 44.9543 KOps/s | $\color{#35bf28}+0.51\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 82.2720μs | 14.5676μs | 68.6453 KOps/s | 68.1825 KOps/s | $\color{#35bf28}+0.68\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 62.8810μs | 36.9858μs | 27.0374 KOps/s | 27.2911 KOps/s | $\color{#d91a1a}-0.93\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 45.7810μs | 22.9428μs | 43.5866 KOps/s | 43.7024 KOps/s | $\color{#d91a1a}-0.27\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 48.7910μs | 24.3066μs | 41.1411 KOps/s | 41.3565 KOps/s | $\color{#d91a1a}-0.52\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 40.8900μs | 14.4896μs | 69.0152 KOps/s | 68.6871 KOps/s | $\color{#35bf28}+0.48\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 54.9110μs | 38.8538μs | 25.7375 KOps/s | 26.3252 KOps/s | $\color{#d91a1a}-2.23\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 44.7010μs | 24.6735μs | 40.5293 KOps/s | 40.5681 KOps/s | $\color{#d91a1a}-0.10\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 52.3510μs | 25.7830μs | 38.7852 KOps/s | 38.5899 KOps/s | $\color{#35bf28}+0.51\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 36.7010μs | 16.3851μs | 61.0312 KOps/s | 62.1328 KOps/s | $\color{#d91a1a}-1.77\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 72.3610μs | 40.9654μs | 24.4108 KOps/s | 25.0857 KOps/s | $\color{#d91a1a}-2.69\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 64.6810μs | 26.7723μs | 37.3520 KOps/s | 37.4268 KOps/s | $\color{#d91a1a}-0.20\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 47.0210μs | 26.1751μs | 38.2043 KOps/s | 38.7556 KOps/s | $\color{#d91a1a}-1.42\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 37.0510μs | 16.4525μs | 60.7810 KOps/s | 61.9868 KOps/s | $\color{#d91a1a}-1.95\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 73.0310μs | 43.3093μs | 23.0897 KOps/s | 23.6016 KOps/s | $\color{#d91a1a}-2.17\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 52.8410μs | 28.5895μs | 34.9779 KOps/s | 35.0209 KOps/s | $\color{#d91a1a}-0.12\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 46.6200μs | 27.5048μs | 36.3573 KOps/s | 36.2468 KOps/s | $\color{#35bf28}+0.30\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 44.5910μs | 18.1231μs | 55.1781 KOps/s | 55.7049 KOps/s | $\color{#d91a1a}-0.95\\%$ | | test_values[generalized_advantage_estimate-True-True] | 26.6924ms | 25.1974ms | 39.6866 Ops/s | 39.8571 Ops/s | $\color{#d91a1a}-0.43\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 89.7498ms | 2.6909ms | 371.6165 Ops/s | 372.8180 Ops/s | $\color{#d91a1a}-0.32\\%$ | | test_values[td0_return_estimate-False-False] | 92.5120μs | 65.2432μs | 15.3273 KOps/s | 15.0585 KOps/s | $\color{#35bf28}+1.78\\%$ | | test_values[td1_return_estimate-False-False] | 58.7964ms | 55.8698ms | 17.8988 Ops/s | 18.0302 Ops/s | $\color{#d91a1a}-0.73\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.2161ms | 1.0748ms | 930.3805 Ops/s | 921.0626 Ops/s | $\color{#35bf28}+1.01\\%$ | | test_values[td_lambda_return_estimate-True-False] | 93.8179ms | 89.1956ms | 11.2113 Ops/s | 11.2797 Ops/s | $\color{#d91a1a}-0.61\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.1962ms | 1.0722ms | 932.6477 Ops/s | 927.8529 Ops/s | $\color{#35bf28}+0.52\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 25.3808ms | 25.1346ms | 39.7857 Ops/s | 40.3314 Ops/s | $\color{#d91a1a}-1.35\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9651ms | 0.7128ms | 1.4029 KOps/s | 1.3865 KOps/s | $\color{#35bf28}+1.18\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7618ms | 0.6622ms | 1.5100 KOps/s | 1.4963 KOps/s | $\color{#35bf28}+0.92\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.4962ms | 1.4639ms | 683.1158 Ops/s | 682.7319 Ops/s | $\color{#35bf28}+0.06\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7479ms | 0.6825ms | 1.4653 KOps/s | 1.4608 KOps/s | $\color{#35bf28}+0.30\\%$ | | test_dqn_speed | 8.0087ms | 1.5118ms | 661.4662 Ops/s | 694.5049 Ops/s | $\color{#d91a1a}-4.76\\%$ | | test_ddpg_speed | 3.4765ms | 3.0727ms | 325.4507 Ops/s | 338.4146 Ops/s | $\color{#d91a1a}-3.83\\%$ | | test_sac_speed | 9.4956ms | 8.6802ms | 115.2048 Ops/s | 118.3929 Ops/s | $\color{#d91a1a}-2.69\\%$ | | test_redq_speed | 14.0516ms | 11.0866ms | 90.1990 Ops/s | 92.5123 Ops/s | $\color{#d91a1a}-2.50\\%$ | | test_redq_deprec_speed | 12.6174ms | 11.7182ms | 85.3375 Ops/s | 78.1966 Ops/s | $\textbf{\color{#35bf28}+9.13\\%}$ | | test_td3_speed | 18.4342ms | 8.7910ms | 113.7524 Ops/s | 119.7345 Ops/s | $\color{#d91a1a}-5.00\\%$ | | test_cql_speed | 27.2855ms | 26.6258ms | 37.5576 Ops/s | 38.5584 Ops/s | $\color{#d91a1a}-2.60\\%$ | | test_a2c_speed | 6.7638ms | 5.9134ms | 169.1070 Ops/s | 176.5928 Ops/s | $\color{#d91a1a}-4.24\\%$ | | test_ppo_speed | 7.3675ms | 6.2360ms | 160.3602 Ops/s | 166.6708 Ops/s | $\color{#d91a1a}-3.79\\%$ | | test_reinforce_speed | 5.3523ms | 4.8324ms | 206.9366 Ops/s | 212.7782 Ops/s | $\color{#d91a1a}-2.75\\%$ | | test_iql_speed | 20.4759ms | 19.9319ms | 50.1709 Ops/s | 51.2437 Ops/s | $\color{#d91a1a}-2.09\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.8102ms | 4.5817ms | 218.2607 Ops/s | 222.4072 Ops/s | $\color{#d91a1a}-1.86\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.1743ms | 0.5524ms | 1.8102 KOps/s | 1.8549 KOps/s | $\color{#d91a1a}-2.41\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7816ms | 0.5302ms | 1.8861 KOps/s | 1.9156 KOps/s | $\color{#d91a1a}-1.54\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.7682ms | 4.5653ms | 219.0418 Ops/s | 219.9605 Ops/s | $\color{#d91a1a}-0.42\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.1972ms | 0.5482ms | 1.8242 KOps/s | 1.8660 KOps/s | $\color{#d91a1a}-2.24\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7454ms | 0.5264ms | 1.8998 KOps/s | 1.9278 KOps/s | $\color{#d91a1a}-1.45\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.3062ms | 2.0404ms | 490.0987 Ops/s | 502.6350 Ops/s | $\color{#d91a1a}-2.49\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 6.0174ms | 1.9589ms | 510.4834 Ops/s | 525.6164 Ops/s | $\color{#d91a1a}-2.88\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.8265ms | 4.6655ms | 214.3389 Ops/s | 211.6417 Ops/s | $\color{#35bf28}+1.27\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9438ms | 0.7035ms | 1.4215 KOps/s | 1.4531 KOps/s | $\color{#d91a1a}-2.18\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.5710ms | 0.6836ms | 1.4627 KOps/s | 1.4907 KOps/s | $\color{#d91a1a}-1.87\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.7234ms | 4.5765ms | 218.5071 Ops/s | 220.7396 Ops/s | $\color{#d91a1a}-1.01\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7118ms | 0.5575ms | 1.7938 KOps/s | 1.8471 KOps/s | $\color{#d91a1a}-2.89\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7536ms | 0.5322ms | 1.8791 KOps/s | 1.9225 KOps/s | $\color{#d91a1a}-2.26\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.7458ms | 4.5369ms | 220.4144 Ops/s | 222.9244 Ops/s | $\color{#d91a1a}-1.13\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.3040ms | 0.5508ms | 1.8154 KOps/s | 1.8641 KOps/s | $\color{#d91a1a}-2.61\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.8836ms | 0.5287ms | 1.8915 KOps/s | 1.9408 KOps/s | $\color{#d91a1a}-2.54\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.8951ms | 4.7349ms | 211.1986 Ops/s | 213.1251 Ops/s | $\color{#d91a1a}-0.90\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8449ms | 0.7062ms | 1.4160 KOps/s | 1.4543 KOps/s | $\color{#d91a1a}-2.64\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9341ms | 0.6861ms | 1.4575 KOps/s | 1.5009 KOps/s | $\color{#d91a1a}-2.89\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1294s | 7.4755ms | 133.7703 Ops/s | 133.2077 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 20.1643ms | 15.8208ms | 63.2080 Ops/s | 63.9874 Ops/s | $\color{#d91a1a}-1.22\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.1897ms | 1.1537ms | 866.7797 Ops/s | 877.9186 Ops/s | $\color{#d91a1a}-1.27\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1284s | 9.9346ms | 100.6583 Ops/s | 138.0108 Ops/s | $\textbf{\color{#d91a1a}-27.06\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 17.9368ms | 15.7330ms | 63.5608 Ops/s | 63.8774 Ops/s | $\color{#d91a1a}-0.50\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.3080ms | 1.1495ms | 869.9265 Ops/s | 879.2294 Ops/s | $\color{#d91a1a}-1.06\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1282s | 7.5853ms | 131.8333 Ops/s | 100.4223 Ops/s | $\textbf{\color{#35bf28}+31.28\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.5891ms | 15.7747ms | 63.3928 Ops/s | 63.7538 Ops/s | $\color{#d91a1a}-0.57\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 7.6049ms | 1.4499ms | 689.7085 Ops/s | 773.8695 Ops/s | $\textbf{\color{#d91a1a}-10.88\\%}$ |