search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.27k stars 302 forks source link

[BugFix] Reinitialize vmap callers after reset of vmap randomness #2315

Closed vmoens closed 2 months ago

vmoens commented 2 months ago

Stack from ghstack (oldest at bottom):

pytorch-bot[bot] commented 2 months ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2315

Note: Links to docs will display an error until the docs builds have been completed.

:x: 3 New Failures, 1 Pending, 1 Unrelated Failure

As of commit d84cbacddb69956926aa7048815fe994d9ef0bba with merge base f840a1a4364bbb0bd33fbff7c4554e75af3ee1db (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2315#27845438326) ([gh](https://github.com/pytorch/rl/actions/runs/10072775503/job/27845438326)) `RuntimeError: Command docker exec -t 193c3a740a1d51eebb0edbc54cdf729d2091416f6dd9b957135b3a572a247e22 /exec failed with exit code 139` * [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2315#27845509807) ([gh](https://github.com/pytorch/rl/actions/runs/10072775510/job/27845509807)) `AttributeError: module 'torch' has no attribute 'compiler'` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2315#27845407900) ([gh](https://github.com/pytorch/rl/actions/runs/10072775497/job/27845407900)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2315#27845480848) ([gh](https://github.com/pytorch/rl/actions/runs/10072775498/job/27845480848)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/f840a1a4364bbb0bd33fbff7c4554e75af3ee1db#27805197076)) `AttributeError: module 'torch' has no attribute 'compiler'`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 2 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 57.7001ms | 57.4561ms | 17.4046 Ops/s | 17.5799 Ops/s | $\color{#d91a1a}-1.00\\%$ | | test_sync | 42.0279ms | 31.7046ms | 31.5412 Ops/s | 31.1489 Ops/s | $\color{#35bf28}+1.26\\%$ | | test_async | 58.5235ms | 29.7244ms | 33.6424 Ops/s | 33.6075 Ops/s | $\color{#35bf28}+0.10\\%$ | | test_simple | 0.4722s | 0.4051s | 2.4686 Ops/s | 2.4827 Ops/s | $\color{#d91a1a}-0.57\\%$ | | test_transformed | 0.6196s | 0.5608s | 1.7831 Ops/s | 1.7894 Ops/s | $\color{#d91a1a}-0.35\\%$ | | test_serial | 1.3029s | 1.2458s | 0.8027 Ops/s | 0.8161 Ops/s | $\color{#d91a1a}-1.64\\%$ | | test_parallel | 1.1575s | 1.1007s | 0.9085 Ops/s | 0.9082 Ops/s | $\color{#35bf28}+0.04\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.2107ms | 25.0572μs | 39.9087 KOps/s | 39.9857 KOps/s | $\color{#d91a1a}-0.19\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 40.3450μs | 14.5702μs | 68.6333 KOps/s | 69.7298 KOps/s | $\color{#d91a1a}-1.57\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 54.4420μs | 14.4538μs | 69.1860 KOps/s | 69.0478 KOps/s | $\color{#35bf28}+0.20\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 33.0020μs | 8.4489μs | 118.3592 KOps/s | 119.0959 KOps/s | $\color{#d91a1a}-0.62\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 55.1740μs | 26.9060μs | 37.1664 KOps/s | 37.4876 KOps/s | $\color{#d91a1a}-0.86\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 41.9180μs | 16.2088μs | 61.6948 KOps/s | 62.2699 KOps/s | $\color{#d91a1a}-0.92\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 82.6250μs | 16.0914μs | 62.1450 KOps/s | 62.2496 KOps/s | $\color{#d91a1a}-0.17\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 35.0050μs | 10.0546μs | 99.4565 KOps/s | 100.2678 KOps/s | $\color{#d91a1a}-0.81\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 62.8470μs | 28.4580μs | 35.1395 KOps/s | 35.3230 KOps/s | $\color{#d91a1a}-0.52\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 41.3870μs | 17.8553μs | 56.0059 KOps/s | 56.1261 KOps/s | $\color{#d91a1a}-0.21\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 58.7700μs | 16.1506μs | 61.9171 KOps/s | 62.4277 KOps/s | $\color{#d91a1a}-0.82\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 36.3280μs | 10.0407μs | 99.5948 KOps/s | 101.7261 KOps/s | $\color{#d91a1a}-2.10\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 82.1110μs | 29.6945μs | 33.6763 KOps/s | 33.7565 KOps/s | $\color{#d91a1a}-0.24\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 54.2310μs | 19.3039μs | 51.8029 KOps/s | 52.7206 KOps/s | $\color{#d91a1a}-1.74\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 44.5030μs | 17.6301μs | 56.7213 KOps/s | 56.6976 KOps/s | $\color{#35bf28}+0.04\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 47.3480μs | 11.4672μs | 87.2051 KOps/s | 88.4905 KOps/s | $\color{#d91a1a}-1.45\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 68.1870μs | 28.5656μs | 35.0071 KOps/s | 35.4232 KOps/s | $\color{#d91a1a}-1.17\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 42.3890μs | 17.9588μs | 55.6829 KOps/s | 57.7523 KOps/s | $\color{#d91a1a}-3.58\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 50.1030μs | 18.9296μs | 52.8272 KOps/s | 53.8901 KOps/s | $\color{#d91a1a}-1.97\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 39.9550μs | 11.3842μs | 87.8408 KOps/s | 89.1993 KOps/s | $\color{#d91a1a}-1.52\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 73.3680μs | 30.0945μs | 33.2286 KOps/s | 33.8055 KOps/s | $\color{#d91a1a}-1.71\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 47.4790μs | 19.3494μs | 51.6811 KOps/s | 52.6641 KOps/s | $\color{#d91a1a}-1.87\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 50.7040μs | 20.0763μs | 49.8100 KOps/s | 50.3503 KOps/s | $\color{#d91a1a}-1.07\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 51.7560μs | 12.7835μs | 78.2260 KOps/s | 79.9324 KOps/s | $\color{#d91a1a}-2.13\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.8519ms | 31.8740μs | 31.3735 KOps/s | 31.3463 KOps/s | $\color{#35bf28}+0.09\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 0.1148ms | 21.1145μs | 47.3609 KOps/s | 48.7323 KOps/s | $\color{#d91a1a}-2.81\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 44.5930μs | 20.3586μs | 49.1194 KOps/s | 50.3010 KOps/s | $\color{#d91a1a}-2.35\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 40.5850μs | 12.8226μs | 77.9875 KOps/s | 79.5385 KOps/s | $\color{#d91a1a}-1.95\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 93.1440μs | 32.7351μs | 30.5482 KOps/s | 30.5258 KOps/s | $\color{#35bf28}+0.07\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 63.3380μs | 22.3778μs | 44.6872 KOps/s | 44.9187 KOps/s | $\color{#d91a1a}-0.52\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 53.1190μs | 21.3939μs | 46.7423 KOps/s | 47.2797 KOps/s | $\color{#d91a1a}-1.14\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 36.8090μs | 14.1816μs | 70.5139 KOps/s | 71.7080 KOps/s | $\color{#d91a1a}-1.67\\%$ | | test_values[generalized_advantage_estimate-True-True] | 13.1188ms | 9.4379ms | 105.9557 Ops/s | 102.0750 Ops/s | $\color{#35bf28}+3.80\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 38.9500ms | 35.3788ms | 28.2655 Ops/s | 27.9400 Ops/s | $\color{#35bf28}+1.17\\%$ | | test_values[td0_return_estimate-False-False] | 0.2204ms | 0.1663ms | 6.0131 KOps/s | 5.9821 KOps/s | $\color{#35bf28}+0.52\\%$ | | test_values[td1_return_estimate-False-False] | 25.3263ms | 23.5985ms | 42.3755 Ops/s | 41.4708 Ops/s | $\color{#35bf28}+2.18\\%$ | | test_values[vec_td1_return_estimate-False-False] | 38.2597ms | 35.4848ms | 28.1811 Ops/s | 27.7956 Ops/s | $\color{#35bf28}+1.39\\%$ | | test_values[td_lambda_return_estimate-True-False] | 37.2098ms | 33.5690ms | 29.7894 Ops/s | 28.7546 Ops/s | $\color{#35bf28}+3.60\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 37.3895ms | 35.4271ms | 28.2270 Ops/s | 27.7834 Ops/s | $\color{#35bf28}+1.60\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.5389ms | 8.2351ms | 121.4318 Ops/s | 119.7602 Ops/s | $\color{#35bf28}+1.40\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.2678ms | 2.0152ms | 496.2254 Ops/s | 529.3282 Ops/s | $\textbf{\color{#d91a1a}-6.25\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.5525ms | 0.3539ms | 2.8256 KOps/s | 2.8227 KOps/s | $\color{#35bf28}+0.10\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 45.3044ms | 43.8414ms | 22.8095 Ops/s | 21.9432 Ops/s | $\color{#35bf28}+3.95\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.8384ms | 3.0396ms | 328.9861 Ops/s | 328.8116 Ops/s | $\color{#35bf28}+0.05\\%$ | | test_dqn_speed | 1.8285ms | 1.3687ms | 730.6389 Ops/s | 720.7009 Ops/s | $\color{#35bf28}+1.38\\%$ | | test_ddpg_speed | 3.7578ms | 2.8915ms | 345.8363 Ops/s | 342.2101 Ops/s | $\color{#35bf28}+1.06\\%$ | | test_sac_speed | 9.8004ms | 8.4117ms | 118.8818 Ops/s | 118.5882 Ops/s | $\color{#35bf28}+0.25\\%$ | | test_redq_speed | 14.7341ms | 13.5443ms | 73.8316 Ops/s | 74.7890 Ops/s | $\color{#d91a1a}-1.28\\%$ | | test_redq_deprec_speed | 15.6387ms | 13.0567ms | 76.5889 Ops/s | 76.2648 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_td3_speed | 8.5798ms | 8.3448ms | 119.8348 Ops/s | 119.7233 Ops/s | $\color{#35bf28}+0.09\\%$ | | test_cql_speed | 38.4071ms | 36.1683ms | 27.6485 Ops/s | 27.5326 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_a2c_speed | 8.3523ms | 7.3971ms | 135.1878 Ops/s | 134.9529 Ops/s | $\color{#35bf28}+0.17\\%$ | | test_ppo_speed | 9.0415ms | 7.6857ms | 130.1119 Ops/s | 129.8716 Ops/s | $\color{#35bf28}+0.19\\%$ | | test_reinforce_speed | 7.4902ms | 6.5768ms | 152.0501 Ops/s | 151.9199 Ops/s | $\color{#35bf28}+0.09\\%$ | | test_iql_speed | 33.0998ms | 32.4651ms | 30.8023 Ops/s | 30.9229 Ops/s | $\color{#d91a1a}-0.39\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 7.4843ms | 4.7889ms | 208.8154 Ops/s | 209.8162 Ops/s | $\color{#d91a1a}-0.48\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7042ms | 0.4746ms | 2.1071 KOps/s | 2.0952 KOps/s | $\color{#35bf28}+0.57\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6724ms | 0.4528ms | 2.2083 KOps/s | 2.1983 KOps/s | $\color{#35bf28}+0.46\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.9970ms | 4.6652ms | 214.3553 Ops/s | 213.4662 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7290ms | 0.4731ms | 2.1135 KOps/s | 2.0017 KOps/s | $\textbf{\color{#35bf28}+5.59\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7224ms | 0.4562ms | 2.1919 KOps/s | 2.2379 KOps/s | $\color{#d91a1a}-2.06\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 1.8109ms | 1.6840ms | 593.8331 Ops/s | 588.6145 Ops/s | $\color{#35bf28}+0.89\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.1484ms | 1.6042ms | 623.3457 Ops/s | 621.9029 Ops/s | $\color{#35bf28}+0.23\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.2904ms | 4.9067ms | 203.8020 Ops/s | 203.0677 Ops/s | $\color{#35bf28}+0.36\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.4066ms | 0.6141ms | 1.6283 KOps/s | 1.6241 KOps/s | $\color{#35bf28}+0.26\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8657ms | 0.5868ms | 1.7040 KOps/s | 1.6973 KOps/s | $\color{#35bf28}+0.40\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.0453ms | 4.7572ms | 210.2080 Ops/s | 207.9627 Ops/s | $\color{#35bf28}+1.08\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6044ms | 0.4784ms | 2.0902 KOps/s | 2.1013 KOps/s | $\color{#d91a1a}-0.52\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 7.3551ms | 0.4641ms | 2.1548 KOps/s | 2.1926 KOps/s | $\color{#d91a1a}-1.72\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 7.6807ms | 4.7357ms | 211.1642 Ops/s | 212.4174 Ops/s | $\color{#d91a1a}-0.59\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.2363ms | 0.4727ms | 2.1154 KOps/s | 2.1126 KOps/s | $\color{#35bf28}+0.13\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6498ms | 0.4475ms | 2.2346 KOps/s | 2.2370 KOps/s | $\color{#d91a1a}-0.11\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0729ms | 4.8807ms | 204.8870 Ops/s | 203.6312 Ops/s | $\color{#35bf28}+0.62\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.1193s | 0.7756ms | 1.2893 KOps/s | 1.6230 KOps/s | $\textbf{\color{#d91a1a}-20.56\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7407ms | 0.5926ms | 1.6876 KOps/s | 1.7063 KOps/s | $\color{#d91a1a}-1.10\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1142s | 6.0253ms | 165.9680 Ops/s | 164.1544 Ops/s | $\color{#35bf28}+1.10\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 16.4379ms | 12.5658ms | 79.5813 Ops/s | 78.7913 Ops/s | $\color{#35bf28}+1.00\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 1.6222ms | 1.0987ms | 910.1378 Ops/s | 924.4193 Ops/s | $\color{#d91a1a}-1.54\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1061s | 5.8281ms | 171.5817 Ops/s | 174.4783 Ops/s | $\color{#d91a1a}-1.66\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 16.3490ms | 12.5902ms | 79.4269 Ops/s | 79.1774 Ops/s | $\color{#35bf28}+0.32\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.7826ms | 1.0874ms | 919.5885 Ops/s | 953.6677 Ops/s | $\color{#d91a1a}-3.57\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1091s | 8.1360ms | 122.9102 Ops/s | 122.9455 Ops/s | $\color{#d91a1a}-0.03\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 16.8266ms | 12.7972ms | 78.1422 Ops/s | 77.8150 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.8705ms | 1.1923ms | 838.7298 Ops/s | 836.5805 Ops/s | $\color{#35bf28}+0.26\\%$ |
github-actions[bot] commented 2 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1064s | 0.1059s | 9.4391 Ops/s | 8.5959 Ops/s | $\textbf{\color{#35bf28}+9.81\\%}$ | | test_sync | 95.0479ms | 94.7143ms | 10.5581 Ops/s | 10.5583 Ops/s | $-0.00\\%$ | | test_async | 0.2471s | 90.7536ms | 11.0188 Ops/s | 11.0508 Ops/s | $\color{#d91a1a}-0.29\\%$ | | test_single_pixels | 0.1187s | 0.1177s | 8.4933 Ops/s | 8.5505 Ops/s | $\color{#d91a1a}-0.67\\%$ | | test_sync_pixels | 78.6323ms | 75.1476ms | 13.3071 Ops/s | 13.7375 Ops/s | $\color{#d91a1a}-3.13\\%$ | | test_async_pixels | 0.1483s | 69.3864ms | 14.4120 Ops/s | 14.5300 Ops/s | $\color{#d91a1a}-0.81\\%$ | | test_simple | 0.8632s | 0.7880s | 1.2690 Ops/s | 1.2242 Ops/s | $\color{#35bf28}+3.66\\%$ | | test_transformed | 1.0874s | 1.0182s | 0.9821 Ops/s | 0.9835 Ops/s | $\color{#d91a1a}-0.14\\%$ | | test_serial | 2.2916s | 2.2188s | 0.4507 Ops/s | 0.4318 Ops/s | $\color{#35bf28}+4.38\\%$ | | test_parallel | 2.0361s | 1.9709s | 0.5074 Ops/s | 0.5052 Ops/s | $\color{#35bf28}+0.43\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1102ms | 38.3158μs | 26.0989 KOps/s | 26.0130 KOps/s | $\color{#35bf28}+0.33\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 44.3610μs | 21.7263μs | 46.0271 KOps/s | 45.9109 KOps/s | $\color{#35bf28}+0.25\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 48.1010μs | 21.8198μs | 45.8300 KOps/s | 46.3121 KOps/s | $\color{#d91a1a}-1.04\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 31.1800μs | 12.2012μs | 81.9594 KOps/s | 80.4421 KOps/s | $\color{#35bf28}+1.89\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 65.3510μs | 40.5389μs | 24.6677 KOps/s | 24.5549 KOps/s | $\color{#35bf28}+0.46\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 50.5210μs | 23.9255μs | 41.7964 KOps/s | 41.7928 KOps/s | $+0.01\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 52.1200μs | 24.0979μs | 41.4975 KOps/s | 42.4027 KOps/s | $\color{#d91a1a}-2.13\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 46.4910μs | 14.5605μs | 68.6790 KOps/s | 68.3962 KOps/s | $\color{#35bf28}+0.41\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 87.2910μs | 43.2150μs | 23.1401 KOps/s | 23.5127 KOps/s | $\color{#d91a1a}-1.58\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 49.2010μs | 26.1776μs | 38.2006 KOps/s | 37.6821 KOps/s | $\color{#35bf28}+1.38\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 52.7910μs | 24.1157μs | 41.4667 KOps/s | 42.7394 KOps/s | $\color{#d91a1a}-2.98\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 40.8010μs | 14.7040μs | 68.0089 KOps/s | 68.2526 KOps/s | $\color{#d91a1a}-0.36\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 83.9320μs | 44.6206μs | 22.4111 KOps/s | 21.9970 KOps/s | $\color{#35bf28}+1.88\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 49.2410μs | 28.4196μs | 35.1870 KOps/s | 34.7591 KOps/s | $\color{#35bf28}+1.23\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 48.4210μs | 26.2363μs | 38.1151 KOps/s | 38.9432 KOps/s | $\color{#d91a1a}-2.13\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 37.5910μs | 16.8382μs | 59.3887 KOps/s | 59.2355 KOps/s | $\color{#35bf28}+0.26\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 70.6710μs | 43.1409μs | 23.1798 KOps/s | 23.6223 KOps/s | $\color{#d91a1a}-1.87\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 58.1710μs | 26.4780μs | 37.7671 KOps/s | 38.1931 KOps/s | $\color{#d91a1a}-1.12\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 51.1110μs | 28.4597μs | 35.1374 KOps/s | 35.2135 KOps/s | $\color{#d91a1a}-0.22\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 38.1600μs | 17.0026μs | 58.8145 KOps/s | 58.7182 KOps/s | $\color{#35bf28}+0.16\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 68.3020μs | 44.5530μs | 22.4452 KOps/s | 21.9236 KOps/s | $\color{#35bf28}+2.38\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 62.0610μs | 28.5652μs | 35.0077 KOps/s | 35.1304 KOps/s | $\color{#d91a1a}-0.35\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 57.7010μs | 30.5386μs | 32.7454 KOps/s | 32.5890 KOps/s | $\color{#35bf28}+0.48\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 38.5710μs | 19.0762μs | 52.4215 KOps/s | 50.5935 KOps/s | $\color{#35bf28}+3.61\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.8399ms | 47.9515μs | 20.8544 KOps/s | 20.9481 KOps/s | $\color{#d91a1a}-0.45\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 58.0510μs | 31.2651μs | 31.9845 KOps/s | 32.2668 KOps/s | $\color{#d91a1a}-0.87\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 55.4110μs | 30.9563μs | 32.3036 KOps/s | 32.5883 KOps/s | $\color{#d91a1a}-0.87\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 45.1310μs | 19.0261μs | 52.5595 KOps/s | 50.6922 KOps/s | $\color{#35bf28}+3.68\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 80.6310μs | 49.1381μs | 20.3508 KOps/s | 20.4648 KOps/s | $\color{#d91a1a}-0.56\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 53.7710μs | 33.3118μs | 30.0194 KOps/s | 30.0311 KOps/s | $\color{#d91a1a}-0.04\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 58.4210μs | 32.5897μs | 30.6845 KOps/s | 30.9405 KOps/s | $\color{#d91a1a}-0.83\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 45.5710μs | 21.5089μs | 46.4924 KOps/s | 47.0463 KOps/s | $\color{#d91a1a}-1.18\\%$ | | test_values[generalized_advantage_estimate-True-True] | 24.1323ms | 23.4727ms | 42.6026 Ops/s | 41.4405 Ops/s | $\color{#35bf28}+2.80\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 96.7185ms | 2.8220ms | 354.3549 Ops/s | 365.9189 Ops/s | $\color{#d91a1a}-3.16\\%$ | | test_values[td0_return_estimate-False-False] | 92.0720μs | 64.8299μs | 15.4250 KOps/s | 15.2497 KOps/s | $\color{#35bf28}+1.15\\%$ | | test_values[td1_return_estimate-False-False] | 53.5193ms | 53.1390ms | 18.8186 Ops/s | 18.3378 Ops/s | $\color{#35bf28}+2.62\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.3015ms | 1.0709ms | 933.7946 Ops/s | 917.8631 Ops/s | $\color{#35bf28}+1.74\\%$ | | test_values[td_lambda_return_estimate-True-False] | 84.5739ms | 84.1732ms | 11.8803 Ops/s | 11.3624 Ops/s | $\color{#35bf28}+4.56\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.2750ms | 1.0670ms | 937.2343 Ops/s | 924.5437 Ops/s | $\color{#35bf28}+1.37\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 23.7982ms | 23.6770ms | 42.2350 Ops/s | 39.6057 Ops/s | $\textbf{\color{#35bf28}+6.64\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9520ms | 0.7046ms | 1.4193 KOps/s | 1.4022 KOps/s | $\color{#35bf28}+1.21\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7384ms | 0.6547ms | 1.5273 KOps/s | 1.5034 KOps/s | $\color{#35bf28}+1.60\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5150ms | 1.4541ms | 687.6870 Ops/s | 682.5911 Ops/s | $\color{#35bf28}+0.75\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.8321ms | 0.6692ms | 1.4943 KOps/s | 1.4245 KOps/s | $\color{#35bf28}+4.90\\%$ | | test_dqn_speed | 7.1190ms | 1.4475ms | 690.8536 Ops/s | 689.3344 Ops/s | $\color{#35bf28}+0.22\\%$ | | test_ddpg_speed | 3.1908ms | 2.9537ms | 338.5615 Ops/s | 334.5781 Ops/s | $\color{#35bf28}+1.19\\%$ | | test_sac_speed | 0.1015s | 9.0720ms | 110.2291 Ops/s | 118.9805 Ops/s | $\textbf{\color{#d91a1a}-7.36\\%}$ | | test_redq_speed | 11.6466ms | 10.8685ms | 92.0094 Ops/s | 92.1446 Ops/s | $\color{#d91a1a}-0.15\\%$ | | test_redq_deprec_speed | 12.4738ms | 11.3444ms | 88.1489 Ops/s | 88.7733 Ops/s | $\color{#d91a1a}-0.70\\%$ | | test_td3_speed | 8.5707ms | 8.3472ms | 119.8009 Ops/s | 120.0787 Ops/s | $\color{#d91a1a}-0.23\\%$ | | test_cql_speed | 27.2529ms | 26.3294ms | 37.9804 Ops/s | 34.8994 Ops/s | $\textbf{\color{#35bf28}+8.83\\%}$ | | test_a2c_speed | 6.0298ms | 5.8011ms | 172.3823 Ops/s | 178.0704 Ops/s | $\color{#d91a1a}-3.19\\%$ | | test_ppo_speed | 6.8418ms | 6.0973ms | 164.0067 Ops/s | 168.2248 Ops/s | $\color{#d91a1a}-2.51\\%$ | | test_reinforce_speed | 5.0314ms | 4.6562ms | 214.7673 Ops/s | 217.1177 Ops/s | $\color{#d91a1a}-1.08\\%$ | | test_iql_speed | 20.2044ms | 19.7575ms | 50.6137 Ops/s | 49.4837 Ops/s | $\color{#35bf28}+2.28\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.9381ms | 6.7435ms | 148.2914 Ops/s | 149.0074 Ops/s | $\color{#d91a1a}-0.48\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.1110s | 0.5871ms | 1.7033 KOps/s | 1.9476 KOps/s | $\textbf{\color{#d91a1a}-12.54\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6474ms | 0.4914ms | 2.0349 KOps/s | 2.0236 KOps/s | $\color{#35bf28}+0.56\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.9147ms | 6.6438ms | 150.5156 Ops/s | 151.5548 Ops/s | $\color{#d91a1a}-0.69\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.0673ms | 0.5082ms | 1.9676 KOps/s | 1.9661 KOps/s | $\color{#35bf28}+0.08\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6634ms | 0.4849ms | 2.0623 KOps/s | 2.0529 KOps/s | $\color{#35bf28}+0.46\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.0867ms | 1.9225ms | 520.1638 Ops/s | 513.5790 Ops/s | $\color{#35bf28}+1.28\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.0672ms | 1.8335ms | 545.4123 Ops/s | 540.3529 Ops/s | $\color{#35bf28}+0.94\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.9241ms | 6.8323ms | 146.3641 Ops/s | 146.4861 Ops/s | $\color{#d91a1a}-0.08\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.7207ms | 0.6919ms | 1.4454 KOps/s | 1.5114 KOps/s | $\color{#d91a1a}-4.37\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8485ms | 0.6352ms | 1.5744 KOps/s | 1.5647 KOps/s | $\color{#35bf28}+0.62\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.9220ms | 6.7397ms | 148.3754 Ops/s | 149.4213 Ops/s | $\color{#d91a1a}-0.70\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.8422ms | 0.5155ms | 1.9398 KOps/s | 1.9435 KOps/s | $\color{#d91a1a}-0.19\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6239ms | 0.4923ms | 2.0312 KOps/s | 2.0165 KOps/s | $\color{#35bf28}+0.73\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 7.0357ms | 6.6659ms | 150.0180 Ops/s | 150.6363 Ops/s | $\color{#d91a1a}-0.41\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.1280s | 0.6869ms | 1.4558 KOps/s | 1.9621 KOps/s | $\textbf{\color{#d91a1a}-25.80\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6754ms | 0.4880ms | 2.0493 KOps/s | 2.0390 KOps/s | $\color{#35bf28}+0.51\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.9328ms | 6.8705ms | 145.5496 Ops/s | 146.0890 Ops/s | $\color{#d91a1a}-0.37\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.7562ms | 0.6557ms | 1.5252 KOps/s | 1.5131 KOps/s | $\color{#35bf28}+0.80\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8260ms | 0.6379ms | 1.5676 KOps/s | 1.5431 KOps/s | $\color{#35bf28}+1.58\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1279s | 7.6597ms | 130.5535 Ops/s | 125.2636 Ops/s | $\color{#35bf28}+4.22\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.8904ms | 16.2467ms | 61.5510 Ops/s | 62.2514 Ops/s | $\color{#d91a1a}-1.13\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 7.3499ms | 1.3630ms | 733.6811 Ops/s | 753.9083 Ops/s | $\color{#d91a1a}-2.68\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1257s | 7.6291ms | 131.0779 Ops/s | 131.5151 Ops/s | $\color{#d91a1a}-0.33\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1361s | 18.6232ms | 53.6964 Ops/s | 62.0125 Ops/s | $\textbf{\color{#d91a1a}-13.41\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 7.3953ms | 1.3188ms | 758.2706 Ops/s | 751.9208 Ops/s | $\color{#35bf28}+0.84\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1248s | 7.7821ms | 128.4999 Ops/s | 129.2091 Ops/s | $\color{#d91a1a}-0.55\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 19.1662ms | 16.4666ms | 60.7290 Ops/s | 62.3925 Ops/s | $\color{#d91a1a}-2.67\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 7.5804ms | 1.4933ms | 669.6449 Ops/s | 736.0176 Ops/s | $\textbf{\color{#d91a1a}-9.02\\%}$ |