[CI] Fix CI

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2245

• :page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

:x: 3 New Failures, 1 Unrelated Failure

As of commit 048c6bf5770adb4687258b19cd278deed6ad8fff with merge base 00b7c2e8f38730747f9484e7fa3b763e509cc914 ():

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2245#26604715669) ([gh](https://github.com/pytorch/rl/actions/runs/9646980630/job/26604715669)) `RuntimeError: Command docker exec -t 620f40f20e7ebb5aef8e81c184ae94d6d34d084872872058b4c81c2a71cea64b /exec failed with exit code 139` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2245#26604719466) ([gh](https://github.com/pytorch/rl/actions/runs/9646980652/job/26604719466)) `RuntimeError: Command docker exec -t 841ca67fc24b7df55a83b7ed6af83b510220e44eec1e253851fc01751f17f43d /exec failed with exit code 1` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2245#26604720843) ([gh](https://github.com/pytorch/rl/actions/runs/9646980656/job/26604720843)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Libs Tests on Linux / unittests-robohive (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2245#26604721950) ([gh](https://github.com/pytorch/rl/actions/runs/9646980644/job/26604721950)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/00b7c2e8f38730747f9484e7fa3b763e509cc914#26593212812)) `test/test_libs.py::TestRoboHive::test_robohive[franka_slide_random-v3-True-True]`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions[bot]]

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results

| Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1203s | 61.8048ms | 16.1800 Ops/s | 17.4789 Ops/s | $\textbf{\color{#d91a1a}-7.43\\%}$ | | test_sync | 47.1236ms | 32.6765ms | 30.6030 Ops/s | 31.1961 Ops/s | $\color{#d91a1a}-1.90\\%$ | | test_async | 47.4761ms | 29.1262ms | 34.3333 Ops/s | 33.0042 Ops/s | $\color{#35bf28}+4.03\\%$ | | test_simple | 0.4779s | 0.4112s | 2.4321 Ops/s | 2.5790 Ops/s | $\textbf{\color{#d91a1a}-5.69\\%}$ | | test_transformed | 0.5615s | 0.5569s | 1.7955 Ops/s | 1.8236 Ops/s | $\color{#d91a1a}-1.54\\%$ | | test_serial | 1.3762s | 1.3182s | 0.7586 Ops/s | 0.7651 Ops/s | $\color{#d91a1a}-0.84\\%$ | | test_parallel | 1.1662s | 1.1162s | 0.8959 Ops/s | 0.8672 Ops/s | $\color{#35bf28}+3.30\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1963ms | 22.8107μs | 43.8391 KOps/s | 43.7811 KOps/s | $\color{#35bf28}+0.13\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 40.5960μs | 13.5204μs | 73.9624 KOps/s | 74.8796 KOps/s | $\color{#d91a1a}-1.22\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 37.8410μs | 13.6147μs | 73.4498 KOps/s | 75.7144 KOps/s | $\color{#d91a1a}-2.99\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 42.9910μs | 7.9802μs | 125.3109 KOps/s | 127.8474 KOps/s | $\color{#d91a1a}-1.98\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 72.4960μs | 24.4258μs | 40.9403 KOps/s | 41.4242 KOps/s | $\color{#d91a1a}-1.17\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 46.4880μs | 14.7488μs | 67.8023 KOps/s | 67.9061 KOps/s | $\color{#d91a1a}-0.15\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 91.4320μs | 14.8456μs | 67.3600 KOps/s | 69.2842 KOps/s | $\color{#d91a1a}-2.78\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 32.9620μs | 9.2845μs | 107.7063 KOps/s | 110.3131 KOps/s | $\color{#d91a1a}-2.36\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 52.9090μs | 26.0629μs | 38.3687 KOps/s | 39.4172 KOps/s | $\color{#d91a1a}-2.66\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 44.6340μs | 16.2014μs | 61.7229 KOps/s | 62.1997 KOps/s | $\color{#d91a1a}-0.77\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 39.4640μs | 14.7928μs | 67.6007 KOps/s | 68.8287 KOps/s | $\color{#d91a1a}-1.78\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 37.2990μs | 9.2131μs | 108.5411 KOps/s | 109.9510 KOps/s | $\color{#d91a1a}-1.28\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 85.2000μs | 26.8361μs | 37.2632 KOps/s | 37.7535 KOps/s | $\color{#d91a1a}-1.30\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 45.7960μs | 17.4997μs | 57.1437 KOps/s | 57.9788 KOps/s | $\color{#d91a1a}-1.44\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 41.6990μs | 16.0143μs | 62.4441 KOps/s | 64.2668 KOps/s | $\color{#d91a1a}-2.84\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 0.1217ms | 10.3735μs | 96.3999 KOps/s | 97.5061 KOps/s | $\color{#d91a1a}-1.13\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 0.1931ms | 26.3336μs | 37.9743 KOps/s | 39.6526 KOps/s | $\color{#d91a1a}-4.23\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 47.1390μs | 16.0457μs | 62.3221 KOps/s | 61.9156 KOps/s | $\color{#35bf28}+0.66\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 47.4790μs | 17.0258μs | 58.7343 KOps/s | 59.3797 KOps/s | $\color{#d91a1a}-1.09\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 37.2800μs | 10.3491μs | 96.6265 KOps/s | 96.0830 KOps/s | $\color{#35bf28}+0.57\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 64.6910μs | 26.9342μs | 37.1276 KOps/s | 37.9927 KOps/s | $\color{#d91a1a}-2.28\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 50.3640μs | 17.3257μs | 57.7178 KOps/s | 57.8300 KOps/s | $\color{#d91a1a}-0.19\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 64.9520μs | 18.2869μs | 54.6841 KOps/s | 55.6572 KOps/s | $\color{#d91a1a}-1.75\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 38.4520μs | 11.5186μs | 86.8161 KOps/s | 87.1306 KOps/s | $\color{#d91a1a}-0.36\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 70.0120μs | 28.4547μs | 35.1436 KOps/s | 36.0706 KOps/s | $\color{#d91a1a}-2.57\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 0.2090ms | 18.5840μs | 53.8097 KOps/s | 53.6693 KOps/s | $\color{#35bf28}+0.26\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 0.1041ms | 18.3718μs | 54.4313 KOps/s | 55.4771 KOps/s | $\color{#d91a1a}-1.89\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 39.5940μs | 11.6275μs | 86.0029 KOps/s | 86.7460 KOps/s | $\color{#d91a1a}-0.86\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 43.0010μs | 29.9408μs | 33.3992 KOps/s | 34.9721 KOps/s | $\color{#d91a1a}-4.50\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 47.3990μs | 19.5492μs | 51.1531 KOps/s | 51.4064 KOps/s | $\color{#d91a1a}-0.49\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 47.3580μs | 19.2349μs | 51.9890 KOps/s | 53.4752 KOps/s | $\color{#d91a1a}-2.78\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 36.7990μs | 12.6659μs | 78.9524 KOps/s | 80.4674 KOps/s | $\color{#d91a1a}-1.88\\%$ | | test_values[generalized_advantage_estimate-True-True] | 13.3311ms | 9.9230ms | 100.7760 Ops/s | 106.0133 Ops/s | $\color{#d91a1a}-4.94\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 36.1870ms | 33.7777ms | 29.6053 Ops/s | 27.7970 Ops/s | $\textbf{\color{#35bf28}+6.51\\%}$ | | test_values[td0_return_estimate-False-False] | 0.2506ms | 0.1818ms | 5.4992 KOps/s | 5.7382 KOps/s | $\color{#d91a1a}-4.17\\%$ | | test_values[td1_return_estimate-False-False] | 27.2145ms | 24.7849ms | 40.3471 Ops/s | 42.1652 Ops/s | $\color{#d91a1a}-4.31\\%$ | | test_values[vec_td1_return_estimate-False-False] | 34.8384ms | 33.8297ms | 29.5598 Ops/s | 27.9738 Ops/s | $\textbf{\color{#35bf28}+5.67\\%}$ | | test_values[td_lambda_return_estimate-True-False] | 39.0518ms | 35.9749ms | 27.7971 Ops/s | 29.0680 Ops/s | $\color{#d91a1a}-4.37\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 35.2505ms | 33.8253ms | 29.5637 Ops/s | 27.9822 Ops/s | $\textbf{\color{#35bf28}+5.65\\%}$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 9.1128ms | 8.5899ms | 116.4157 Ops/s | 119.9887 Ops/s | $\color{#d91a1a}-2.98\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.4891ms | 2.0548ms | 486.6537 Ops/s | 497.2124 Ops/s | $\color{#d91a1a}-2.12\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.6668ms | 0.3657ms | 2.7341 KOps/s | 2.7615 KOps/s | $\color{#d91a1a}-0.99\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 41.7515ms | 39.8243ms | 25.1103 Ops/s | 22.6612 Ops/s | $\textbf{\color{#35bf28}+10.81\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.7935ms | 3.0639ms | 326.3812 Ops/s | 326.6691 Ops/s | $\color{#d91a1a}-0.09\\%$ | | test_dqn_speed | 7.3325ms | 1.3980ms | 715.3202 Ops/s | 726.4519 Ops/s | $\color{#d91a1a}-1.53\\%$ | | test_ddpg_speed | 3.6341ms | 2.9578ms | 338.0840 Ops/s | 337.5007 Ops/s | $\color{#35bf28}+0.17\\%$ | | test_sac_speed | 9.9788ms | 8.9459ms | 111.7825 Ops/s | 112.4681 Ops/s | $\color{#d91a1a}-0.61\\%$ | | test_redq_speed | 0.1064s | 15.6817ms | 63.7687 Ops/s | 69.9574 Ops/s | $\textbf{\color{#d91a1a}-8.85\\%}$ | | test_redq_deprec_speed | 15.1868ms | 14.4024ms | 69.4326 Ops/s | 69.4754 Ops/s | $\color{#d91a1a}-0.06\\%$ | | test_td3_speed | 9.3269ms | 8.9090ms | 112.2460 Ops/s | 113.6981 Ops/s | $\color{#d91a1a}-1.28\\%$ | | test_cql_speed | 39.2004ms | 37.7456ms | 26.4931 Ops/s | 26.3018 Ops/s | $\color{#35bf28}+0.73\\%$ | | test_a2c_speed | 11.4164ms | 7.6469ms | 130.7725 Ops/s | 125.3227 Ops/s | $\color{#35bf28}+4.35\\%$ | | test_ppo_speed | 10.8542ms | 8.1082ms | 123.3313 Ops/s | 120.8234 Ops/s | $\color{#35bf28}+2.08\\%$ | | test_reinforce_speed | 8.1385ms | 6.8780ms | 145.3912 Ops/s | 143.9659 Ops/s | $\color{#35bf28}+0.99\\%$ | | test_iql_speed | 36.1832ms | 34.0606ms | 29.3594 Ops/s | 29.1544 Ops/s | $\color{#35bf28}+0.70\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.1646ms | 3.9081ms | 255.8817 Ops/s | 267.7935 Ops/s | $\color{#d91a1a}-4.45\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.9103ms | 0.5477ms | 1.8257 KOps/s | 1.6860 KOps/s | $\textbf{\color{#35bf28}+8.29\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7846ms | 0.4979ms | 2.0083 KOps/s | 2.0228 KOps/s | $\color{#d91a1a}-0.71\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.9850ms | 3.8646ms | 258.7567 Ops/s | 263.9201 Ops/s | $\color{#d91a1a}-1.96\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.3996ms | 0.5279ms | 1.8943 KOps/s | 1.9573 KOps/s | $\color{#d91a1a}-3.22\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 3.7477ms | 0.4943ms | 2.0232 KOps/s | 2.0647 KOps/s | $\color{#d91a1a}-2.01\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.5924ms | 1.7912ms | 558.2868 Ops/s | 570.2183 Ops/s | $\color{#d91a1a}-2.09\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 4.9939ms | 1.6958ms | 589.7002 Ops/s | 599.3400 Ops/s | $\color{#d91a1a}-1.61\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.2902ms | 3.9333ms | 254.2399 Ops/s | 253.3614 Ops/s | $\color{#35bf28}+0.35\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9968ms | 0.6603ms | 1.5145 KOps/s | 1.5258 KOps/s | $\color{#d91a1a}-0.74\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 3.9678ms | 0.6341ms | 1.5770 KOps/s | 1.5777 KOps/s | $\color{#d91a1a}-0.05\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.1863ms | 3.6906ms | 270.9611 Ops/s | 262.2280 Ops/s | $\color{#35bf28}+3.33\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.8476ms | 0.5188ms | 1.9274 KOps/s | 1.9282 KOps/s | $\color{#d91a1a}-0.04\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7518ms | 0.4899ms | 2.0414 KOps/s | 2.0319 KOps/s | $\color{#35bf28}+0.47\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.1558ms | 3.7893ms | 263.8980 Ops/s | 261.8396 Ops/s | $\color{#35bf28}+0.79\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8300ms | 0.5165ms | 1.9360 KOps/s | 1.9458 KOps/s | $\color{#d91a1a}-0.51\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6244ms | 0.4930ms | 2.0286 KOps/s | 2.0342 KOps/s | $\color{#d91a1a}-0.27\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.5264ms | 3.8822ms | 257.5843 Ops/s | 255.1886 Ops/s | $\color{#35bf28}+0.94\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.7938ms | 0.6623ms | 1.5099 KOps/s | 1.5230 KOps/s | $\color{#d91a1a}-0.86\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9148ms | 0.6470ms | 1.5456 KOps/s | 1.5703 KOps/s | $\color{#d91a1a}-1.58\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1242s | 6.2962ms | 158.8255 Ops/s | 112.3205 Ops/s | $\textbf{\color{#35bf28}+41.40\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 0.1208s | 15.3726ms | 65.0506 Ops/s | 75.9559 Ops/s | $\textbf{\color{#d91a1a}-14.36\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 1.1439ms | 1.0804ms | 925.6248 Ops/s | 915.6668 Ops/s | $\color{#35bf28}+1.09\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1179s | 6.1706ms | 162.0596 Ops/s | 158.3853 Ops/s | $\color{#35bf28}+2.32\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 15.5940ms | 13.1624ms | 75.9741 Ops/s | 76.2816 Ops/s | $\color{#d91a1a}-0.40\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.1486ms | 1.0726ms | 932.3154 Ops/s | 845.2568 Ops/s | $\textbf{\color{#35bf28}+10.30\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1159s | 6.1962ms | 161.3896 Ops/s | 154.7486 Ops/s | $\color{#35bf28}+4.29\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 15.7179ms | 13.2155ms | 75.6687 Ops/s | 63.6792 Ops/s | $\textbf{\color{#35bf28}+18.83\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.8235ms | 1.2400ms | 806.4482 Ops/s | 803.2986 Ops/s | $\color{#35bf28}+0.39\\%$ |

[github-actions[bot]]

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results

| Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1162s | 0.1159s | 8.6283 Ops/s | 8.6504 Ops/s | $\color{#d91a1a}-0.25\\%$ | | test_sync | 0.1040s | 0.1031s | 9.7000 Ops/s | 9.6458 Ops/s | $\color{#35bf28}+0.56\\%$ | | test_async | 0.1903s | 96.2964ms | 10.3846 Ops/s | 10.2067 Ops/s | $\color{#35bf28}+1.74\\%$ | | test_single_pixels | 0.1296s | 0.1277s | 7.8310 Ops/s | 7.9061 Ops/s | $\color{#d91a1a}-0.95\\%$ | | test_sync_pixels | 84.4031ms | 80.4576ms | 12.4289 Ops/s | 11.9778 Ops/s | $\color{#35bf28}+3.77\\%$ | | test_async_pixels | 0.1508s | 65.9735ms | 15.1576 Ops/s | 14.7906 Ops/s | $\color{#35bf28}+2.48\\%$ | | test_simple | 0.7960s | 0.7942s | 1.2592 Ops/s | 1.2664 Ops/s | $\color{#d91a1a}-0.57\\%$ | | test_transformed | 1.0531s | 1.0520s | 0.9505 Ops/s | 0.9527 Ops/s | $\color{#d91a1a}-0.23\\%$ | | test_serial | 2.5219s | 2.4648s | 0.4057 Ops/s | 0.4084 Ops/s | $\color{#d91a1a}-0.67\\%$ | | test_parallel | 2.4009s | 2.3430s | 0.4268 Ops/s | 0.4252 Ops/s | $\color{#35bf28}+0.38\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 77.0620μs | 35.2459μs | 28.3721 KOps/s | 28.5754 KOps/s | $\color{#d91a1a}-0.71\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 0.1621ms | 20.1297μs | 49.6778 KOps/s | 49.0567 KOps/s | $\color{#35bf28}+1.27\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 0.1754ms | 20.0243μs | 49.9392 KOps/s | 50.2129 KOps/s | $\color{#d91a1a}-0.55\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 0.1707ms | 11.6451μs | 85.8733 KOps/s | 85.7112 KOps/s | $\color{#35bf28}+0.19\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 0.2168ms | 36.9401μs | 27.0709 KOps/s | 27.3417 KOps/s | $\color{#d91a1a}-0.99\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 0.2037ms | 22.0793μs | 45.2913 KOps/s | 44.9119 KOps/s | $\color{#35bf28}+0.84\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 0.2026ms | 21.6902μs | 46.1037 KOps/s | 45.5699 KOps/s | $\color{#35bf28}+1.17\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 0.1863ms | 13.5143μs | 73.9957 KOps/s | 73.6902 KOps/s | $\color{#35bf28}+0.41\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 64.2410μs | 38.5487μs | 25.9412 KOps/s | 25.5445 KOps/s | $\color{#35bf28}+1.55\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 42.2910μs | 24.0925μs | 41.5066 KOps/s | 40.9344 KOps/s | $\color{#35bf28}+1.40\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 40.6710μs | 21.8027μs | 45.8658 KOps/s | 45.7385 KOps/s | $\color{#35bf28}+0.28\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 30.2100μs | 13.3364μs | 74.9830 KOps/s | 73.4734 KOps/s | $\color{#35bf28}+2.05\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 83.6610μs | 40.3794μs | 24.7651 KOps/s | 24.5684 KOps/s | $\color{#35bf28}+0.80\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 99.0220μs | 25.4816μs | 39.2440 KOps/s | 37.7279 KOps/s | $\color{#35bf28}+4.02\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 47.5120μs | 23.3659μs | 42.7974 KOps/s | 42.2189 KOps/s | $\color{#35bf28}+1.37\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 34.5610μs | 15.1755μs | 65.8955 KOps/s | 65.0225 KOps/s | $\color{#35bf28}+1.34\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 54.0720μs | 38.0761μs | 26.2632 KOps/s | 25.4746 KOps/s | $\color{#35bf28}+3.10\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 0.1148ms | 23.8522μs | 41.9249 KOps/s | 41.1827 KOps/s | $\color{#35bf28}+1.80\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 0.1285ms | 25.5189μs | 39.1867 KOps/s | 38.5221 KOps/s | $\color{#35bf28}+1.73\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 45.7800μs | 15.2000μs | 65.7893 KOps/s | 64.9037 KOps/s | $\color{#35bf28}+1.36\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 0.1177ms | 39.8382μs | 25.1016 KOps/s | 24.3979 KOps/s | $\color{#35bf28}+2.88\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 53.4110μs | 25.6991μs | 38.9119 KOps/s | 38.5019 KOps/s | $\color{#35bf28}+1.06\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 48.5100μs | 27.5490μs | 36.2990 KOps/s | 35.8755 KOps/s | $\color{#35bf28}+1.18\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 0.1015ms | 16.9250μs | 59.0840 KOps/s | 58.0317 KOps/s | $\color{#35bf28}+1.81\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 0.1140ms | 42.2936μs | 23.6442 KOps/s | 23.5694 KOps/s | $\color{#35bf28}+0.32\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 51.6110μs | 27.7436μs | 36.0443 KOps/s | 35.7944 KOps/s | $\color{#35bf28}+0.70\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 76.7800μs | 27.5635μs | 36.2799 KOps/s | 36.3204 KOps/s | $\color{#d91a1a}-0.11\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 0.1550ms | 17.1121μs | 58.4382 KOps/s | 58.3461 KOps/s | $\color{#35bf28}+0.16\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 64.3010μs | 44.4246μs | 22.5101 KOps/s | 22.2545 KOps/s | $\color{#35bf28}+1.15\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 54.2810μs | 29.7034μs | 33.6661 KOps/s | 33.2794 KOps/s | $\color{#35bf28}+1.16\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 77.2510μs | 29.2196μs | 34.2236 KOps/s | 34.2651 KOps/s | $\color{#d91a1a}-0.12\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 38.0300μs | 18.9911μs | 52.6564 KOps/s | 52.2816 KOps/s | $\color{#35bf28}+0.72\\%$ | | test_values[generalized_advantage_estimate-True-True] | 23.9110ms | 23.2764ms | 42.9620 Ops/s | 43.9800 Ops/s | $\color{#d91a1a}-2.31\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 87.9456ms | 2.6404ms | 378.7273 Ops/s | 357.8784 Ops/s | $\textbf{\color{#35bf28}+5.83\\%}$ | | test_values[td0_return_estimate-False-False] | 89.3010μs | 65.1555μs | 15.3479 KOps/s | 15.9962 KOps/s | $\color{#d91a1a}-4.05\\%$ | | test_values[td1_return_estimate-False-False] | 57.1885ms | 53.1867ms | 18.8017 Ops/s | 19.4755 Ops/s | $\color{#d91a1a}-3.46\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.3419ms | 1.0616ms | 941.9797 Ops/s | 951.2208 Ops/s | $\color{#d91a1a}-0.97\\%$ | | test_values[td_lambda_return_estimate-True-False] | 90.0204ms | 83.9798ms | 11.9076 Ops/s | 12.1576 Ops/s | $\color{#d91a1a}-2.06\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.4139ms | 1.0734ms | 931.6082 Ops/s | 949.5585 Ops/s | $\color{#d91a1a}-1.89\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 25.7263ms | 25.4261ms | 39.3296 Ops/s | 40.9871 Ops/s | $\color{#d91a1a}-4.04\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9489ms | 0.6924ms | 1.4443 KOps/s | 1.4590 KOps/s | $\color{#d91a1a}-1.01\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7823ms | 0.6414ms | 1.5590 KOps/s | 1.5725 KOps/s | $\color{#d91a1a}-0.86\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.7231ms | 1.4518ms | 688.8207 Ops/s | 692.5719 Ops/s | $\color{#d91a1a}-0.54\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.8585ms | 0.6931ms | 1.4429 KOps/s | 1.5224 KOps/s | $\textbf{\color{#d91a1a}-5.22\\%}$ | | test_dqn_speed | 83.5371ms | 1.6045ms | 623.2613 Ops/s | 689.1044 Ops/s | $\textbf{\color{#d91a1a}-9.55\\%}$ | | test_ddpg_speed | 3.4036ms | 2.9908ms | 334.3553 Ops/s | 311.5592 Ops/s | $\textbf{\color{#35bf28}+7.32\\%}$ | | test_sac_speed | 9.2965ms | 8.6138ms | 116.0932 Ops/s | 118.5630 Ops/s | $\color{#d91a1a}-2.08\\%$ | | test_redq_speed | 12.2312ms | 11.0671ms | 90.3576 Ops/s | 91.6221 Ops/s | $\color{#d91a1a}-1.38\\%$ | | test_redq_deprec_speed | 0.1118s | 12.7877ms | 78.1999 Ops/s | 84.9150 Ops/s | $\textbf{\color{#d91a1a}-7.91\\%}$ | | test_td3_speed | 8.6248ms | 8.4206ms | 118.7559 Ops/s | 118.4110 Ops/s | $\color{#35bf28}+0.29\\%$ | | test_cql_speed | 27.3601ms | 26.5291ms | 37.6945 Ops/s | 38.3102 Ops/s | $\color{#d91a1a}-1.61\\%$ | | test_a2c_speed | 6.3039ms | 5.7657ms | 173.4397 Ops/s | 174.5862 Ops/s | $\color{#d91a1a}-0.66\\%$ | | test_ppo_speed | 6.5534ms | 6.0786ms | 164.5105 Ops/s | 165.6751 Ops/s | $\color{#d91a1a}-0.70\\%$ | | test_reinforce_speed | 5.1483ms | 4.8221ms | 207.3775 Ops/s | 211.4184 Ops/s | $\color{#d91a1a}-1.91\\%$ | | test_iql_speed | 20.6256ms | 20.1138ms | 49.7171 Ops/s | 49.9036 Ops/s | $\color{#d91a1a}-0.37\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.0354ms | 4.6968ms | 212.9102 Ops/s | 212.8979 Ops/s | $+0.01\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.5263ms | 0.6052ms | 1.6522 KOps/s | 1.6541 KOps/s | $\color{#d91a1a}-0.11\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7457ms | 0.5834ms | 1.7140 KOps/s | 1.7186 KOps/s | $\color{#d91a1a}-0.27\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.0819ms | 4.6466ms | 215.2132 Ops/s | 216.1625 Ops/s | $\color{#d91a1a}-0.44\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.3281ms | 0.6055ms | 1.6514 KOps/s | 1.6685 KOps/s | $\color{#d91a1a}-1.03\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7996ms | 0.5878ms | 1.7013 KOps/s | 1.7379 KOps/s | $\color{#d91a1a}-2.11\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.7259ms | 2.1439ms | 466.4411 Ops/s | 475.4816 Ops/s | $\color{#d91a1a}-1.90\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.3487ms | 2.0277ms | 493.1698 Ops/s | 500.6616 Ops/s | $\color{#d91a1a}-1.50\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.1427ms | 4.8253ms | 207.2403 Ops/s | 209.3549 Ops/s | $\color{#d91a1a}-1.01\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9998ms | 0.7622ms | 1.3120 KOps/s | 1.3257 KOps/s | $\color{#d91a1a}-1.03\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.5296ms | 0.7503ms | 1.3329 KOps/s | 1.2172 KOps/s | $\textbf{\color{#35bf28}+9.50\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.9599ms | 4.6796ms | 213.6951 Ops/s | 214.3325 Ops/s | $\color{#d91a1a}-0.30\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7867ms | 0.6072ms | 1.6468 KOps/s | 1.6507 KOps/s | $\color{#d91a1a}-0.23\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7799ms | 0.5877ms | 1.7014 KOps/s | 1.7082 KOps/s | $\color{#d91a1a}-0.40\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.0624ms | 4.6327ms | 215.8565 Ops/s | 215.9328 Ops/s | $\color{#d91a1a}-0.04\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8692ms | 0.6040ms | 1.6555 KOps/s | 1.6642 KOps/s | $\color{#d91a1a}-0.52\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.1423s | 0.7904ms | 1.2652 KOps/s | 1.7211 KOps/s | $\textbf{\color{#d91a1a}-26.49\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.1881ms | 4.8388ms | 206.6610 Ops/s | 206.5434 Ops/s | $\color{#35bf28}+0.06\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9489ms | 0.7546ms | 1.3252 KOps/s | 1.3089 KOps/s | $\color{#35bf28}+1.25\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9669ms | 0.7322ms | 1.3657 KOps/s | 1.3467 KOps/s | $\color{#35bf28}+1.41\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1318s | 7.4684ms | 133.8977 Ops/s | 130.5322 Ops/s | $\color{#35bf28}+2.58\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 20.3719ms | 15.6169ms | 64.0332 Ops/s | 63.6871 Ops/s | $\color{#35bf28}+0.54\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 1.5125ms | 1.2914ms | 774.3789 Ops/s | 741.0902 Ops/s | $\color{#35bf28}+4.49\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1313s | 9.9954ms | 100.0465 Ops/s | 133.6900 Ops/s | $\textbf{\color{#d91a1a}-25.17\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 17.7909ms | 15.5105ms | 64.4725 Ops/s | 55.5838 Ops/s | $\textbf{\color{#35bf28}+15.99\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 7.6725ms | 1.4257ms | 701.3986 Ops/s | 750.9398 Ops/s | $\textbf{\color{#d91a1a}-6.60\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1291s | 7.6068ms | 131.4609 Ops/s | 130.9809 Ops/s | $\color{#35bf28}+0.37\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.2370ms | 15.7460ms | 63.5081 Ops/s | 64.1449 Ops/s | $\color{#d91a1a}-0.99\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 7.4373ms | 1.5994ms | 625.2333 Ops/s | 653.7476 Ops/s | $\color{#d91a1a}-4.36\\%$ |