[BugFix] Fix collector tests where device ordinal is needed

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2240

• :page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

:x: 6 New Failures, 2 Unrelated Failures

As of commit 7034c42080bb6b931f944badcd80d7bac4d0841a with merge base c44a52144d2f2f8e802f460046edafdea2bc24a0 ():

NEW FAILURES - The following jobs have failed:

* [Examples Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2240#26466367272) ([gh](https://github.com/pytorch/rl/actions/runs/9597397756/job/26466367272)) `curl: (22) The requested URL returned error:` * [Generate documentation / build-docs (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2240#26466367806) ([gh](https://github.com/pytorch/rl/actions/runs/9597397748/job/26466367806)) * [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2240#26466367448) ([gh](https://github.com/pytorch/rl/actions/runs/9597397757/job/26466367448)) `RuntimeError: Command docker exec -t 0127b50a34cff692cbc381850ab5af5a5772ecdd852385b172990bc9cefb9fc7 /exec failed with exit code 139` * [Unit-tests on Linux / tests-cpu (3.9) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2240#26466369919) ([gh](https://github.com/pytorch/rl/actions/runs/9597397755/job/26466369919)) `curl: (22) The requested URL returned error:` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2240#26466372375) ([gh](https://github.com/pytorch/rl/actions/runs/9597397755/job/26466372375)) `curl: (22) The requested URL returned error:` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2240#26466366174) ([gh](https://github.com/pytorch/rl/actions/runs/9597397745/job/26466366174)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2240#26466375490) ([gh](https://github.com/pytorch/rl/actions/runs/9597397760/job/26466375490)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/c44a52144d2f2f8e802f460046edafdea2bc24a0#26430824129)) `##[error]The operation was canceled.` * [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2240#26466371945) ([gh](https://github.com/pytorch/rl/actions/runs/9597397755/job/26466371945)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/c44a52144d2f2f8e802f460046edafdea2bc24a0#26430822586)) `##[error]The operation was canceled.`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions[bot]]

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results

| Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1136s | 58.7639ms | 17.0173 Ops/s | 17.6354 Ops/s | $\color{#d91a1a}-3.50\\%$ | | test_sync | 41.2208ms | 30.2709ms | 33.0351 Ops/s | 31.6801 Ops/s | $\color{#35bf28}+4.28\\%$ | | test_async | 56.1920ms | 29.4072ms | 34.0053 Ops/s | 34.8747 Ops/s | $\color{#d91a1a}-2.49\\%$ | | test_simple | 0.3734s | 0.3720s | 2.6881 Ops/s | 2.6571 Ops/s | $\color{#35bf28}+1.16\\%$ | | test_transformed | 0.5319s | 0.5303s | 1.8856 Ops/s | 1.8719 Ops/s | $\color{#35bf28}+0.73\\%$ | | test_serial | 1.3276s | 1.2638s | 0.7913 Ops/s | 0.7890 Ops/s | $\color{#35bf28}+0.28\\%$ | | test_parallel | 1.1345s | 1.0872s | 0.9198 Ops/s | 0.9230 Ops/s | $\color{#d91a1a}-0.35\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1368ms | 22.3723μs | 44.6982 KOps/s | 43.8459 KOps/s | $\color{#35bf28}+1.94\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 48.8110μs | 13.1025μs | 76.3214 KOps/s | 75.2389 KOps/s | $\color{#35bf28}+1.44\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 50.8450μs | 12.8825μs | 77.6247 KOps/s | 75.3949 KOps/s | $\color{#35bf28}+2.96\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 35.3760μs | 7.5586μs | 132.2998 KOps/s | 129.3877 KOps/s | $\color{#35bf28}+2.25\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 60.3820μs | 23.3724μs | 42.7856 KOps/s | 41.2713 KOps/s | $\color{#35bf28}+3.67\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 39.1830μs | 14.3542μs | 69.6659 KOps/s | 69.6668 KOps/s | $-0.00\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 44.2220μs | 14.1758μs | 70.5430 KOps/s | 68.4607 KOps/s | $\color{#35bf28}+3.04\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 29.0950μs | 9.0142μs | 110.9361 KOps/s | 111.2497 KOps/s | $\color{#d91a1a}-0.28\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 62.3870μs | 25.0326μs | 39.9478 KOps/s | 39.2646 KOps/s | $\color{#35bf28}+1.74\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 47.6920μs | 15.7168μs | 63.6263 KOps/s | 62.8389 KOps/s | $\color{#35bf28}+1.25\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 39.9740μs | 14.2553μs | 70.1494 KOps/s | 68.3455 KOps/s | $\color{#35bf28}+2.64\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 45.0140μs | 8.9454μs | 111.7894 KOps/s | 111.4029 KOps/s | $\color{#35bf28}+0.35\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 64.0800μs | 26.0010μs | 38.4601 KOps/s | 37.3572 KOps/s | $\color{#35bf28}+2.95\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 60.0110μs | 16.9121μs | 59.1294 KOps/s | 58.9620 KOps/s | $\color{#35bf28}+0.28\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 56.9260μs | 15.3088μs | 65.3219 KOps/s | 62.8998 KOps/s | $\color{#35bf28}+3.85\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 42.4280μs | 10.1137μs | 98.8758 KOps/s | 97.7142 KOps/s | $\color{#35bf28}+1.19\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 68.4070μs | 24.8429μs | 40.2529 KOps/s | 38.9846 KOps/s | $\color{#35bf28}+3.25\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 40.3650μs | 15.6541μs | 63.8811 KOps/s | 62.7576 KOps/s | $\color{#35bf28}+1.79\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 62.3960μs | 16.5233μs | 60.5206 KOps/s | 58.6635 KOps/s | $\color{#35bf28}+3.17\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 34.1340μs | 10.0446μs | 99.5564 KOps/s | 95.8529 KOps/s | $\color{#35bf28}+3.86\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 69.1290μs | 26.0030μs | 38.4571 KOps/s | 37.6374 KOps/s | $\color{#35bf28}+2.18\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 62.8870μs | 16.9235μs | 59.0896 KOps/s | 57.9690 KOps/s | $\color{#35bf28}+1.93\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 57.5670μs | 17.7528μs | 56.3292 KOps/s | 54.6298 KOps/s | $\color{#35bf28}+3.11\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 48.6710μs | 11.3331μs | 88.2370 KOps/s | 86.9028 KOps/s | $\color{#35bf28}+1.54\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 58.3790μs | 27.4563μs | 36.4215 KOps/s | 35.5236 KOps/s | $\color{#35bf28}+2.53\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 61.1240μs | 18.1312μs | 55.1537 KOps/s | 54.4338 KOps/s | $\color{#35bf28}+1.32\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 60.4120μs | 17.5778μs | 56.8898 KOps/s | 54.5662 KOps/s | $\color{#35bf28}+4.26\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 37.6500μs | 11.2905μs | 88.5699 KOps/s | 86.8383 KOps/s | $\color{#35bf28}+1.99\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 54.9620μs | 29.1661μs | 34.2863 KOps/s | 33.8746 KOps/s | $\color{#35bf28}+1.22\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 58.8200μs | 19.2616μs | 51.9169 KOps/s | 51.0141 KOps/s | $\color{#35bf28}+1.77\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 47.4380μs | 18.6717μs | 53.5569 KOps/s | 52.0641 KOps/s | $\color{#35bf28}+2.87\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 56.1340μs | 12.3832μs | 80.7548 KOps/s | 79.1945 KOps/s | $\color{#35bf28}+1.97\\%$ | | test_values[generalized_advantage_estimate-True-True] | 12.8674ms | 9.3772ms | 106.6418 Ops/s | 103.3385 Ops/s | $\color{#35bf28}+3.20\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 43.4680ms | 36.5552ms | 27.3559 Ops/s | 28.0360 Ops/s | $\color{#d91a1a}-2.43\\%$ | | test_values[td0_return_estimate-False-False] | 0.2539ms | 0.1629ms | 6.1373 KOps/s | 5.9481 KOps/s | $\color{#35bf28}+3.18\\%$ | | test_values[td1_return_estimate-False-False] | 23.6970ms | 23.3449ms | 42.8358 Ops/s | 42.6868 Ops/s | $\color{#35bf28}+0.35\\%$ | | test_values[vec_td1_return_estimate-False-False] | 50.5122ms | 36.5396ms | 27.3676 Ops/s | 27.6312 Ops/s | $\color{#d91a1a}-0.95\\%$ | | test_values[td_lambda_return_estimate-True-False] | 37.3220ms | 33.7987ms | 29.5870 Ops/s | 29.8850 Ops/s | $\color{#d91a1a}-1.00\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 40.0997ms | 37.2427ms | 26.8509 Ops/s | 27.5456 Ops/s | $\color{#d91a1a}-2.52\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.4403ms | 8.2291ms | 121.5194 Ops/s | 121.2796 Ops/s | $\color{#35bf28}+0.20\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.6263ms | 1.9492ms | 513.0330 Ops/s | 537.6175 Ops/s | $\color{#d91a1a}-4.57\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.5371ms | 0.3684ms | 2.7148 KOps/s | 2.7899 KOps/s | $\color{#d91a1a}-2.69\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 57.2985ms | 50.1030ms | 19.9589 Ops/s | 22.1171 Ops/s | $\textbf{\color{#d91a1a}-9.76\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.8084ms | 3.0144ms | 331.7400 Ops/s | 331.0594 Ops/s | $\color{#35bf28}+0.21\\%$ | | test_dqn_speed | 7.9402ms | 1.3869ms | 721.0393 Ops/s | 717.9293 Ops/s | $\color{#35bf28}+0.43\\%$ | | test_ddpg_speed | 3.6882ms | 2.9008ms | 344.7372 Ops/s | 345.0463 Ops/s | $\color{#d91a1a}-0.09\\%$ | | test_sac_speed | 9.7719ms | 8.6843ms | 115.1502 Ops/s | 115.2164 Ops/s | $\color{#d91a1a}-0.06\\%$ | | test_redq_speed | 14.9767ms | 13.5464ms | 73.8204 Ops/s | 69.9056 Ops/s | $\textbf{\color{#35bf28}+5.60\\%}$ | | test_redq_deprec_speed | 0.1180s | 15.2579ms | 65.5400 Ops/s | 69.2509 Ops/s | $\textbf{\color{#d91a1a}-5.36\\%}$ | | test_td3_speed | 8.8915ms | 8.5461ms | 117.0127 Ops/s | 116.1488 Ops/s | $\color{#35bf28}+0.74\\%$ | | test_cql_speed | 39.5613ms | 38.1070ms | 26.2419 Ops/s | 27.1692 Ops/s | $\color{#d91a1a}-3.41\\%$ | | test_a2c_speed | 8.8253ms | 7.4817ms | 133.6601 Ops/s | 134.6352 Ops/s | $\color{#d91a1a}-0.72\\%$ | | test_ppo_speed | 8.2096ms | 7.7271ms | 129.4141 Ops/s | 130.4619 Ops/s | $\color{#d91a1a}-0.80\\%$ | | test_reinforce_speed | 7.8899ms | 6.6667ms | 149.9985 Ops/s | 150.9690 Ops/s | $\color{#d91a1a}-0.64\\%$ | | test_iql_speed | 36.1666ms | 33.7544ms | 29.6258 Ops/s | 30.4113 Ops/s | $\color{#d91a1a}-2.58\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.0547ms | 3.4373ms | 290.9253 Ops/s | 295.6651 Ops/s | $\color{#d91a1a}-1.60\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7933ms | 0.4935ms | 2.0262 KOps/s | 1.9906 KOps/s | $\color{#35bf28}+1.79\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.9225ms | 0.4726ms | 2.1157 KOps/s | 2.0927 KOps/s | $\color{#35bf28}+1.10\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.7188ms | 3.3441ms | 299.0376 Ops/s | 296.6384 Ops/s | $\color{#35bf28}+0.81\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8927ms | 0.4887ms | 2.0461 KOps/s | 2.0155 KOps/s | $\color{#35bf28}+1.52\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.8795ms | 0.4670ms | 2.1415 KOps/s | 2.1241 KOps/s | $\color{#35bf28}+0.82\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 3.0428ms | 1.7396ms | 574.8474 Ops/s | 568.2382 Ops/s | $\color{#35bf28}+1.16\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.6534ms | 1.6403ms | 609.6539 Ops/s | 605.2080 Ops/s | $\color{#35bf28}+0.73\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.9054ms | 3.4932ms | 286.2734 Ops/s | 276.2547 Ops/s | $\color{#35bf28}+3.63\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.1531ms | 0.6104ms | 1.6383 KOps/s | 1.6179 KOps/s | $\color{#35bf28}+1.27\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8670ms | 0.5861ms | 1.7063 KOps/s | 1.6911 KOps/s | $\color{#35bf28}+0.90\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.2124ms | 3.4345ms | 291.1614 Ops/s | 284.3044 Ops/s | $\color{#35bf28}+2.41\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6240ms | 0.4943ms | 2.0231 KOps/s | 1.9720 KOps/s | $\color{#35bf28}+2.59\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.6806ms | 0.4828ms | 2.0714 KOps/s | 2.0397 KOps/s | $\color{#35bf28}+1.56\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.8990ms | 3.4036ms | 293.8042 Ops/s | 278.5074 Ops/s | $\textbf{\color{#35bf28}+5.49\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.5969ms | 0.4891ms | 2.0444 KOps/s | 1.9820 KOps/s | $\color{#35bf28}+3.15\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7928ms | 0.4704ms | 2.1257 KOps/s | 2.0863 KOps/s | $\color{#35bf28}+1.89\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.6103ms | 3.4947ms | 286.1508 Ops/s | 272.0073 Ops/s | $\textbf{\color{#35bf28}+5.20\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.0953ms | 0.6143ms | 1.6279 KOps/s | 1.6031 KOps/s | $\color{#35bf28}+1.54\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7975ms | 0.5834ms | 1.7140 KOps/s | 1.6721 KOps/s | $\color{#35bf28}+2.51\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1275s | 6.1973ms | 161.3614 Ops/s | 159.8362 Ops/s | $\color{#35bf28}+0.95\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 0.1198s | 14.8443ms | 67.3658 Ops/s | 64.0232 Ops/s | $\textbf{\color{#35bf28}+5.22\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 3.9348ms | 1.3154ms | 760.2342 Ops/s | 867.4278 Ops/s | $\textbf{\color{#d91a1a}-12.36\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1152s | 5.9936ms | 166.8448 Ops/s | 157.5046 Ops/s | $\textbf{\color{#35bf28}+5.93\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 14.6115ms | 12.7372ms | 78.5104 Ops/s | 78.0422 Ops/s | $\color{#35bf28}+0.60\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 6.3465ms | 1.2976ms | 770.6612 Ops/s | 859.4881 Ops/s | $\textbf{\color{#d91a1a}-10.33\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1082s | 6.0070ms | 166.4715 Ops/s | 158.7445 Ops/s | $\color{#35bf28}+4.87\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 15.2753ms | 12.9417ms | 77.2697 Ops/s | 77.4953 Ops/s | $\color{#d91a1a}-0.29\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.9992ms | 1.3018ms | 768.1422 Ops/s | 759.0989 Ops/s | $\color{#35bf28}+1.19\\%$ |

[github-actions[bot]]

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results

| Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1240s | 0.1225s | 8.1637 Ops/s | 8.1556 Ops/s | $\color{#35bf28}+0.10\\%$ | | test_sync | 0.1072s | 0.1059s | 9.4414 Ops/s | 9.8502 Ops/s | $\color{#d91a1a}-4.15\\%$ | | test_async | 0.2007s | 0.1010s | 9.9019 Ops/s | 12.1813 Ops/s | $\textbf{\color{#d91a1a}-18.71\\%}$ | | test_single_pixels | 0.1312s | 0.1306s | 7.6542 Ops/s | 7.5398 Ops/s | $\color{#35bf28}+1.52\\%$ | | test_sync_pixels | 85.7188ms | 84.6487ms | 11.8135 Ops/s | 11.6648 Ops/s | $\color{#35bf28}+1.27\\%$ | | test_async_pixels | 0.1679s | 82.9701ms | 12.0525 Ops/s | 14.1960 Ops/s | $\textbf{\color{#d91a1a}-15.10\\%}$ | | test_simple | 0.8320s | 0.8311s | 1.2032 Ops/s | 1.1517 Ops/s | $\color{#35bf28}+4.48\\%$ | | test_transformed | 1.0917s | 1.0912s | 0.9165 Ops/s | 0.8995 Ops/s | $\color{#35bf28}+1.88\\%$ | | test_serial | 2.6305s | 2.5666s | 0.3896 Ops/s | 0.3807 Ops/s | $\color{#35bf28}+2.34\\%$ | | test_parallel | 2.4530s | 2.3912s | 0.4182 Ops/s | 0.4088 Ops/s | $\color{#35bf28}+2.31\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 97.8820μs | 33.6784μs | 29.6926 KOps/s | 28.7005 KOps/s | $\color{#35bf28}+3.46\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 41.5010μs | 19.5279μs | 51.2089 KOps/s | 50.0754 KOps/s | $\color{#35bf28}+2.26\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 36.2700μs | 19.3985μs | 51.5503 KOps/s | 51.3204 KOps/s | $\color{#35bf28}+0.45\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 32.6900μs | 11.1277μs | 89.8657 KOps/s | 87.8755 KOps/s | $\color{#35bf28}+2.26\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 64.8610μs | 35.3054μs | 28.3243 KOps/s | 27.8947 KOps/s | $\color{#35bf28}+1.54\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 48.8410μs | 20.9859μs | 47.6509 KOps/s | 46.4441 KOps/s | $\color{#35bf28}+2.60\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 43.2110μs | 21.0063μs | 47.6047 KOps/s | 47.0414 KOps/s | $\color{#35bf28}+1.20\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 35.6100μs | 12.9871μs | 76.9996 KOps/s | 75.9811 KOps/s | $\color{#35bf28}+1.34\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 64.0310μs | 37.2726μs | 26.8294 KOps/s | 26.2380 KOps/s | $\color{#35bf28}+2.25\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 45.9810μs | 23.3475μs | 42.8311 KOps/s | 42.2302 KOps/s | $\color{#35bf28}+1.42\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 43.1610μs | 20.9624μs | 47.7044 KOps/s | 47.0082 KOps/s | $\color{#35bf28}+1.48\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 30.0400μs | 12.9181μs | 77.4110 KOps/s | 75.4511 KOps/s | $\color{#35bf28}+2.60\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 87.7330μs | 38.9488μs | 25.6747 KOps/s | 25.2927 KOps/s | $\color{#35bf28}+1.51\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 41.7210μs | 24.6017μs | 40.6475 KOps/s | 39.5963 KOps/s | $\color{#35bf28}+2.65\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 46.8600μs | 22.4098μs | 44.6234 KOps/s | 43.6414 KOps/s | $\color{#35bf28}+2.25\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 37.7010μs | 14.8982μs | 67.1224 KOps/s | 66.5665 KOps/s | $\color{#35bf28}+0.84\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 63.9110μs | 37.5752μs | 26.6133 KOps/s | 26.0626 KOps/s | $\color{#35bf28}+2.11\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 48.5210μs | 23.2542μs | 43.0030 KOps/s | 41.9621 KOps/s | $\color{#35bf28}+2.48\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 48.6510μs | 25.1857μs | 39.7051 KOps/s | 38.8307 KOps/s | $\color{#35bf28}+2.25\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 33.0510μs | 14.9218μs | 67.0161 KOps/s | 66.0092 KOps/s | $\color{#35bf28}+1.53\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 62.6910μs | 38.6272μs | 25.8885 KOps/s | 25.0388 KOps/s | $\color{#35bf28}+3.39\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 43.4310μs | 25.0095μs | 39.9848 KOps/s | 39.3174 KOps/s | $\color{#35bf28}+1.70\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 54.6310μs | 26.7129μs | 37.4351 KOps/s | 37.0161 KOps/s | $\color{#35bf28}+1.13\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 35.6610μs | 16.6909μs | 59.9131 KOps/s | 58.7192 KOps/s | $\color{#35bf28}+2.03\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 93.2410μs | 39.6234μs | 25.2376 KOps/s | 23.7737 KOps/s | $\textbf{\color{#35bf28}+6.16\\%}$ | | test_step_mdp_speed[False-False-True-True-False] | 49.5610μs | 26.1867μs | 38.1873 KOps/s | 36.4620 KOps/s | $\color{#35bf28}+4.73\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 65.4410μs | 26.6488μs | 37.5252 KOps/s | 37.2371 KOps/s | $\color{#35bf28}+0.77\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 42.1910μs | 16.1837μs | 61.7906 KOps/s | 59.0157 KOps/s | $\color{#35bf28}+4.70\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 56.5310μs | 43.3247μs | 23.0815 KOps/s | 22.8249 KOps/s | $\color{#35bf28}+1.12\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 55.6010μs | 28.5027μs | 35.0844 KOps/s | 34.1978 KOps/s | $\color{#35bf28}+2.59\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 59.2210μs | 27.7392μs | 36.0500 KOps/s | 34.9159 KOps/s | $\color{#35bf28}+3.25\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 49.3910μs | 18.1019μs | 55.2428 KOps/s | 53.4529 KOps/s | $\color{#35bf28}+3.35\\%$ | | test_values[generalized_advantage_estimate-True-True] | 27.6952ms | 27.0637ms | 36.9499 Ops/s | 36.2383 Ops/s | $\color{#35bf28}+1.96\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 96.3814ms | 2.8703ms | 348.3939 Ops/s | 359.8543 Ops/s | $\color{#d91a1a}-3.18\\%$ | | test_values[td0_return_estimate-False-False] | 95.0810μs | 70.6381μs | 14.1567 KOps/s | 13.7052 KOps/s | $\color{#35bf28}+3.29\\%$ | | test_values[td1_return_estimate-False-False] | 61.8628ms | 60.7270ms | 16.4671 Ops/s | 16.3872 Ops/s | $\color{#35bf28}+0.49\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.5035ms | 1.1531ms | 867.2301 Ops/s | 865.0978 Ops/s | $\color{#35bf28}+0.25\\%$ | | test_values[td_lambda_return_estimate-True-False] | 96.5725ms | 95.7765ms | 10.4410 Ops/s | 10.3272 Ops/s | $\color{#35bf28}+1.10\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.5333ms | 1.1484ms | 870.7891 Ops/s | 867.7401 Ops/s | $\color{#35bf28}+0.35\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 28.0075ms | 27.6443ms | 36.1739 Ops/s | 35.7993 Ops/s | $\color{#35bf28}+1.05\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 1.0256ms | 0.7823ms | 1.2783 KOps/s | 1.2574 KOps/s | $\color{#35bf28}+1.66\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.8500ms | 0.7291ms | 1.3716 KOps/s | 1.3546 KOps/s | $\color{#35bf28}+1.26\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5719ms | 1.5196ms | 658.0853 Ops/s | 652.2983 Ops/s | $\color{#35bf28}+0.89\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.8027ms | 0.7448ms | 1.3426 KOps/s | 1.3269 KOps/s | $\color{#35bf28}+1.18\\%$ | | test_dqn_speed | 80.0705ms | 1.6480ms | 606.8097 Ops/s | 651.8352 Ops/s | $\textbf{\color{#d91a1a}-6.91\\%}$ | | test_ddpg_speed | 3.3197ms | 3.0984ms | 322.7515 Ops/s | 321.6452 Ops/s | $\color{#35bf28}+0.34\\%$ | | test_sac_speed | 9.3849ms | 8.8157ms | 113.4338 Ops/s | 112.0658 Ops/s | $\color{#35bf28}+1.22\\%$ | | test_redq_speed | 11.4281ms | 10.7868ms | 92.7058 Ops/s | 91.1555 Ops/s | $\color{#35bf28}+1.70\\%$ | | test_redq_deprec_speed | 12.7409ms | 12.1030ms | 82.6243 Ops/s | 81.4261 Ops/s | $\color{#35bf28}+1.47\\%$ | | test_td3_speed | 18.4325ms | 8.8333ms | 113.2086 Ops/s | 110.6784 Ops/s | $\color{#35bf28}+2.29\\%$ | | test_cql_speed | 29.1640ms | 26.3272ms | 37.9835 Ops/s | 37.3829 Ops/s | $\color{#35bf28}+1.61\\%$ | | test_a2c_speed | 6.0042ms | 5.7796ms | 173.0221 Ops/s | 170.3101 Ops/s | $\color{#35bf28}+1.59\\%$ | | test_ppo_speed | 6.3377ms | 6.1362ms | 162.9671 Ops/s | 161.1617 Ops/s | $\color{#35bf28}+1.12\\%$ | | test_reinforce_speed | 5.5708ms | 4.7246ms | 211.6594 Ops/s | 210.8126 Ops/s | $\color{#35bf28}+0.40\\%$ | | test_iql_speed | 20.5368ms | 20.0779ms | 49.8061 Ops/s | 49.5972 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.0166ms | 4.8136ms | 207.7431 Ops/s | 204.8901 Ops/s | $\color{#35bf28}+1.39\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.3251ms | 0.5979ms | 1.6724 KOps/s | 1.6559 KOps/s | $\color{#35bf28}+1.00\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7081ms | 0.5695ms | 1.7560 KOps/s | 1.7306 KOps/s | $\color{#35bf28}+1.47\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.1536ms | 4.7917ms | 208.6925 Ops/s | 206.9948 Ops/s | $\color{#35bf28}+0.82\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.3155ms | 0.5925ms | 1.6878 KOps/s | 1.6764 KOps/s | $\color{#35bf28}+0.68\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6929ms | 0.5627ms | 1.7773 KOps/s | 1.7578 KOps/s | $\color{#35bf28}+1.11\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.7291ms | 2.1997ms | 454.6124 Ops/s | 447.5850 Ops/s | $\color{#35bf28}+1.57\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.3137ms | 2.0862ms | 479.3431 Ops/s | 470.2571 Ops/s | $\color{#35bf28}+1.93\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0539ms | 4.9379ms | 202.5155 Ops/s | 200.7207 Ops/s | $\color{#35bf28}+0.89\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8516ms | 0.7214ms | 1.3862 KOps/s | 1.3599 KOps/s | $\color{#35bf28}+1.93\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 5.8531ms | 0.7073ms | 1.4138 KOps/s | 1.4110 KOps/s | $\color{#35bf28}+0.20\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.0088ms | 4.8114ms | 207.8387 Ops/s | 205.6990 Ops/s | $\color{#35bf28}+1.04\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7267ms | 0.5994ms | 1.6685 KOps/s | 1.6506 KOps/s | $\color{#35bf28}+1.08\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6781ms | 0.5683ms | 1.7597 KOps/s | 1.7107 KOps/s | $\color{#35bf28}+2.87\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.0490ms | 4.7887ms | 208.8228 Ops/s | 205.6697 Ops/s | $\color{#35bf28}+1.53\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.2952ms | 0.5896ms | 1.6961 KOps/s | 1.6626 KOps/s | $\color{#35bf28}+2.02\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6877ms | 0.5641ms | 1.7726 KOps/s | 1.7481 KOps/s | $\color{#35bf28}+1.41\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0693ms | 4.9674ms | 201.3129 Ops/s | 199.3542 Ops/s | $\color{#35bf28}+0.98\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8348ms | 0.7247ms | 1.3799 KOps/s | 1.3526 KOps/s | $\color{#35bf28}+2.02\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8153ms | 0.6977ms | 1.4334 KOps/s | 1.3969 KOps/s | $\color{#35bf28}+2.61\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1279s | 8.0368ms | 124.4282 Ops/s | 123.8779 Ops/s | $\color{#35bf28}+0.44\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 19.6082ms | 17.0036ms | 58.8109 Ops/s | 57.5955 Ops/s | $\color{#35bf28}+2.11\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.3903ms | 1.2971ms | 770.9697 Ops/s | 740.1477 Ops/s | $\color{#35bf28}+4.16\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1255s | 8.0113ms | 124.8232 Ops/s | 123.5421 Ops/s | $\color{#35bf28}+1.04\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1388s | 19.5600ms | 51.1248 Ops/s | 57.4140 Ops/s | $\textbf{\color{#d91a1a}-10.95\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.3613ms | 1.2996ms | 769.4946 Ops/s | 731.8581 Ops/s | $\textbf{\color{#35bf28}+5.14\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1247s | 8.1403ms | 122.8459 Ops/s | 93.9615 Ops/s | $\textbf{\color{#35bf28}+30.74\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 19.5386ms | 17.1172ms | 58.4208 Ops/s | 56.9039 Ops/s | $\color{#35bf28}+2.67\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 7.5793ms | 1.5850ms | 630.9038 Ops/s | 665.6926 Ops/s | $\textbf{\color{#d91a1a}-5.23\\%}$ |