search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.01k stars 269 forks source link

[BugFix] Fix non-tensor passage in _StepMDP #2260

Closed vmoens closed 1 week ago

pytorch-bot[bot] commented 1 week ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2260

Note: Links to docs will display an error until the docs builds have been completed.

:x: 5 New Failures

As of commit 5a854e90042a03c36a3a108683a929c82b485c9f with merge base 39462f011d3fe2b7bba1e4c9acc9648b68bc6ff5 (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2260#26929568480) ([gh](https://github.com/pytorch/rl/actions/runs/9757339789/job/26929568480)) `RuntimeError: Command docker exec -t 41c48d3a5e82f69ceda8ddc26ea793e2fab165f7853faac056c444e2cf5f3802 /exec failed with exit code 139` * [Lint / python-source-and-configs / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2260#26929548175) ([gh](https://github.com/pytorch/rl/actions/runs/9757339781/job/26929548175)) `RuntimeError: Command docker exec -t 5de872f06b6922a49cec1377fa6f3efcfc9b7de06ac7d01ecc9d57c90c9cc0ce /exec failed with exit code 1` * [RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2260#26929546902) ([gh](https://github.com/pytorch/rl/actions/runs/9757339804/job/26929546902)) `RuntimeError: Command docker exec -t 638b1659bf95c770d29d7cf4bb9ed07ad52bce98de781327039be45259770c30 /exec failed with exit code 1` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2260#26929569044) ([gh](https://github.com/pytorch/rl/actions/runs/9757339788/job/26929569044)) `RuntimeError: Command docker exec -t 93ca15d47e48507093da8efd769987cd19aad2ca430b886635d3730d3b0edfeb /exec failed with exit code 1` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2260#26929523314) ([gh](https://github.com/pytorch/rl/actions/runs/9757339793/job/26929523314)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 week ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1214s | 59.9803ms | 16.6722 Ops/s | 18.0891 Ops/s | $\textbf{\color{#d91a1a}-7.83\\%}$ | | test_sync | 32.6320ms | 30.6895ms | 32.5844 Ops/s | 32.4949 Ops/s | $\color{#35bf28}+0.28\\%$ | | test_async | 54.0017ms | 27.9359ms | 35.7962 Ops/s | 34.8407 Ops/s | $\color{#35bf28}+2.74\\%$ | | test_simple | 0.3892s | 0.3824s | 2.6153 Ops/s | 2.6525 Ops/s | $\color{#d91a1a}-1.40\\%$ | | test_transformed | 0.5404s | 0.5388s | 1.8558 Ops/s | 1.8347 Ops/s | $\color{#35bf28}+1.15\\%$ | | test_serial | 1.3237s | 1.2712s | 0.7867 Ops/s | 0.7860 Ops/s | $\color{#35bf28}+0.09\\%$ | | test_parallel | 1.1541s | 1.1066s | 0.9037 Ops/s | 0.9214 Ops/s | $\color{#d91a1a}-1.93\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 60.3330μs | 22.0963μs | 45.2564 KOps/s | 44.8717 KOps/s | $\color{#35bf28}+0.86\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 41.7680μs | 13.1203μs | 76.2180 KOps/s | 75.9383 KOps/s | $\color{#35bf28}+0.37\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 49.4620μs | 12.8639μs | 77.7368 KOps/s | 77.4251 KOps/s | $\color{#35bf28}+0.40\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 30.7070μs | 7.5518μs | 132.4182 KOps/s | 130.7798 KOps/s | $\color{#35bf28}+1.25\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 55.2130μs | 23.5311μs | 42.4970 KOps/s | 42.1076 KOps/s | $\color{#35bf28}+0.92\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 64.5620μs | 14.2223μs | 70.3119 KOps/s | 69.2173 KOps/s | $\color{#35bf28}+1.58\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 42.0080μs | 13.9947μs | 71.4557 KOps/s | 70.5120 KOps/s | $\color{#35bf28}+1.34\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 39.0420μs | 8.7589μs | 114.1694 KOps/s | 112.4135 KOps/s | $\color{#35bf28}+1.56\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 57.2370μs | 24.8765μs | 40.1985 KOps/s | 39.7449 KOps/s | $\color{#35bf28}+1.14\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 40.0650μs | 15.6524μs | 63.8880 KOps/s | 63.8150 KOps/s | $\color{#35bf28}+0.11\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 37.0290μs | 14.0251μs | 71.3009 KOps/s | 70.2692 KOps/s | $\color{#35bf28}+1.47\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 46.6970μs | 8.7474μs | 114.3201 KOps/s | 112.4051 KOps/s | $\color{#35bf28}+1.70\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 55.5840μs | 26.2003μs | 38.1676 KOps/s | 38.4390 KOps/s | $\color{#d91a1a}-0.71\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 44.2030μs | 16.7768μs | 59.6061 KOps/s | 59.1231 KOps/s | $\color{#35bf28}+0.82\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 48.7510μs | 15.1787μs | 65.8817 KOps/s | 65.0159 KOps/s | $\color{#35bf28}+1.33\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 50.9970μs | 9.8769μs | 101.2467 KOps/s | 98.3544 KOps/s | $\color{#35bf28}+2.94\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 54.2010μs | 24.9330μs | 40.1076 KOps/s | 39.7682 KOps/s | $\color{#35bf28}+0.85\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 58.4180μs | 15.6895μs | 63.7369 KOps/s | 63.8872 KOps/s | $\color{#d91a1a}-0.24\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 45.7860μs | 16.5976μs | 60.2498 KOps/s | 60.1403 KOps/s | $\color{#35bf28}+0.18\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 40.1150μs | 9.9953μs | 100.0466 KOps/s | 97.6634 KOps/s | $\color{#35bf28}+2.44\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 56.0150μs | 26.0533μs | 38.3828 KOps/s | 38.2578 KOps/s | $\color{#35bf28}+0.33\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 53.3690μs | 16.7255μs | 59.7891 KOps/s | 59.1580 KOps/s | $\color{#35bf28}+1.07\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 51.4560μs | 17.4499μs | 57.3071 KOps/s | 56.3569 KOps/s | $\color{#35bf28}+1.69\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 45.7660μs | 11.1764μs | 89.4746 KOps/s | 88.0011 KOps/s | $\color{#35bf28}+1.67\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 54.8220μs | 27.4059μs | 36.4885 KOps/s | 36.7097 KOps/s | $\color{#d91a1a}-0.60\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 62.0970μs | 17.9645μs | 55.6652 KOps/s | 54.6050 KOps/s | $\color{#35bf28}+1.94\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 45.7350μs | 17.5511μs | 56.9766 KOps/s | 57.0550 KOps/s | $\color{#d91a1a}-0.14\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 56.8260μs | 11.1981μs | 89.3006 KOps/s | 88.5188 KOps/s | $\color{#35bf28}+0.88\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 40.3250μs | 28.7893μs | 34.7351 KOps/s | 34.7942 KOps/s | $\color{#d91a1a}-0.17\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 45.7550μs | 19.2510μs | 51.9455 KOps/s | 52.1949 KOps/s | $\color{#d91a1a}-0.48\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 42.3790μs | 18.6471μs | 53.6276 KOps/s | 53.7471 KOps/s | $\color{#d91a1a}-0.22\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 48.2600μs | 12.3476μs | 80.9872 KOps/s | 80.4022 KOps/s | $\color{#35bf28}+0.73\\%$ | | test_values[generalized_advantage_estimate-True-True] | 9.6513ms | 9.3785ms | 106.6269 Ops/s | 105.9874 Ops/s | $\color{#35bf28}+0.60\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 36.4112ms | 33.7890ms | 29.5954 Ops/s | 27.9500 Ops/s | $\textbf{\color{#35bf28}+5.89\\%}$ | | test_values[td0_return_estimate-False-False] | 0.2407ms | 0.1659ms | 6.0286 KOps/s | 6.0565 KOps/s | $\color{#d91a1a}-0.46\\%$ | | test_values[td1_return_estimate-False-False] | 26.4337ms | 23.4392ms | 42.6635 Ops/s | 41.8854 Ops/s | $\color{#35bf28}+1.86\\%$ | | test_values[vec_td1_return_estimate-False-False] | 35.7413ms | 33.9207ms | 29.4805 Ops/s | 27.9159 Ops/s | $\textbf{\color{#35bf28}+5.60\\%}$ | | test_values[td_lambda_return_estimate-True-False] | 38.0324ms | 33.7431ms | 29.6357 Ops/s | 28.1884 Ops/s | $\textbf{\color{#35bf28}+5.13\\%}$ | | test_values[vec_td_lambda_return_estimate-True-False] | 35.1651ms | 33.8121ms | 29.5752 Ops/s | 27.7225 Ops/s | $\textbf{\color{#35bf28}+6.68\\%}$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.5285ms | 8.3291ms | 120.0613 Ops/s | 118.2549 Ops/s | $\color{#35bf28}+1.53\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.3539ms | 1.9821ms | 504.5157 Ops/s | 558.0242 Ops/s | $\textbf{\color{#d91a1a}-9.59\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4564ms | 0.3638ms | 2.7488 KOps/s | 2.7475 KOps/s | $\color{#35bf28}+0.05\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 45.6141ms | 44.0478ms | 22.7026 Ops/s | 20.7857 Ops/s | $\textbf{\color{#35bf28}+9.22\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.7003ms | 3.0368ms | 329.2903 Ops/s | 330.6623 Ops/s | $\color{#d91a1a}-0.41\\%$ | | test_dqn_speed | 1.8660ms | 1.4001ms | 714.2215 Ops/s | 738.3617 Ops/s | $\color{#d91a1a}-3.27\\%$ | | test_ddpg_speed | 3.6370ms | 2.9161ms | 342.9182 Ops/s | 351.4902 Ops/s | $\color{#d91a1a}-2.44\\%$ | | test_sac_speed | 10.5058ms | 8.8447ms | 113.0616 Ops/s | 116.0347 Ops/s | $\color{#d91a1a}-2.56\\%$ | | test_redq_speed | 15.2162ms | 13.7386ms | 72.7874 Ops/s | 73.5075 Ops/s | $\color{#d91a1a}-0.98\\%$ | | test_redq_deprec_speed | 14.5170ms | 13.4994ms | 74.0773 Ops/s | 68.1296 Ops/s | $\textbf{\color{#35bf28}+8.73\\%}$ | | test_td3_speed | 18.2958ms | 8.7683ms | 114.0472 Ops/s | 117.3567 Ops/s | $\color{#d91a1a}-2.82\\%$ | | test_cql_speed | 38.5666ms | 37.3062ms | 26.8052 Ops/s | 27.2272 Ops/s | $\color{#d91a1a}-1.55\\%$ | | test_a2c_speed | 8.0100ms | 7.5131ms | 133.1002 Ops/s | 133.7666 Ops/s | $\color{#d91a1a}-0.50\\%$ | | test_ppo_speed | 8.9306ms | 7.7948ms | 128.2904 Ops/s | 128.5440 Ops/s | $\color{#d91a1a}-0.20\\%$ | | test_reinforce_speed | 7.5790ms | 6.7249ms | 148.7016 Ops/s | 150.1312 Ops/s | $\color{#d91a1a}-0.95\\%$ | | test_iql_speed | 34.4814ms | 32.8478ms | 30.4434 Ops/s | 30.3419 Ops/s | $\color{#35bf28}+0.33\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.8808ms | 3.5318ms | 283.1441 Ops/s | 284.6696 Ops/s | $\color{#d91a1a}-0.54\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.8737ms | 0.5004ms | 1.9983 KOps/s | 1.7905 KOps/s | $\textbf{\color{#35bf28}+11.61\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 98.4919ms | 0.5400ms | 1.8518 KOps/s | 2.1517 KOps/s | $\textbf{\color{#d91a1a}-13.94\\%}$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.4207ms | 3.4973ms | 285.9385 Ops/s | 285.8151 Ops/s | $\color{#35bf28}+0.04\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.9791ms | 0.4956ms | 2.0179 KOps/s | 2.0609 KOps/s | $\color{#d91a1a}-2.09\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.8869ms | 0.4676ms | 2.1387 KOps/s | 2.1594 KOps/s | $\color{#d91a1a}-0.96\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.4810ms | 1.7701ms | 564.9396 Ops/s | 577.5065 Ops/s | $\color{#d91a1a}-2.18\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.4035ms | 1.6667ms | 599.9790 Ops/s | 606.1511 Ops/s | $\color{#d91a1a}-1.02\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.1583ms | 3.6293ms | 275.5323 Ops/s | 271.8915 Ops/s | $\color{#35bf28}+1.34\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.1494ms | 0.6397ms | 1.5633 KOps/s | 1.5947 KOps/s | $\color{#d91a1a}-1.97\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9962ms | 0.6169ms | 1.6210 KOps/s | 1.6642 KOps/s | $\color{#d91a1a}-2.60\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.4549ms | 3.5370ms | 282.7253 Ops/s | 287.0542 Ops/s | $\color{#d91a1a}-1.51\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.0349ms | 0.5004ms | 1.9982 KOps/s | 2.0490 KOps/s | $\color{#d91a1a}-2.48\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7560ms | 0.4794ms | 2.0859 KOps/s | 2.1245 KOps/s | $\color{#d91a1a}-1.81\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.8196ms | 3.4656ms | 288.5493 Ops/s | 288.4743 Ops/s | $\color{#35bf28}+0.03\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.5910ms | 0.4915ms | 2.0344 KOps/s | 2.0646 KOps/s | $\color{#d91a1a}-1.46\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 3.7825ms | 0.4742ms | 2.1087 KOps/s | 2.1446 KOps/s | $\color{#d91a1a}-1.67\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.3517ms | 3.6256ms | 275.8177 Ops/s | 275.7541 Ops/s | $\color{#35bf28}+0.02\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.3553ms | 0.6481ms | 1.5429 KOps/s | 1.5813 KOps/s | $\color{#d91a1a}-2.43\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7841ms | 0.6131ms | 1.6311 KOps/s | 1.6513 KOps/s | $\color{#d91a1a}-1.22\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1212s | 8.2776ms | 120.8081 Ops/s | 118.5307 Ops/s | $\color{#35bf28}+1.92\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 15.3004ms | 12.7169ms | 78.6354 Ops/s | 79.7687 Ops/s | $\color{#d91a1a}-1.42\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 4.4726ms | 1.1799ms | 847.5388 Ops/s | 948.5899 Ops/s | $\textbf{\color{#d91a1a}-10.65\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1180s | 5.9692ms | 167.5266 Ops/s | 169.4694 Ops/s | $\color{#d91a1a}-1.15\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 14.9893ms | 12.6731ms | 78.9071 Ops/s | 79.4269 Ops/s | $\color{#d91a1a}-0.65\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 3.8234ms | 1.1069ms | 903.4265 Ops/s | 942.5714 Ops/s | $\color{#d91a1a}-4.15\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1121s | 6.0129ms | 166.3090 Ops/s | 119.1843 Ops/s | $\textbf{\color{#35bf28}+39.54\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 15.3456ms | 12.8765ms | 77.6606 Ops/s | 78.3942 Ops/s | $\color{#d91a1a}-0.94\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.7384ms | 1.2403ms | 806.2289 Ops/s | 832.4584 Ops/s | $\color{#d91a1a}-3.15\\%$ |
github-actions[bot] commented 1 week ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}15$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1604s | 0.1069s | 9.3512 Ops/s | 9.7647 Ops/s | $\color{#d91a1a}-4.23\\%$ | | test_sync | 88.7251ms | 87.4709ms | 11.4324 Ops/s | 11.1522 Ops/s | $\color{#35bf28}+2.51\\%$ | | test_async | 0.1651s | 81.5658ms | 12.2600 Ops/s | 11.9747 Ops/s | $\color{#35bf28}+2.38\\%$ | | test_single_pixels | 0.1125s | 0.1117s | 8.9564 Ops/s | 8.8710 Ops/s | $\color{#35bf28}+0.96\\%$ | | test_sync_pixels | 77.0606ms | 70.2990ms | 14.2250 Ops/s | 15.0183 Ops/s | $\textbf{\color{#d91a1a}-5.28\\%}$ | | test_async_pixels | 0.1211s | 55.7742ms | 17.9294 Ops/s | 17.8755 Ops/s | $\color{#35bf28}+0.30\\%$ | | test_simple | 0.7209s | 0.7162s | 1.3963 Ops/s | 1.3354 Ops/s | $\color{#35bf28}+4.56\\%$ | | test_transformed | 1.0134s | 0.9562s | 1.0458 Ops/s | 1.0298 Ops/s | $\color{#35bf28}+1.55\\%$ | | test_serial | 2.1805s | 2.1183s | 0.4721 Ops/s | 0.4594 Ops/s | $\color{#35bf28}+2.75\\%$ | | test_parallel | 1.8830s | 1.8306s | 0.5463 Ops/s | 0.5369 Ops/s | $\color{#35bf28}+1.75\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 79.8210μs | 34.3286μs | 29.1302 KOps/s | 30.2558 KOps/s | $\color{#d91a1a}-3.72\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 46.7810μs | 19.8055μs | 50.4909 KOps/s | 51.9428 KOps/s | $\color{#d91a1a}-2.80\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 43.7200μs | 19.9651μs | 50.0874 KOps/s | 54.1194 KOps/s | $\textbf{\color{#d91a1a}-7.45\\%}$ | | test_step_mdp_speed[True-True-True-False-False] | 35.6200μs | 11.5026μs | 86.9366 KOps/s | 91.8896 KOps/s | $\textbf{\color{#d91a1a}-5.39\\%}$ | | test_step_mdp_speed[True-True-False-True-True] | 64.7210μs | 36.7405μs | 27.2180 KOps/s | 28.9360 KOps/s | $\textbf{\color{#d91a1a}-5.94\\%}$ | | test_step_mdp_speed[True-True-False-True-False] | 47.4610μs | 21.9185μs | 45.6235 KOps/s | 48.1002 KOps/s | $\textbf{\color{#d91a1a}-5.15\\%}$ | | test_step_mdp_speed[True-True-False-False-True] | 52.0500μs | 21.7732μs | 45.9280 KOps/s | 48.6942 KOps/s | $\textbf{\color{#d91a1a}-5.68\\%}$ | | test_step_mdp_speed[True-True-False-False-False] | 37.4910μs | 13.3339μs | 74.9967 KOps/s | 78.9615 KOps/s | $\textbf{\color{#d91a1a}-5.02\\%}$ | | test_step_mdp_speed[True-False-True-True-True] | 82.3910μs | 37.7845μs | 26.4659 KOps/s | 27.1969 KOps/s | $\color{#d91a1a}-2.69\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 48.6210μs | 23.7681μs | 42.0731 KOps/s | 43.9610 KOps/s | $\color{#d91a1a}-4.29\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 46.5210μs | 21.4748μs | 46.5663 KOps/s | 48.8558 KOps/s | $\color{#d91a1a}-4.69\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 42.1800μs | 13.2441μs | 75.5053 KOps/s | 78.5280 KOps/s | $\color{#d91a1a}-3.85\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 74.0810μs | 40.0875μs | 24.9454 KOps/s | 26.2916 KOps/s | $\textbf{\color{#d91a1a}-5.12\\%}$ | | test_step_mdp_speed[True-False-False-True-False] | 49.5610μs | 25.3685μs | 39.4189 KOps/s | 40.8588 KOps/s | $\color{#d91a1a}-3.52\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 48.1420μs | 23.2761μs | 42.9626 KOps/s | 45.3123 KOps/s | $\textbf{\color{#d91a1a}-5.19\\%}$ | | test_step_mdp_speed[True-False-False-False-False] | 37.8400μs | 15.2246μs | 65.6833 KOps/s | 69.7254 KOps/s | $\textbf{\color{#d91a1a}-5.80\\%}$ | | test_step_mdp_speed[False-True-True-True-True] | 69.1620μs | 37.9626μs | 26.3417 KOps/s | 27.5215 KOps/s | $\color{#d91a1a}-4.29\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 49.6110μs | 23.6944μs | 42.2041 KOps/s | 44.1930 KOps/s | $\color{#d91a1a}-4.50\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 53.2100μs | 25.3855μs | 39.3926 KOps/s | 41.0938 KOps/s | $\color{#d91a1a}-4.14\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 36.2510μs | 15.1053μs | 66.2018 KOps/s | 69.4607 KOps/s | $\color{#d91a1a}-4.69\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 66.1500μs | 39.7184μs | 25.1773 KOps/s | 26.3047 KOps/s | $\color{#d91a1a}-4.29\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 49.3300μs | 25.2724μs | 39.5688 KOps/s | 39.9379 KOps/s | $\color{#d91a1a}-0.92\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 55.5410μs | 27.6557μs | 36.1589 KOps/s | 37.4086 KOps/s | $\color{#d91a1a}-3.34\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 38.4400μs | 17.0567μs | 58.6280 KOps/s | 59.9920 KOps/s | $\color{#d91a1a}-2.27\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 69.6510μs | 41.9712μs | 23.8259 KOps/s | 24.3101 KOps/s | $\color{#d91a1a}-1.99\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 58.5220μs | 27.6308μs | 36.1914 KOps/s | 36.5818 KOps/s | $\color{#d91a1a}-1.07\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 55.0810μs | 27.3429μs | 36.5726 KOps/s | 37.5877 KOps/s | $\color{#d91a1a}-2.70\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 41.6820μs | 17.0478μs | 58.6586 KOps/s | 60.2279 KOps/s | $\color{#d91a1a}-2.61\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 67.8200μs | 44.9638μs | 22.2401 KOps/s | 22.9487 KOps/s | $\color{#d91a1a}-3.09\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 75.5220μs | 29.4618μs | 33.9423 KOps/s | 33.9162 KOps/s | $\color{#35bf28}+0.08\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 53.0910μs | 28.9158μs | 34.5832 KOps/s | 35.0990 KOps/s | $\color{#d91a1a}-1.47\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 40.0600μs | 18.5830μs | 53.8127 KOps/s | 54.1472 KOps/s | $\color{#d91a1a}-0.62\\%$ | | test_values[generalized_advantage_estimate-True-True] | 25.1712ms | 24.8172ms | 40.2946 Ops/s | 39.8344 Ops/s | $\color{#35bf28}+1.16\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 91.9210ms | 2.7295ms | 366.3715 Ops/s | 355.9722 Ops/s | $\color{#35bf28}+2.92\\%$ | | test_values[td0_return_estimate-False-False] | 93.8710μs | 64.9319μs | 15.4007 KOps/s | 14.4511 KOps/s | $\textbf{\color{#35bf28}+6.57\\%}$ | | test_values[td1_return_estimate-False-False] | 56.0182ms | 55.2006ms | 18.1158 Ops/s | 17.4404 Ops/s | $\color{#35bf28}+3.87\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.4556ms | 1.0698ms | 934.7725 Ops/s | 928.2925 Ops/s | $\color{#35bf28}+0.70\\%$ | | test_values[td_lambda_return_estimate-True-False] | 89.8521ms | 88.0009ms | 11.3635 Ops/s | 10.8855 Ops/s | $\color{#35bf28}+4.39\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.4492ms | 1.0708ms | 933.9035 Ops/s | 920.7289 Ops/s | $\color{#35bf28}+1.43\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 25.7917ms | 25.5490ms | 39.1404 Ops/s | 39.4257 Ops/s | $\color{#d91a1a}-0.72\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9485ms | 0.6978ms | 1.4332 KOps/s | 1.3323 KOps/s | $\textbf{\color{#35bf28}+7.57\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7391ms | 0.6699ms | 1.4927 KOps/s | 1.4328 KOps/s | $\color{#35bf28}+4.18\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5884ms | 1.4501ms | 689.6209 Ops/s | 683.9510 Ops/s | $\color{#35bf28}+0.83\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7082ms | 0.6653ms | 1.5030 KOps/s | 1.4844 KOps/s | $\color{#35bf28}+1.25\\%$ | | test_dqn_speed | 7.7089ms | 1.3843ms | 722.3827 Ops/s | 701.0743 Ops/s | $\color{#35bf28}+3.04\\%$ | | test_ddpg_speed | 3.2379ms | 2.8616ms | 349.4587 Ops/s | 340.9599 Ops/s | $\color{#35bf28}+2.49\\%$ | | test_sac_speed | 8.8515ms | 8.2803ms | 120.7682 Ops/s | 120.8229 Ops/s | $\color{#d91a1a}-0.05\\%$ | | test_redq_speed | 0.1056s | 11.7477ms | 85.1231 Ops/s | 93.2960 Ops/s | $\textbf{\color{#d91a1a}-8.76\\%}$ | | test_redq_deprec_speed | 12.0532ms | 11.3650ms | 87.9895 Ops/s | 78.3523 Ops/s | $\textbf{\color{#35bf28}+12.30\\%}$ | | test_td3_speed | 10.0003ms | 8.1762ms | 122.3062 Ops/s | 121.6445 Ops/s | $\color{#35bf28}+0.54\\%$ | | test_cql_speed | 26.4551ms | 25.3782ms | 39.4040 Ops/s | 38.5148 Ops/s | $\color{#35bf28}+2.31\\%$ | | test_a2c_speed | 6.3529ms | 5.6073ms | 178.3381 Ops/s | 174.6136 Ops/s | $\color{#35bf28}+2.13\\%$ | | test_ppo_speed | 6.3150ms | 5.9572ms | 167.8645 Ops/s | 166.1805 Ops/s | $\color{#35bf28}+1.01\\%$ | | test_reinforce_speed | 5.4244ms | 4.6064ms | 217.0908 Ops/s | 214.2066 Ops/s | $\color{#35bf28}+1.35\\%$ | | test_iql_speed | 19.7731ms | 19.2375ms | 51.9818 Ops/s | 49.9158 Ops/s | $\color{#35bf28}+4.14\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.9354ms | 4.6828ms | 213.5486 Ops/s | 217.0527 Ops/s | $\color{#d91a1a}-1.61\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.5001ms | 0.5423ms | 1.8439 KOps/s | 1.8291 KOps/s | $\color{#35bf28}+0.81\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6993ms | 0.5226ms | 1.9134 KOps/s | 1.9050 KOps/s | $\color{#35bf28}+0.44\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.8777ms | 4.6359ms | 215.7100 Ops/s | 218.4147 Ops/s | $\color{#d91a1a}-1.24\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.5466ms | 0.5363ms | 1.8646 KOps/s | 1.8591 KOps/s | $\color{#35bf28}+0.29\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7243ms | 0.5176ms | 1.9321 KOps/s | 1.9043 KOps/s | $\color{#35bf28}+1.46\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 5.8433ms | 1.9832ms | 504.2400 Ops/s | 507.9478 Ops/s | $\color{#d91a1a}-0.73\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.0663ms | 1.8807ms | 531.7302 Ops/s | 533.3100 Ops/s | $\color{#d91a1a}-0.30\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0507ms | 4.8099ms | 207.9061 Ops/s | 208.4956 Ops/s | $\color{#d91a1a}-0.28\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.5613ms | 0.6801ms | 1.4704 KOps/s | 1.4584 KOps/s | $\color{#35bf28}+0.83\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 1.0777ms | 0.6647ms | 1.5044 KOps/s | 1.4945 KOps/s | $\color{#35bf28}+0.66\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.8234ms | 4.6566ms | 214.7505 Ops/s | 213.5104 Ops/s | $\color{#35bf28}+0.58\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.8006ms | 0.5425ms | 1.8434 KOps/s | 1.8411 KOps/s | $\color{#35bf28}+0.13\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7415ms | 0.5242ms | 1.9076 KOps/s | 1.9131 KOps/s | $\color{#d91a1a}-0.28\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.9163ms | 4.6195ms | 216.4713 Ops/s | 213.6947 Ops/s | $\color{#35bf28}+1.30\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7585ms | 0.5403ms | 1.8507 KOps/s | 1.8606 KOps/s | $\color{#d91a1a}-0.53\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6968ms | 0.5193ms | 1.9257 KOps/s | 1.9221 KOps/s | $\color{#35bf28}+0.19\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0492ms | 4.8059ms | 208.0790 Ops/s | 205.5750 Ops/s | $\color{#35bf28}+1.22\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.5232ms | 0.6890ms | 1.4514 KOps/s | 1.4575 KOps/s | $\color{#d91a1a}-0.42\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9495ms | 0.6688ms | 1.4952 KOps/s | 1.4954 KOps/s | $\color{#d91a1a}-0.01\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1524s | 7.9112ms | 126.4023 Ops/s | 136.7589 Ops/s | $\textbf{\color{#d91a1a}-7.57\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 17.2960ms | 14.9820ms | 66.7467 Ops/s | 65.0675 Ops/s | $\color{#35bf28}+2.58\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 6.7860ms | 1.3113ms | 762.5929 Ops/s | 894.1631 Ops/s | $\textbf{\color{#d91a1a}-14.71\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1274s | 7.3744ms | 135.6051 Ops/s | 138.1285 Ops/s | $\color{#d91a1a}-1.83\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 17.5720ms | 15.0922ms | 66.2594 Ops/s | 56.1534 Ops/s | $\textbf{\color{#35bf28}+18.00\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 6.5993ms | 1.2518ms | 798.8374 Ops/s | 887.7296 Ops/s | $\textbf{\color{#d91a1a}-10.01\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1290s | 10.0462ms | 99.5396 Ops/s | 136.2470 Ops/s | $\textbf{\color{#d91a1a}-26.94\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 17.5455ms | 15.2425ms | 65.6059 Ops/s | 64.4503 Ops/s | $\color{#35bf28}+1.79\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.4979ms | 1.3183ms | 758.5421 Ops/s | 713.2625 Ops/s | $\textbf{\color{#35bf28}+6.35\\%}$ |