search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.01k stars 269 forks source link

[Feature] Make ProbabilisticActor compatible with Composite distributions #2220

Closed vmoens closed 4 weeks ago

pytorch-bot[bot] commented 1 month ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2220

Note: Links to docs will display an error until the docs builds have been completed.

:x: 3 New Failures, 2 Unrelated Failures

As of commit 0c604e7e11b084607b109c7152d024a56dfe6cd6 with merge base 3787a9edeea03cb9e98b928824072b939c327776 (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2220#26063067256) ([gh](https://github.com/pytorch/rl/actions/runs/9461716532/job/26063067256)) `RuntimeError: Command docker exec -t 00abb6bb74b55be0a92ee7367d04ad5dd1eebae9a18796a38be541a274e2d517 /exec failed with exit code 139` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2220#26063069934) ([gh](https://github.com/pytorch/rl/actions/runs/9461716555/job/26063069934)) `RuntimeError: Command docker exec -t 4fe5ec11ab9280f6a462ce1a36aee3a6b90240942fc9d4a2212c85cb53c9e9e8 /exec failed with exit code 1` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2220#26063065851) ([gh](https://github.com/pytorch/rl/actions/runs/9461716548/job/26063065851)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2220#26063072810) ([gh](https://github.com/pytorch/rl/actions/runs/9461716539/job/26063072810)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/3787a9edeea03cb9e98b928824072b939c327776#26063036923)) `##[error]The operation was canceled.` * [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2220#26063069643) ([gh](https://github.com/pytorch/rl/actions/runs/9461716555/job/26063069643)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/3787a9edeea03cb9e98b928824072b939c327776#26063035036)) `##[error]The operation was canceled.`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1208s | 60.4128ms | 16.5528 Ops/s | 17.1410 Ops/s | $\color{#d91a1a}-3.43\\%$ | | test_sync | 39.0389ms | 31.4674ms | 31.7789 Ops/s | 31.1697 Ops/s | $\color{#35bf28}+1.95\\%$ | | test_async | 59.2461ms | 29.6140ms | 33.7678 Ops/s | 35.9820 Ops/s | $\textbf{\color{#d91a1a}-6.15\\%}$ | | test_simple | 0.4580s | 0.3940s | 2.5378 Ops/s | 2.6044 Ops/s | $\color{#d91a1a}-2.56\\%$ | | test_transformed | 0.5359s | 0.5328s | 1.8767 Ops/s | 1.7639 Ops/s | $\textbf{\color{#35bf28}+6.40\\%}$ | | test_serial | 1.3241s | 1.2668s | 0.7894 Ops/s | 0.7623 Ops/s | $\color{#35bf28}+3.54\\%$ | | test_parallel | 1.1440s | 1.0817s | 0.9245 Ops/s | 0.9298 Ops/s | $\color{#d91a1a}-0.57\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1996ms | 21.8022μs | 45.8670 KOps/s | 46.4636 KOps/s | $\color{#d91a1a}-1.28\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 50.7350μs | 13.1343μs | 76.1367 KOps/s | 75.9406 KOps/s | $\color{#35bf28}+0.26\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 51.1060μs | 12.7895μs | 78.1891 KOps/s | 77.6131 KOps/s | $\color{#35bf28}+0.74\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 52.6690μs | 7.7689μs | 128.7178 KOps/s | 128.1189 KOps/s | $\color{#35bf28}+0.47\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 57.6880μs | 23.2292μs | 43.0493 KOps/s | 43.9340 KOps/s | $\color{#d91a1a}-2.01\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 51.4970μs | 14.6292μs | 68.3563 KOps/s | 68.5721 KOps/s | $\color{#d91a1a}-0.31\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 41.5280μs | 14.2224μs | 70.3118 KOps/s | 70.8934 KOps/s | $\color{#d91a1a}-0.82\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 43.0410μs | 9.1229μs | 109.6144 KOps/s | 111.7511 KOps/s | $\color{#d91a1a}-1.91\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 59.1800μs | 24.6642μs | 40.5445 KOps/s | 40.7595 KOps/s | $\color{#d91a1a}-0.53\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 62.0760μs | 15.9678μs | 62.6262 KOps/s | 63.9270 KOps/s | $\color{#d91a1a}-2.03\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 45.3850μs | 14.1810μs | 70.5171 KOps/s | 70.3476 KOps/s | $\color{#35bf28}+0.24\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 45.4850μs | 9.0096μs | 110.9932 KOps/s | 109.6319 KOps/s | $\color{#35bf28}+1.24\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 62.8470μs | 25.8042μs | 38.7533 KOps/s | 39.7456 KOps/s | $\color{#d91a1a}-2.50\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 56.8470μs | 17.0949μs | 58.4968 KOps/s | 58.4584 KOps/s | $\color{#35bf28}+0.07\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 62.7670μs | 15.3189μs | 65.2787 KOps/s | 65.8038 KOps/s | $\color{#d91a1a}-0.80\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 37.8510μs | 10.1396μs | 98.6232 KOps/s | 98.2408 KOps/s | $\color{#35bf28}+0.39\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 59.9630μs | 24.6724μs | 40.5312 KOps/s | 41.3775 KOps/s | $\color{#d91a1a}-2.05\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 50.4140μs | 15.8413μs | 63.1260 KOps/s | 63.1297 KOps/s | $-0.01\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 66.4550μs | 16.4397μs | 60.8285 KOps/s | 61.5912 KOps/s | $\color{#d91a1a}-1.24\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 50.3340μs | 10.2266μs | 97.7839 KOps/s | 98.0596 KOps/s | $\color{#d91a1a}-0.28\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 75.4510μs | 25.8094μs | 38.7456 KOps/s | 39.0918 KOps/s | $\color{#d91a1a}-0.89\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 42.5700μs | 17.0817μs | 58.5421 KOps/s | 58.6418 KOps/s | $\color{#d91a1a}-0.17\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 43.7120μs | 17.6920μs | 56.5228 KOps/s | 57.2082 KOps/s | $\color{#d91a1a}-1.20\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 37.3900μs | 11.5506μs | 86.5752 KOps/s | 87.9441 KOps/s | $\color{#d91a1a}-1.56\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 65.7030μs | 27.1248μs | 36.8667 KOps/s | 37.2953 KOps/s | $\color{#d91a1a}-1.15\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 50.4950μs | 18.4632μs | 54.1617 KOps/s | 54.2735 KOps/s | $\color{#d91a1a}-0.21\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 64.2300μs | 17.5995μs | 56.8196 KOps/s | 58.1199 KOps/s | $\color{#d91a1a}-2.24\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 53.2400μs | 11.5040μs | 86.9266 KOps/s | 88.5161 KOps/s | $\color{#d91a1a}-1.80\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 44.6430μs | 28.6723μs | 34.8769 KOps/s | 35.5866 KOps/s | $\color{#d91a1a}-1.99\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 44.5630μs | 19.6457μs | 50.9017 KOps/s | 52.3181 KOps/s | $\color{#d91a1a}-2.71\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 60.7130μs | 18.7601μs | 53.3045 KOps/s | 54.7993 KOps/s | $\color{#d91a1a}-2.73\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 38.0910μs | 12.6421μs | 79.1010 KOps/s | 80.4149 KOps/s | $\color{#d91a1a}-1.63\\%$ | | test_values[generalized_advantage_estimate-True-True] | 9.9859ms | 9.6262ms | 103.8830 Ops/s | 105.7405 Ops/s | $\color{#d91a1a}-1.76\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 50.3621ms | 35.6645ms | 28.0391 Ops/s | 28.3836 Ops/s | $\color{#d91a1a}-1.21\\%$ | | test_values[td0_return_estimate-False-False] | 0.2426ms | 0.1787ms | 5.5955 KOps/s | 5.8626 KOps/s | $\color{#d91a1a}-4.56\\%$ | | test_values[td1_return_estimate-False-False] | 25.8171ms | 23.7707ms | 42.0686 Ops/s | 41.6069 Ops/s | $\color{#35bf28}+1.11\\%$ | | test_values[vec_td1_return_estimate-False-False] | 37.9262ms | 35.4332ms | 28.2221 Ops/s | 28.2708 Ops/s | $\color{#d91a1a}-0.17\\%$ | | test_values[td_lambda_return_estimate-True-False] | 35.7745ms | 34.3421ms | 29.1188 Ops/s | 28.3258 Ops/s | $\color{#35bf28}+2.80\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 38.5542ms | 35.3193ms | 28.3131 Ops/s | 28.2918 Ops/s | $\color{#35bf28}+0.08\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.3434ms | 8.2076ms | 121.8386 Ops/s | 115.8715 Ops/s | $\textbf{\color{#35bf28}+5.15\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.2362ms | 1.8099ms | 552.5144 Ops/s | 488.3303 Ops/s | $\textbf{\color{#35bf28}+13.14\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4544ms | 0.3515ms | 2.8446 KOps/s | 2.7458 KOps/s | $\color{#35bf28}+3.60\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 48.5960ms | 45.9510ms | 21.7623 Ops/s | 22.3051 Ops/s | $\color{#d91a1a}-2.43\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.7558ms | 3.0477ms | 328.1145 Ops/s | 321.3985 Ops/s | $\color{#35bf28}+2.09\\%$ | | test_dqn_speed | 1.5079ms | 1.3618ms | 734.3254 Ops/s | 717.5998 Ops/s | $\color{#35bf28}+2.33\\%$ | | test_ddpg_speed | 3.4643ms | 2.8720ms | 348.1929 Ops/s | 336.1410 Ops/s | $\color{#35bf28}+3.59\\%$ | | test_sac_speed | 10.0632ms | 8.5702ms | 116.6839 Ops/s | 113.3896 Ops/s | $\color{#35bf28}+2.91\\%$ | | test_redq_speed | 15.0874ms | 13.5657ms | 73.7153 Ops/s | 70.5450 Ops/s | $\color{#35bf28}+4.49\\%$ | | test_redq_deprec_speed | 22.4671ms | 14.2169ms | 70.3386 Ops/s | 67.2040 Ops/s | $\color{#35bf28}+4.66\\%$ | | test_td3_speed | 8.8279ms | 8.4822ms | 117.8935 Ops/s | 114.1373 Ops/s | $\color{#35bf28}+3.29\\%$ | | test_cql_speed | 39.6776ms | 37.3588ms | 26.7674 Ops/s | 26.7660 Ops/s | $+0.01\\%$ | | test_a2c_speed | 8.8029ms | 7.6324ms | 131.0204 Ops/s | 129.0036 Ops/s | $\color{#35bf28}+1.56\\%$ | | test_ppo_speed | 8.8209ms | 7.8519ms | 127.3574 Ops/s | 126.4879 Ops/s | $\color{#35bf28}+0.69\\%$ | | test_reinforce_speed | 10.9492ms | 6.8321ms | 146.3680 Ops/s | 147.5586 Ops/s | $\color{#d91a1a}-0.81\\%$ | | test_iql_speed | 39.5787ms | 33.7253ms | 29.6513 Ops/s | 29.7755 Ops/s | $\color{#d91a1a}-0.42\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.2955ms | 3.6742ms | 272.1716 Ops/s | 269.6716 Ops/s | $\color{#35bf28}+0.93\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.8564ms | 0.5082ms | 1.9678 KOps/s | 1.9199 KOps/s | $\color{#35bf28}+2.49\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 1.0516ms | 0.4811ms | 2.0787 KOps/s | 2.0442 KOps/s | $\color{#35bf28}+1.69\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.3884ms | 3.6517ms | 273.8470 Ops/s | 268.3054 Ops/s | $\color{#35bf28}+2.07\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.0365ms | 0.5067ms | 1.9737 KOps/s | 1.9432 KOps/s | $\color{#35bf28}+1.57\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.8008ms | 0.4794ms | 2.0858 KOps/s | 2.0724 KOps/s | $\color{#35bf28}+0.65\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.1507ms | 1.7149ms | 583.1131 Ops/s | 574.9517 Ops/s | $\color{#35bf28}+1.42\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 1.7742ms | 1.6187ms | 617.7838 Ops/s | 605.2823 Ops/s | $\color{#35bf28}+2.07\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.4299ms | 3.8099ms | 262.4762 Ops/s | 262.9240 Ops/s | $\color{#d91a1a}-0.17\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.1402ms | 0.6296ms | 1.5884 KOps/s | 1.3616 KOps/s | $\textbf{\color{#35bf28}+16.66\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9675ms | 0.6042ms | 1.6551 KOps/s | 1.6567 KOps/s | $\color{#d91a1a}-0.10\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.0671ms | 3.7488ms | 266.7510 Ops/s | 268.7687 Ops/s | $\color{#d91a1a}-0.75\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.1158ms | 0.5066ms | 1.9740 KOps/s | 1.9146 KOps/s | $\color{#35bf28}+3.10\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7297ms | 0.4867ms | 2.0548 KOps/s | 2.0219 KOps/s | $\color{#35bf28}+1.63\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.6069ms | 3.7207ms | 268.7698 Ops/s | 271.5775 Ops/s | $\color{#d91a1a}-1.03\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7825ms | 0.5009ms | 1.9963 KOps/s | 1.9538 KOps/s | $\color{#35bf28}+2.17\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 3.5986ms | 0.4900ms | 2.0410 KOps/s | 2.0614 KOps/s | $\color{#d91a1a}-0.99\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.0010ms | 3.7826ms | 264.3709 Ops/s | 263.0552 Ops/s | $\color{#35bf28}+0.50\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.3135ms | 0.6250ms | 1.5999 KOps/s | 1.5810 KOps/s | $\color{#35bf28}+1.19\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9034ms | 0.6023ms | 1.6603 KOps/s | 1.6355 KOps/s | $\color{#35bf28}+1.52\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1145s | 5.9599ms | 167.7890 Ops/s | 122.7383 Ops/s | $\textbf{\color{#35bf28}+36.70\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 15.3675ms | 12.6548ms | 79.0217 Ops/s | 76.8412 Ops/s | $\color{#35bf28}+2.84\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 1.3654ms | 1.0731ms | 931.8412 Ops/s | 843.6743 Ops/s | $\textbf{\color{#35bf28}+10.45\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1040s | 7.6647ms | 130.4683 Ops/s | 176.9937 Ops/s | $\textbf{\color{#d91a1a}-26.29\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 15.9385ms | 12.8509ms | 77.8155 Ops/s | 75.8089 Ops/s | $\color{#35bf28}+2.65\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 4.0652ms | 1.1455ms | 872.9572 Ops/s | 922.2030 Ops/s | $\textbf{\color{#d91a1a}-5.34\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1059s | 6.0771ms | 164.5512 Ops/s | 126.5785 Ops/s | $\textbf{\color{#35bf28}+30.00\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 15.7878ms | 13.0126ms | 76.8487 Ops/s | 76.5253 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 3.9548ms | 1.2671ms | 789.2263 Ops/s | 809.5860 Ops/s | $\color{#d91a1a}-2.51\\%$ |
github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1170s | 0.1162s | 8.6037 Ops/s | 8.4691 Ops/s | $\color{#35bf28}+1.59\\%$ | | test_sync | 0.1016s | 98.3283ms | 10.1700 Ops/s | 9.5062 Ops/s | $\textbf{\color{#35bf28}+6.98\\%}$ | | test_async | 0.1920s | 77.7158ms | 12.8674 Ops/s | 12.1763 Ops/s | $\textbf{\color{#35bf28}+5.68\\%}$ | | test_single_pixels | 0.1279s | 0.1276s | 7.8359 Ops/s | 7.7983 Ops/s | $\color{#35bf28}+0.48\\%$ | | test_sync_pixels | 85.4130ms | 83.8345ms | 11.9283 Ops/s | 11.9400 Ops/s | $\color{#d91a1a}-0.10\\%$ | | test_async_pixels | 0.1590s | 68.8200ms | 14.5307 Ops/s | 14.2992 Ops/s | $\color{#35bf28}+1.62\\%$ | | test_simple | 0.8829s | 0.8263s | 1.2102 Ops/s | 1.2279 Ops/s | $\color{#d91a1a}-1.44\\%$ | | test_transformed | 1.1417s | 1.0760s | 0.9294 Ops/s | 0.9221 Ops/s | $\color{#35bf28}+0.79\\%$ | | test_serial | 2.5263s | 2.4626s | 0.4061 Ops/s | 0.3992 Ops/s | $\color{#35bf28}+1.73\\%$ | | test_parallel | 2.4165s | 2.3555s | 0.4245 Ops/s | 0.4234 Ops/s | $\color{#35bf28}+0.27\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1021ms | 32.7181μs | 30.5641 KOps/s | 29.5154 KOps/s | $\color{#35bf28}+3.55\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 44.6510μs | 19.6053μs | 51.0067 KOps/s | 50.0836 KOps/s | $\color{#35bf28}+1.84\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 45.5610μs | 18.6205μs | 53.7041 KOps/s | 51.6027 KOps/s | $\color{#35bf28}+4.07\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 28.0100μs | 11.1417μs | 89.7529 KOps/s | 88.3282 KOps/s | $\color{#35bf28}+1.61\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 75.7210μs | 34.8249μs | 28.7151 KOps/s | 28.2355 KOps/s | $\color{#35bf28}+1.70\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 37.1510μs | 21.0873μs | 47.4220 KOps/s | 46.4546 KOps/s | $\color{#35bf28}+2.08\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 39.9100μs | 20.3956μs | 49.0302 KOps/s | 47.3818 KOps/s | $\color{#35bf28}+3.48\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 37.4310μs | 12.8854μs | 77.6073 KOps/s | 75.6011 KOps/s | $\color{#35bf28}+2.65\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 53.5210μs | 36.8689μs | 27.1231 KOps/s | 26.7666 KOps/s | $\color{#35bf28}+1.33\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 41.5710μs | 23.3299μs | 42.8635 KOps/s | 42.5219 KOps/s | $\color{#35bf28}+0.80\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 39.0210μs | 20.4898μs | 48.8048 KOps/s | 47.0993 KOps/s | $\color{#35bf28}+3.62\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 28.4700μs | 12.8808μs | 77.6352 KOps/s | 74.8457 KOps/s | $\color{#35bf28}+3.73\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 64.1210μs | 38.4551μs | 26.0044 KOps/s | 25.4579 KOps/s | $\color{#35bf28}+2.15\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 65.8510μs | 24.6669μs | 40.5402 KOps/s | 39.5279 KOps/s | $\color{#35bf28}+2.56\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 57.6310μs | 21.8853μs | 45.6927 KOps/s | 43.8585 KOps/s | $\color{#35bf28}+4.18\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 35.9200μs | 14.8287μs | 67.4368 KOps/s | 66.3519 KOps/s | $\color{#35bf28}+1.64\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 64.5900μs | 36.5598μs | 27.3524 KOps/s | 26.5465 KOps/s | $\color{#35bf28}+3.04\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 47.9900μs | 23.5592μs | 42.4463 KOps/s | 41.8390 KOps/s | $\color{#35bf28}+1.45\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 52.4610μs | 24.7983μs | 40.3254 KOps/s | 39.3469 KOps/s | $\color{#35bf28}+2.49\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 39.0510μs | 14.8971μs | 67.1270 KOps/s | 66.3003 KOps/s | $\color{#35bf28}+1.25\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 65.5700μs | 38.6142μs | 25.8972 KOps/s | 25.3163 KOps/s | $\color{#35bf28}+2.29\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 41.8910μs | 25.3743μs | 39.4100 KOps/s | 39.3266 KOps/s | $\color{#35bf28}+0.21\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 56.0710μs | 26.3263μs | 37.9848 KOps/s | 36.7506 KOps/s | $\color{#35bf28}+3.36\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 35.5900μs | 16.6956μs | 59.8959 KOps/s | 59.0481 KOps/s | $\color{#35bf28}+1.44\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 66.0210μs | 40.5245μs | 24.6765 KOps/s | 24.2216 KOps/s | $\color{#35bf28}+1.88\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 44.3510μs | 26.8984μs | 37.1770 KOps/s | 36.1313 KOps/s | $\color{#35bf28}+2.89\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 41.5000μs | 26.1797μs | 38.1975 KOps/s | 36.7624 KOps/s | $\color{#35bf28}+3.90\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 34.1800μs | 16.6407μs | 60.0937 KOps/s | 58.7284 KOps/s | $\color{#35bf28}+2.32\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 72.9410μs | 42.9450μs | 23.2856 KOps/s | 22.9873 KOps/s | $\color{#35bf28}+1.30\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 50.2900μs | 29.0542μs | 34.4185 KOps/s | 33.5340 KOps/s | $\color{#35bf28}+2.64\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 55.2810μs | 28.0830μs | 35.6088 KOps/s | 34.5697 KOps/s | $\color{#35bf28}+3.01\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 34.3100μs | 18.4024μs | 54.3406 KOps/s | 53.1645 KOps/s | $\color{#35bf28}+2.21\\%$ | | test_values[generalized_advantage_estimate-True-True] | 25.8263ms | 24.8723ms | 40.2054 Ops/s | 39.5725 Ops/s | $\color{#35bf28}+1.60\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 89.3971ms | 2.6920ms | 371.4768 Ops/s | 375.0360 Ops/s | $\color{#d91a1a}-0.95\\%$ | | test_values[td0_return_estimate-False-False] | 86.9510μs | 66.3750μs | 15.0659 KOps/s | 15.0292 KOps/s | $\color{#35bf28}+0.24\\%$ | | test_values[td1_return_estimate-False-False] | 56.4160ms | 55.8571ms | 17.9028 Ops/s | 17.7158 Ops/s | $\color{#35bf28}+1.06\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.2602ms | 1.0866ms | 920.2820 Ops/s | 917.5687 Ops/s | $\color{#35bf28}+0.30\\%$ | | test_values[td_lambda_return_estimate-True-False] | 92.7590ms | 89.3074ms | 11.1973 Ops/s | 11.2374 Ops/s | $\color{#d91a1a}-0.36\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.2053ms | 1.0846ms | 921.9941 Ops/s | 918.6842 Ops/s | $\color{#35bf28}+0.36\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 25.2955ms | 25.1117ms | 39.8221 Ops/s | 39.2408 Ops/s | $\color{#35bf28}+1.48\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9673ms | 0.7263ms | 1.3768 KOps/s | 1.3701 KOps/s | $\color{#35bf28}+0.50\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7690ms | 0.6731ms | 1.4857 KOps/s | 1.4717 KOps/s | $\color{#35bf28}+0.95\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5719ms | 1.4773ms | 676.9184 Ops/s | 676.8534 Ops/s | $+0.01\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7131ms | 0.6900ms | 1.4494 KOps/s | 1.4366 KOps/s | $\color{#35bf28}+0.89\\%$ | | test_dqn_speed | 1.8402ms | 1.4764ms | 677.3335 Ops/s | 681.3367 Ops/s | $\color{#d91a1a}-0.59\\%$ | | test_ddpg_speed | 3.2818ms | 2.9782ms | 335.7714 Ops/s | 332.4097 Ops/s | $\color{#35bf28}+1.01\\%$ | | test_sac_speed | 8.7716ms | 8.5420ms | 117.0693 Ops/s | 116.2632 Ops/s | $\color{#35bf28}+0.69\\%$ | | test_redq_speed | 12.7859ms | 10.8040ms | 92.5583 Ops/s | 92.5222 Ops/s | $\color{#35bf28}+0.04\\%$ | | test_redq_deprec_speed | 11.8500ms | 11.3902ms | 87.7950 Ops/s | 85.7499 Ops/s | $\color{#35bf28}+2.39\\%$ | | test_td3_speed | 8.5661ms | 8.4959ms | 117.7044 Ops/s | 117.7325 Ops/s | $\color{#d91a1a}-0.02\\%$ | | test_cql_speed | 27.6626ms | 26.1180ms | 38.2878 Ops/s | 38.2015 Ops/s | $\color{#35bf28}+0.23\\%$ | | test_a2c_speed | 5.8702ms | 5.6051ms | 178.4104 Ops/s | 171.9770 Ops/s | $\color{#35bf28}+3.74\\%$ | | test_ppo_speed | 6.1634ms | 5.9783ms | 167.2715 Ops/s | 162.3729 Ops/s | $\color{#35bf28}+3.02\\%$ | | test_reinforce_speed | 5.3025ms | 4.5831ms | 218.1939 Ops/s | 210.9889 Ops/s | $\color{#35bf28}+3.41\\%$ | | test_iql_speed | 20.1228ms | 19.5618ms | 51.1200 Ops/s | 49.8275 Ops/s | $\color{#35bf28}+2.59\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.1475ms | 4.8417ms | 206.5398 Ops/s | 204.1792 Ops/s | $\color{#35bf28}+1.16\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.2980ms | 0.6060ms | 1.6502 KOps/s | 1.6423 KOps/s | $\color{#35bf28}+0.48\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7807ms | 0.5782ms | 1.7294 KOps/s | 1.7145 KOps/s | $\color{#35bf28}+0.87\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.0340ms | 4.7993ms | 208.3655 Ops/s | 204.9362 Ops/s | $\color{#35bf28}+1.67\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.3783ms | 0.5981ms | 1.6720 KOps/s | 1.6562 KOps/s | $\color{#35bf28}+0.95\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7612ms | 0.5696ms | 1.7558 KOps/s | 1.7205 KOps/s | $\color{#35bf28}+2.05\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 4.9557ms | 2.1317ms | 469.1180 Ops/s | 460.8999 Ops/s | $\color{#35bf28}+1.78\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2895ms | 2.0093ms | 497.6953 Ops/s | 484.8079 Ops/s | $\color{#35bf28}+2.66\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0535ms | 4.9750ms | 201.0041 Ops/s | 198.0499 Ops/s | $\color{#35bf28}+1.49\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9979ms | 0.7321ms | 1.3659 KOps/s | 1.3485 KOps/s | $\color{#35bf28}+1.30\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.8066ms | 0.7051ms | 1.4183 KOps/s | 1.4079 KOps/s | $\color{#35bf28}+0.74\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.9331ms | 4.8310ms | 206.9944 Ops/s | 201.7133 Ops/s | $\color{#35bf28}+2.62\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7134ms | 0.6040ms | 1.6555 KOps/s | 1.6309 KOps/s | $\color{#35bf28}+1.51\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 4.5181ms | 0.5808ms | 1.7218 KOps/s | 1.6942 KOps/s | $\color{#35bf28}+1.62\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.0545ms | 4.8162ms | 207.6306 Ops/s | 203.3116 Ops/s | $\color{#35bf28}+2.12\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.3780ms | 0.5932ms | 1.6857 KOps/s | 1.6549 KOps/s | $\color{#35bf28}+1.86\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6856ms | 0.5727ms | 1.7460 KOps/s | 1.7399 KOps/s | $\color{#35bf28}+0.35\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0166ms | 4.9703ms | 201.1933 Ops/s | 197.0124 Ops/s | $\color{#35bf28}+2.12\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8498ms | 0.7240ms | 1.3812 KOps/s | 1.3434 KOps/s | $\color{#35bf28}+2.81\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9483ms | 0.7076ms | 1.4133 KOps/s | 1.3889 KOps/s | $\color{#35bf28}+1.76\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1231s | 9.6043ms | 104.1205 Ops/s | 131.5250 Ops/s | $\textbf{\color{#d91a1a}-20.84\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.8657ms | 16.4161ms | 60.9160 Ops/s | 58.4079 Ops/s | $\color{#35bf28}+4.29\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.2954ms | 1.3093ms | 763.7475 Ops/s | 755.3734 Ops/s | $\color{#35bf28}+1.11\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1048s | 7.1628ms | 139.6106 Ops/s | 139.2785 Ops/s | $\color{#35bf28}+0.24\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 18.9436ms | 16.3696ms | 61.0888 Ops/s | 52.4642 Ops/s | $\textbf{\color{#35bf28}+16.44\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.4595ms | 1.3265ms | 753.8608 Ops/s | 738.9639 Ops/s | $\color{#35bf28}+2.02\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1063s | 9.3511ms | 106.9390 Ops/s | 135.9787 Ops/s | $\textbf{\color{#d91a1a}-21.36\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 19.0483ms | 16.6635ms | 60.0112 Ops/s | 58.5012 Ops/s | $\color{#35bf28}+2.58\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 7.6094ms | 1.5839ms | 631.3661 Ops/s | 664.1759 Ops/s | $\color{#d91a1a}-4.94\\%$ |