search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.27k stars 302 forks source link

[Refactor] Deprecate NormalParamWrapper #2308

Closed vmoens closed 2 months ago

pytorch-bot[bot] commented 2 months ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2308

Note: Links to docs will display an error until the docs builds have been completed.

:x: 1 New Failure, 1 Pending, 1 Unrelated Failure

As of commit 5066ec0f57ac8afb6c87b9a60b6449b21eed24ce with merge base 94abb502157febcf05c28a5afdea04ba1c3e2dfb (image):

NEW FAILURE - The following job has failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2308#27922630165) ([gh](https://github.com/pytorch/rl/actions/runs/10097540821/job/27922630165)) `RuntimeError: Command docker exec -t f2db62e89ee5463aafd4fe592c469eb3af0a7301027d7f7ac4675153cb4e4697 /exec failed with exit code 139`

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2308#27922628948) ([gh](https://github.com/pytorch/rl/actions/runs/10097540806/job/27922628948)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/94abb502157febcf05c28a5afdea04ba1c3e2dfb#27922434139)) `test/test_transforms.py::TestActionDiscretizer::test_trans_parallel_env_check[False]`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 2 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 59.5193ms | 58.6612ms | 17.0470 Ops/s | 17.1092 Ops/s | $\color{#d91a1a}-0.36\\%$ | | test_sync | 42.3374ms | 31.9010ms | 31.3470 Ops/s | 30.8822 Ops/s | $\color{#35bf28}+1.51\\%$ | | test_async | 75.5890ms | 30.2112ms | 33.1003 Ops/s | 32.3573 Ops/s | $\color{#35bf28}+2.30\\%$ | | test_simple | 0.4909s | 0.4134s | 2.4191 Ops/s | 2.3575 Ops/s | $\color{#35bf28}+2.61\\%$ | | test_transformed | 0.6348s | 0.5757s | 1.7370 Ops/s | 1.7610 Ops/s | $\color{#d91a1a}-1.36\\%$ | | test_serial | 1.3128s | 1.2483s | 0.8011 Ops/s | 0.7864 Ops/s | $\color{#35bf28}+1.87\\%$ | | test_parallel | 1.1565s | 1.0941s | 0.9140 Ops/s | 0.8747 Ops/s | $\color{#35bf28}+4.49\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1940ms | 24.1255μs | 41.4499 KOps/s | 40.9981 KOps/s | $\color{#35bf28}+1.10\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 77.6780μs | 14.0206μs | 71.3234 KOps/s | 69.9380 KOps/s | $\color{#35bf28}+1.98\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 41.8280μs | 13.7971μs | 72.4791 KOps/s | 71.7323 KOps/s | $\color{#35bf28}+1.04\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 33.1720μs | 8.1650μs | 122.4747 KOps/s | 121.2782 KOps/s | $\color{#35bf28}+0.99\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 0.1081ms | 25.8451μs | 38.6920 KOps/s | 38.0733 KOps/s | $\color{#35bf28}+1.63\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 46.6670μs | 15.5926μs | 64.1329 KOps/s | 63.0527 KOps/s | $\color{#35bf28}+1.71\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 51.6750μs | 15.3201μs | 65.2737 KOps/s | 64.3631 KOps/s | $\color{#35bf28}+1.41\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 41.1060μs | 9.5520μs | 104.6906 KOps/s | 103.0699 KOps/s | $\color{#35bf28}+1.57\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 60.2920μs | 27.6052μs | 36.2250 KOps/s | 35.4510 KOps/s | $\color{#35bf28}+2.18\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 60.9130μs | 17.2345μs | 58.0231 KOps/s | 57.6825 KOps/s | $\color{#35bf28}+0.59\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 42.2990μs | 15.5551μs | 64.2877 KOps/s | 63.6105 KOps/s | $\color{#35bf28}+1.06\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 36.1170μs | 9.6436μs | 103.6956 KOps/s | 101.6943 KOps/s | $\color{#35bf28}+1.97\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 67.1950μs | 29.3687μs | 34.0499 KOps/s | 33.1371 KOps/s | $\color{#35bf28}+2.75\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 46.4160μs | 18.7722μs | 53.2704 KOps/s | 53.1977 KOps/s | $\color{#35bf28}+0.14\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 51.0050μs | 17.0800μs | 58.5479 KOps/s | 57.8838 KOps/s | $\color{#35bf28}+1.15\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 38.0510μs | 11.1483μs | 89.6996 KOps/s | 90.2983 KOps/s | $\color{#d91a1a}-0.66\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 64.7110μs | 27.8302μs | 35.9322 KOps/s | 35.5439 KOps/s | $\color{#35bf28}+1.09\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 41.4570μs | 17.2470μs | 57.9810 KOps/s | 57.1681 KOps/s | $\color{#35bf28}+1.42\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 51.0850μs | 18.2760μs | 54.7166 KOps/s | 51.9655 KOps/s | $\textbf{\color{#35bf28}+5.29\\%}$ | | test_step_mdp_speed[False-True-True-False-False] | 34.6940μs | 10.9272μs | 91.5147 KOps/s | 90.1554 KOps/s | $\color{#35bf28}+1.51\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 81.2600μs | 29.0242μs | 34.4540 KOps/s | 34.1128 KOps/s | $\color{#35bf28}+1.00\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 49.3120μs | 18.8854μs | 52.9511 KOps/s | 53.0107 KOps/s | $\color{#d91a1a}-0.11\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 52.7980μs | 20.1873μs | 49.5360 KOps/s | 48.7566 KOps/s | $\color{#35bf28}+1.60\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 51.5650μs | 12.3801μs | 80.7748 KOps/s | 80.3086 KOps/s | $\color{#35bf28}+0.58\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.8967ms | 30.9925μs | 32.2659 KOps/s | 31.0675 KOps/s | $\color{#35bf28}+3.86\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 51.4550μs | 20.2505μs | 49.3815 KOps/s | 49.2781 KOps/s | $\color{#35bf28}+0.21\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 46.9570μs | 20.0387μs | 49.9035 KOps/s | 52.4627 KOps/s | $\color{#d91a1a}-4.88\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 31.9390μs | 12.3637μs | 80.8818 KOps/s | 81.8636 KOps/s | $\color{#d91a1a}-1.20\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 92.4420μs | 31.9937μs | 31.2562 KOps/s | 31.8615 KOps/s | $\color{#d91a1a}-1.90\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 61.3540μs | 21.5181μs | 46.4724 KOps/s | 46.8146 KOps/s | $\color{#d91a1a}-0.73\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 70.1100μs | 21.1061μs | 47.3797 KOps/s | 48.9936 KOps/s | $\color{#d91a1a}-3.29\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 59.5700μs | 13.7608μs | 72.6700 KOps/s | 74.6654 KOps/s | $\color{#d91a1a}-2.67\\%$ | | test_values[generalized_advantage_estimate-True-True] | 11.0288ms | 9.3864ms | 106.5366 Ops/s | 102.0460 Ops/s | $\color{#35bf28}+4.40\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 37.5237ms | 35.6793ms | 28.0274 Ops/s | 28.1524 Ops/s | $\color{#d91a1a}-0.44\\%$ | | test_values[td0_return_estimate-False-False] | 0.2342ms | 0.1722ms | 5.8082 KOps/s | 5.8659 KOps/s | $\color{#d91a1a}-0.98\\%$ | | test_values[td1_return_estimate-False-False] | 27.2885ms | 23.5235ms | 42.5106 Ops/s | 41.1189 Ops/s | $\color{#35bf28}+3.38\\%$ | | test_values[vec_td1_return_estimate-False-False] | 37.8276ms | 35.5327ms | 28.1431 Ops/s | 28.1477 Ops/s | $\color{#d91a1a}-0.02\\%$ | | test_values[td_lambda_return_estimate-True-False] | 38.0143ms | 33.8244ms | 29.5645 Ops/s | 28.6514 Ops/s | $\color{#35bf28}+3.19\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 38.3904ms | 35.6503ms | 28.0502 Ops/s | 27.9300 Ops/s | $\color{#35bf28}+0.43\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 11.5350ms | 8.1793ms | 122.2602 Ops/s | 117.8315 Ops/s | $\color{#35bf28}+3.76\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.3461ms | 2.0307ms | 492.4437 Ops/s | 507.8029 Ops/s | $\color{#d91a1a}-3.02\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4503ms | 0.3573ms | 2.7989 KOps/s | 2.7559 KOps/s | $\color{#35bf28}+1.56\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 50.0232ms | 48.2641ms | 20.7193 Ops/s | 22.4025 Ops/s | $\textbf{\color{#d91a1a}-7.51\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 4.1141ms | 3.0378ms | 329.1853 Ops/s | 326.4260 Ops/s | $\color{#35bf28}+0.85\\%$ | | test_dqn_speed | 1.8029ms | 1.3729ms | 728.4077 Ops/s | 720.3764 Ops/s | $\color{#35bf28}+1.11\\%$ | | test_ddpg_speed | 3.7556ms | 2.9073ms | 343.9584 Ops/s | 342.4722 Ops/s | $\color{#35bf28}+0.43\\%$ | | test_sac_speed | 10.6448ms | 8.4779ms | 117.9531 Ops/s | 115.1256 Ops/s | $\color{#35bf28}+2.46\\%$ | | test_redq_speed | 15.6624ms | 13.7815ms | 72.5611 Ops/s | 71.5106 Ops/s | $\color{#35bf28}+1.47\\%$ | | test_redq_deprec_speed | 16.9362ms | 13.2548ms | 75.4443 Ops/s | 71.5093 Ops/s | $\textbf{\color{#35bf28}+5.50\\%}$ | | test_td3_speed | 9.0144ms | 8.4279ms | 118.6540 Ops/s | 118.1054 Ops/s | $\color{#35bf28}+0.46\\%$ | | test_cql_speed | 39.5668ms | 36.7201ms | 27.2330 Ops/s | 26.8929 Ops/s | $\color{#35bf28}+1.26\\%$ | | test_a2c_speed | 8.7339ms | 7.4706ms | 133.8575 Ops/s | 133.8022 Ops/s | $\color{#35bf28}+0.04\\%$ | | test_ppo_speed | 8.7883ms | 7.7140ms | 129.6346 Ops/s | 129.3882 Ops/s | $\color{#35bf28}+0.19\\%$ | | test_reinforce_speed | 8.1635ms | 6.5902ms | 151.7411 Ops/s | 152.4676 Ops/s | $\color{#d91a1a}-0.48\\%$ | | test_iql_speed | 34.0641ms | 32.6554ms | 30.6228 Ops/s | 30.4314 Ops/s | $\color{#35bf28}+0.63\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.5214ms | 4.8391ms | 206.6500 Ops/s | 207.8897 Ops/s | $\color{#d91a1a}-0.60\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6998ms | 0.4806ms | 2.0807 KOps/s | 2.0830 KOps/s | $\color{#d91a1a}-0.11\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7346ms | 0.4582ms | 2.1822 KOps/s | 2.2065 KOps/s | $\color{#d91a1a}-1.10\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 7.2749ms | 4.8672ms | 205.4572 Ops/s | 208.2311 Ops/s | $\color{#d91a1a}-1.33\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.2898ms | 0.4719ms | 2.1192 KOps/s | 2.1209 KOps/s | $\color{#d91a1a}-0.08\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7041ms | 0.4540ms | 2.2026 KOps/s | 2.1922 KOps/s | $\color{#35bf28}+0.48\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.3873ms | 1.7027ms | 587.3171 Ops/s | 583.0057 Ops/s | $\color{#35bf28}+0.74\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.1081ms | 1.6116ms | 620.5166 Ops/s | 617.0233 Ops/s | $\color{#35bf28}+0.57\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 7.7446ms | 5.0120ms | 199.5222 Ops/s | 196.7061 Ops/s | $\color{#35bf28}+1.43\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9452ms | 0.6207ms | 1.6112 KOps/s | 1.5752 KOps/s | $\color{#35bf28}+2.28\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9481ms | 0.5921ms | 1.6890 KOps/s | 1.6643 KOps/s | $\color{#35bf28}+1.48\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 7.5752ms | 4.8543ms | 206.0047 Ops/s | 197.1966 Ops/s | $\color{#35bf28}+4.47\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.8070ms | 0.4864ms | 2.0560 KOps/s | 2.0267 KOps/s | $\color{#35bf28}+1.45\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.5531ms | 0.4576ms | 2.1853 KOps/s | 2.1452 KOps/s | $\color{#35bf28}+1.87\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 7.7724ms | 4.8434ms | 206.4668 Ops/s | 201.9991 Ops/s | $\color{#35bf28}+2.21\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.8850ms | 0.4760ms | 2.1010 KOps/s | 2.0377 KOps/s | $\color{#35bf28}+3.11\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6180ms | 0.4551ms | 2.1972 KOps/s | 2.0790 KOps/s | $\textbf{\color{#35bf28}+5.68\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 8.1812ms | 5.1470ms | 194.2869 Ops/s | 195.1857 Ops/s | $\color{#d91a1a}-0.46\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9038ms | 0.6429ms | 1.5555 KOps/s | 1.5591 KOps/s | $\color{#d91a1a}-0.23\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8028ms | 0.6100ms | 1.6394 KOps/s | 1.6747 KOps/s | $\color{#d91a1a}-2.10\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1369s | 8.6721ms | 115.3125 Ops/s | 117.8260 Ops/s | $\color{#d91a1a}-2.13\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 17.8394ms | 12.9470ms | 77.2382 Ops/s | 77.4138 Ops/s | $\color{#d91a1a}-0.23\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 1.8724ms | 1.0813ms | 924.7927 Ops/s | 888.3999 Ops/s | $\color{#35bf28}+4.10\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1123s | 5.8954ms | 169.6225 Ops/s | 165.6868 Ops/s | $\color{#35bf28}+2.38\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 17.3978ms | 12.8878ms | 77.5925 Ops/s | 70.5878 Ops/s | $\textbf{\color{#35bf28}+9.92\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 4.6885ms | 1.1643ms | 858.9083 Ops/s | 907.7016 Ops/s | $\textbf{\color{#d91a1a}-5.38\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1097s | 6.0021ms | 166.6076 Ops/s | 164.5490 Ops/s | $\color{#35bf28}+1.25\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 0.1223s | 15.1826ms | 65.8648 Ops/s | 77.3056 Ops/s | $\textbf{\color{#d91a1a}-14.80\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 4.9630ms | 1.3475ms | 742.1381 Ops/s | 728.1222 Ops/s | $\color{#35bf28}+1.92\\%$ |
github-actions[bot] commented 2 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1066s | 0.1063s | 9.4043 Ops/s | 9.3433 Ops/s | $\color{#35bf28}+0.65\\%$ | | test_sync | 96.2661ms | 95.4303ms | 10.4789 Ops/s | 10.4543 Ops/s | $\color{#35bf28}+0.23\\%$ | | test_async | 0.2341s | 89.5024ms | 11.1729 Ops/s | 11.2280 Ops/s | $\color{#d91a1a}-0.49\\%$ | | test_single_pixels | 0.1180s | 0.1169s | 8.5522 Ops/s | 8.5604 Ops/s | $\color{#d91a1a}-0.10\\%$ | | test_sync_pixels | 76.1508ms | 73.9376ms | 13.5249 Ops/s | 13.1044 Ops/s | $\color{#35bf28}+3.21\\%$ | | test_async_pixels | 0.1379s | 68.6228ms | 14.5724 Ops/s | 14.3987 Ops/s | $\color{#35bf28}+1.21\\%$ | | test_simple | 0.8648s | 0.7876s | 1.2696 Ops/s | 1.2506 Ops/s | $\color{#35bf28}+1.53\\%$ | | test_transformed | 1.0962s | 1.0215s | 0.9790 Ops/s | 0.9847 Ops/s | $\color{#d91a1a}-0.58\\%$ | | test_serial | 2.2929s | 2.2202s | 0.4504 Ops/s | 0.4423 Ops/s | $\color{#35bf28}+1.84\\%$ | | test_parallel | 2.0312s | 1.9745s | 0.5065 Ops/s | 0.5047 Ops/s | $\color{#35bf28}+0.35\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 99.3220μs | 36.3516μs | 27.5091 KOps/s | 27.0616 KOps/s | $\color{#35bf28}+1.65\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 0.1104ms | 20.5263μs | 48.7181 KOps/s | 47.4845 KOps/s | $\color{#35bf28}+2.60\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 0.1542ms | 20.2946μs | 49.2742 KOps/s | 48.0702 KOps/s | $\color{#35bf28}+2.50\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 31.5200μs | 11.6346μs | 85.9506 KOps/s | 84.8336 KOps/s | $\color{#35bf28}+1.32\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 0.1108ms | 38.0196μs | 26.3022 KOps/s | 25.6026 KOps/s | $\color{#35bf28}+2.73\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 59.8810μs | 22.2470μs | 44.9500 KOps/s | 42.6712 KOps/s | $\textbf{\color{#35bf28}+5.34\\%}$ | | test_step_mdp_speed[True-True-False-False-True] | 47.9710μs | 21.9700μs | 45.5165 KOps/s | 43.2025 KOps/s | $\textbf{\color{#35bf28}+5.36\\%}$ | | test_step_mdp_speed[True-True-False-False-False] | 36.6400μs | 13.5789μs | 73.6437 KOps/s | 71.1043 KOps/s | $\color{#35bf28}+3.57\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 67.3220μs | 39.9481μs | 25.0325 KOps/s | 24.3005 KOps/s | $\color{#35bf28}+3.01\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 50.8610μs | 24.3726μs | 41.0297 KOps/s | 39.3283 KOps/s | $\color{#35bf28}+4.33\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 43.6500μs | 22.0349μs | 45.3826 KOps/s | 43.3904 KOps/s | $\color{#35bf28}+4.59\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 30.1110μs | 13.8027μs | 72.4494 KOps/s | 71.4333 KOps/s | $\color{#35bf28}+1.42\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 85.8220μs | 42.1924μs | 23.7010 KOps/s | 23.3410 KOps/s | $\color{#35bf28}+1.54\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 45.0610μs | 26.6110μs | 37.5785 KOps/s | 36.6565 KOps/s | $\color{#35bf28}+2.52\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 49.9310μs | 24.1825μs | 41.3522 KOps/s | 40.2984 KOps/s | $\color{#35bf28}+2.61\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 42.2700μs | 15.7467μs | 63.5055 KOps/s | 62.4376 KOps/s | $\color{#35bf28}+1.71\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 0.1971ms | 40.2311μs | 24.8564 KOps/s | 24.3288 KOps/s | $\color{#35bf28}+2.17\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 0.1925ms | 24.5908μs | 40.6656 KOps/s | 39.6123 KOps/s | $\color{#35bf28}+2.66\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 0.2229ms | 26.6730μs | 37.4911 KOps/s | 36.5505 KOps/s | $\color{#35bf28}+2.57\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 98.1610μs | 15.4285μs | 64.8149 KOps/s | 63.6780 KOps/s | $\color{#35bf28}+1.79\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 69.6520μs | 42.5364μs | 23.5093 KOps/s | 23.3371 KOps/s | $\color{#35bf28}+0.74\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 0.2098ms | 26.7241μs | 37.4194 KOps/s | 36.7021 KOps/s | $\color{#35bf28}+1.95\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 49.4110μs | 28.8630μs | 34.6464 KOps/s | 34.1548 KOps/s | $\color{#35bf28}+1.44\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 0.1611ms | 17.8598μs | 55.9917 KOps/s | 55.9093 KOps/s | $\color{#35bf28}+0.15\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 4.2464ms | 45.4508μs | 22.0018 KOps/s | 21.8380 KOps/s | $\color{#35bf28}+0.75\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 58.7410μs | 29.2350μs | 34.2056 KOps/s | 33.7155 KOps/s | $\color{#35bf28}+1.45\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 47.4710μs | 29.0612μs | 34.4102 KOps/s | 34.1912 KOps/s | $\color{#35bf28}+0.64\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 54.5910μs | 17.7601μs | 56.3061 KOps/s | 55.9531 KOps/s | $\color{#35bf28}+0.63\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 70.7720μs | 46.7244μs | 21.4021 KOps/s | 21.4793 KOps/s | $\color{#d91a1a}-0.36\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 89.3520μs | 31.4919μs | 31.7542 KOps/s | 31.7218 KOps/s | $\color{#35bf28}+0.10\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 48.9710μs | 30.5925μs | 32.6877 KOps/s | 32.2621 KOps/s | $\color{#35bf28}+1.32\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 46.8610μs | 19.6031μs | 51.0123 KOps/s | 50.5178 KOps/s | $\color{#35bf28}+0.98\\%$ | | test_values[generalized_advantage_estimate-True-True] | 25.5960ms | 24.2111ms | 41.3034 Ops/s | 40.8663 Ops/s | $\color{#35bf28}+1.07\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 92.7786ms | 2.7522ms | 363.3490 Ops/s | 352.2619 Ops/s | $\color{#35bf28}+3.15\\%$ | | test_values[td0_return_estimate-False-False] | 89.8930μs | 65.1168μs | 15.3570 KOps/s | 15.3743 KOps/s | $\color{#d91a1a}-0.11\\%$ | | test_values[td1_return_estimate-False-False] | 54.4745ms | 53.9639ms | 18.5309 Ops/s | 18.3676 Ops/s | $\color{#35bf28}+0.89\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.2871ms | 1.0793ms | 926.5261 Ops/s | 915.0027 Ops/s | $\color{#35bf28}+1.26\\%$ | | test_values[td_lambda_return_estimate-True-False] | 86.6609ms | 86.1266ms | 11.6108 Ops/s | 11.5495 Ops/s | $\color{#35bf28}+0.53\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.2429ms | 1.0742ms | 930.8927 Ops/s | 927.0705 Ops/s | $\color{#35bf28}+0.41\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 24.3280ms | 23.9260ms | 41.7956 Ops/s | 40.6299 Ops/s | $\color{#35bf28}+2.87\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9722ms | 0.7189ms | 1.3909 KOps/s | 1.3717 KOps/s | $\color{#35bf28}+1.40\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.8292ms | 0.6663ms | 1.5008 KOps/s | 1.4787 KOps/s | $\color{#35bf28}+1.49\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.6188ms | 1.4600ms | 684.9349 Ops/s | 679.7104 Ops/s | $\color{#35bf28}+0.77\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.8505ms | 0.6790ms | 1.4728 KOps/s | 1.4043 KOps/s | $\color{#35bf28}+4.87\\%$ | | test_dqn_speed | 7.9479ms | 1.4384ms | 695.1970 Ops/s | 694.0519 Ops/s | $\color{#35bf28}+0.16\\%$ | | test_ddpg_speed | 3.2032ms | 2.9444ms | 339.6333 Ops/s | 335.1645 Ops/s | $\color{#35bf28}+1.33\\%$ | | test_sac_speed | 0.1062s | 9.1952ms | 108.7525 Ops/s | 117.5763 Ops/s | $\textbf{\color{#d91a1a}-7.50\\%}$ | | test_redq_speed | 11.8472ms | 10.9756ms | 91.1108 Ops/s | 90.3937 Ops/s | $\color{#35bf28}+0.79\\%$ | | test_redq_deprec_speed | 12.1221ms | 11.3171ms | 88.3619 Ops/s | 87.1165 Ops/s | $\color{#35bf28}+1.43\\%$ | | test_td3_speed | 8.3992ms | 8.3142ms | 120.2767 Ops/s | 116.1787 Ops/s | $\color{#35bf28}+3.53\\%$ | | test_cql_speed | 27.7663ms | 26.3347ms | 37.9728 Ops/s | 34.0097 Ops/s | $\textbf{\color{#35bf28}+11.65\\%}$ | | test_a2c_speed | 6.0190ms | 5.7393ms | 174.2386 Ops/s | 171.1350 Ops/s | $\color{#35bf28}+1.81\\%$ | | test_ppo_speed | 6.5467ms | 6.0754ms | 164.5988 Ops/s | 161.2187 Ops/s | $\color{#35bf28}+2.10\\%$ | | test_reinforce_speed | 4.9044ms | 4.6474ms | 215.1728 Ops/s | 211.0269 Ops/s | $\color{#35bf28}+1.96\\%$ | | test_iql_speed | 20.7358ms | 20.0143ms | 49.9644 Ops/s | 48.7838 Ops/s | $\color{#35bf28}+2.42\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.7052ms | 6.5337ms | 153.0537 Ops/s | 149.2710 Ops/s | $\color{#35bf28}+2.53\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.1152s | 0.5981ms | 1.6720 KOps/s | 1.9037 KOps/s | $\textbf{\color{#d91a1a}-12.17\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6749ms | 0.4981ms | 2.0078 KOps/s | 1.9818 KOps/s | $\color{#35bf28}+1.31\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.8742ms | 6.4316ms | 155.4815 Ops/s | 151.3327 Ops/s | $\color{#35bf28}+2.74\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8100ms | 0.5076ms | 1.9702 KOps/s | 1.9492 KOps/s | $\color{#35bf28}+1.08\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7020ms | 0.4913ms | 2.0355 KOps/s | 2.0305 KOps/s | $\color{#35bf28}+0.25\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.1749ms | 1.9464ms | 513.7737 Ops/s | 504.1554 Ops/s | $\color{#35bf28}+1.91\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2410ms | 1.8544ms | 539.2622 Ops/s | 530.1695 Ops/s | $\color{#35bf28}+1.72\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 8.5169ms | 6.7615ms | 147.8957 Ops/s | 146.2500 Ops/s | $\color{#35bf28}+1.13\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.1340s | 0.7770ms | 1.2869 KOps/s | 1.5061 KOps/s | $\textbf{\color{#d91a1a}-14.55\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8333ms | 0.6423ms | 1.5570 KOps/s | 1.5533 KOps/s | $\color{#35bf28}+0.24\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.7545ms | 6.5503ms | 152.6644 Ops/s | 149.1843 Ops/s | $\color{#35bf28}+2.33\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7106ms | 0.5184ms | 1.9291 KOps/s | 1.9253 KOps/s | $\color{#35bf28}+0.20\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6936ms | 0.5023ms | 1.9907 KOps/s | 2.0016 KOps/s | $\color{#d91a1a}-0.54\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.7705ms | 6.4157ms | 155.8678 Ops/s | 151.5038 Ops/s | $\color{#35bf28}+2.88\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.0958ms | 0.5138ms | 1.9464 KOps/s | 1.9437 KOps/s | $\color{#35bf28}+0.14\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6886ms | 0.4909ms | 2.0372 KOps/s | 2.0023 KOps/s | $\color{#35bf28}+1.74\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.9270ms | 6.6999ms | 149.2557 Ops/s | 146.9276 Ops/s | $\color{#35bf28}+1.58\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.9494ms | 0.6801ms | 1.4703 KOps/s | 1.4919 KOps/s | $\color{#d91a1a}-1.45\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8444ms | 0.6591ms | 1.5173 KOps/s | 1.5312 KOps/s | $\color{#d91a1a}-0.90\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1507s | 8.0784ms | 123.7871 Ops/s | 122.1472 Ops/s | $\color{#35bf28}+1.34\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.3873ms | 15.7404ms | 63.5308 Ops/s | 62.0401 Ops/s | $\color{#35bf28}+2.40\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 6.6274ms | 1.3518ms | 739.7556 Ops/s | 795.4765 Ops/s | $\textbf{\color{#d91a1a}-7.00\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1279s | 7.6246ms | 131.1537 Ops/s | 130.0973 Ops/s | $\color{#35bf28}+0.81\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1395s | 18.2733ms | 54.7245 Ops/s | 61.6684 Ops/s | $\textbf{\color{#d91a1a}-11.26\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.0078ms | 1.1186ms | 893.9930 Ops/s | 797.7798 Ops/s | $\textbf{\color{#35bf28}+12.06\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1297s | 7.8363ms | 127.6120 Ops/s | 127.9071 Ops/s | $\color{#d91a1a}-0.23\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.7539ms | 15.8142ms | 63.2342 Ops/s | 61.2385 Ops/s | $\color{#35bf28}+3.26\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 6.3870ms | 1.5135ms | 660.7065 Ops/s | 656.1810 Ops/s | $\color{#35bf28}+0.69\\%$ |