search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.05k stars 273 forks source link

[Versioning] Bump torch 2.0 as minimal version #2200

Closed vmoens closed 1 month ago

vmoens commented 1 month ago

We want torchrl to be compatible with previous versions of PT, mainly because some other libs require older versions of the it. 1.13 becomes really old so we can bump to 2.0. This PR resolved the olddeps tests for that bumb

pytorch-bot[bot] commented 1 month ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2200

Note: Links to docs will display an error until the docs builds have been completed.

:x: 6 New Failures

As of commit 9f6df1d873cf162f5bfebd0299f1e204f06301fe with merge base 4d37ee18ebb0e77c8dadd5532d370162f395d605 (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2200#26022760360) ([gh](https://github.com/pytorch/rl/actions/runs/9448537724/job/26022760360)) `RuntimeError: Command docker exec -t f6a1a06d54cb6df82f962e01890cfbbcc449880163586719f32eac9ba0d62d5c /exec failed with exit code 139` * [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2200#26022784773) ([gh](https://github.com/pytorch/rl/actions/runs/9448537728/job/26022784773)) `##[error]The operation was canceled.` * [Libs Tests on Linux / unittests-minari (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2200#26022785028) ([gh](https://github.com/pytorch/rl/actions/runs/9448537728/job/26022785028)) `test/test_libs.py::TestMinari::test_load[antmaze-umaze-diverse-v0-True]` * [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2200#26022792534) ([gh](https://github.com/pytorch/rl/actions/runs/9448537732/job/26022792534)) `##[error]The operation was canceled.` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2200#26022792203) ([gh](https://github.com/pytorch/rl/actions/runs/9448537732/job/26022792203)) `RuntimeError: Command docker exec -t c8ffb552b987bbd42a0f6d275583cc13054e52436ad369cc26fd7fa9f40445ad /exec failed with exit code 1` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2200#26022763408) ([gh](https://github.com/pytorch/rl/actions/runs/9448537735/job/26022763408)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1339s | 61.7577ms | 16.1923 Ops/s | 17.3188 Ops/s | $\textbf{\color{#d91a1a}-6.50\\%}$ | | test_sync | 49.4049ms | 32.4192ms | 30.8459 Ops/s | 28.4305 Ops/s | $\textbf{\color{#35bf28}+8.50\\%}$ | | test_async | 58.1103ms | 29.3548ms | 34.0660 Ops/s | 33.4532 Ops/s | $\color{#35bf28}+1.83\\%$ | | test_simple | 0.4872s | 0.4108s | 2.4340 Ops/s | 2.5019 Ops/s | $\color{#d91a1a}-2.71\\%$ | | test_transformed | 0.5470s | 0.5407s | 1.8495 Ops/s | 1.8674 Ops/s | $\color{#d91a1a}-0.96\\%$ | | test_serial | 1.3810s | 1.3037s | 0.7671 Ops/s | 0.7755 Ops/s | $\color{#d91a1a}-1.09\\%$ | | test_parallel | 1.1478s | 1.0835s | 0.9230 Ops/s | 0.9205 Ops/s | $\color{#35bf28}+0.27\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1809ms | 21.5474μs | 46.4092 KOps/s | 45.9907 KOps/s | $\color{#35bf28}+0.91\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 38.8430μs | 13.1628μs | 75.9720 KOps/s | 74.7350 KOps/s | $\color{#35bf28}+1.66\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 53.1510μs | 12.6976μs | 78.7552 KOps/s | 79.6398 KOps/s | $\color{#d91a1a}-1.11\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 33.0820μs | 7.7115μs | 129.6773 KOps/s | 131.4336 KOps/s | $\color{#d91a1a}-1.34\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 58.5590μs | 23.0446μs | 43.3940 KOps/s | 44.0017 KOps/s | $\color{#d91a1a}-1.38\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 58.0080μs | 14.3936μs | 69.4752 KOps/s | 70.4372 KOps/s | $\color{#d91a1a}-1.37\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 71.3440μs | 14.0278μs | 71.2871 KOps/s | 72.7345 KOps/s | $\color{#d91a1a}-1.99\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 83.5870μs | 8.8537μs | 112.9465 KOps/s | 113.4952 KOps/s | $\color{#d91a1a}-0.48\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 70.1610μs | 24.4269μs | 40.9384 KOps/s | 41.7759 KOps/s | $\color{#d91a1a}-2.00\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 46.8380μs | 15.7657μs | 63.4289 KOps/s | 64.5704 KOps/s | $\color{#d91a1a}-1.77\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 52.6380μs | 13.8604μs | 72.1478 KOps/s | 72.8182 KOps/s | $\color{#d91a1a}-0.92\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 96.3300μs | 8.9155μs | 112.1642 KOps/s | 114.0357 KOps/s | $\color{#d91a1a}-1.64\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 89.6380μs | 25.6139μs | 39.0413 KOps/s | 40.0836 KOps/s | $\color{#d91a1a}-2.60\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 0.1198ms | 17.5721μs | 56.9084 KOps/s | 59.8995 KOps/s | $\color{#d91a1a}-4.99\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 78.4360μs | 15.1567μs | 65.9773 KOps/s | 67.5084 KOps/s | $\color{#d91a1a}-2.27\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 36.6680μs | 10.1058μs | 98.9530 KOps/s | 100.5936 KOps/s | $\color{#d91a1a}-1.63\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 61.3750μs | 24.3699μs | 41.0343 KOps/s | 41.5651 KOps/s | $\color{#d91a1a}-1.28\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 65.1420μs | 15.7770μs | 63.3834 KOps/s | 64.7396 KOps/s | $\color{#d91a1a}-2.09\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 69.0490μs | 16.1026μs | 62.1017 KOps/s | 62.2847 KOps/s | $\color{#d91a1a}-0.29\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 40.1950μs | 10.1370μs | 98.6490 KOps/s | 99.7542 KOps/s | $\color{#d91a1a}-1.11\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 67.0460μs | 25.5299μs | 39.1697 KOps/s | 40.0909 KOps/s | $\color{#d91a1a}-2.30\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 47.8600μs | 16.9854μs | 58.8742 KOps/s | 60.3890 KOps/s | $\color{#d91a1a}-2.51\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 46.6470μs | 17.2460μs | 57.9845 KOps/s | 58.7943 KOps/s | $\color{#d91a1a}-1.38\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 0.1481ms | 11.3605μs | 88.0242 KOps/s | 90.2824 KOps/s | $\color{#d91a1a}-2.50\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 88.0450μs | 26.7820μs | 37.3384 KOps/s | 37.5733 KOps/s | $\color{#d91a1a}-0.63\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 45.3550μs | 18.2127μs | 54.9068 KOps/s | 56.0896 KOps/s | $\color{#d91a1a}-2.11\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 51.5060μs | 17.4417μs | 57.3339 KOps/s | 58.6907 KOps/s | $\color{#d91a1a}-2.31\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 58.4690μs | 11.2517μs | 88.8752 KOps/s | 90.3187 KOps/s | $\color{#d91a1a}-1.60\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 60.1020μs | 28.6121μs | 34.9503 KOps/s | 35.9664 KOps/s | $\color{#d91a1a}-2.83\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 77.2940μs | 19.3896μs | 51.5740 KOps/s | 52.7573 KOps/s | $\color{#d91a1a}-2.24\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 77.0140μs | 18.2997μs | 54.6458 KOps/s | 55.7325 KOps/s | $\color{#d91a1a}-1.95\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 38.3520μs | 12.3160μs | 81.1952 KOps/s | 83.3272 KOps/s | $\color{#d91a1a}-2.56\\%$ | | test_values[generalized_advantage_estimate-True-True] | 11.8079ms | 9.9031ms | 100.9789 Ops/s | 101.0508 Ops/s | $\color{#d91a1a}-0.07\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 42.3641ms | 36.1823ms | 27.6378 Ops/s | 28.0120 Ops/s | $\color{#d91a1a}-1.34\\%$ | | test_values[td0_return_estimate-False-False] | 0.2267ms | 0.1728ms | 5.7877 KOps/s | 5.6213 KOps/s | $\color{#35bf28}+2.96\\%$ | | test_values[td1_return_estimate-False-False] | 25.6394ms | 24.4264ms | 40.9394 Ops/s | 41.3033 Ops/s | $\color{#d91a1a}-0.88\\%$ | | test_values[vec_td1_return_estimate-False-False] | 38.5122ms | 35.6700ms | 28.0348 Ops/s | 28.1335 Ops/s | $\color{#d91a1a}-0.35\\%$ | | test_values[td_lambda_return_estimate-True-False] | 36.2900ms | 35.0573ms | 28.5248 Ops/s | 28.7799 Ops/s | $\color{#d91a1a}-0.89\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 37.8334ms | 35.7254ms | 27.9913 Ops/s | 27.9075 Ops/s | $\color{#35bf28}+0.30\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.7078ms | 8.3959ms | 119.1055 Ops/s | 119.5596 Ops/s | $\color{#d91a1a}-0.38\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.0253ms | 1.8048ms | 554.0838 Ops/s | 505.7999 Ops/s | $\textbf{\color{#35bf28}+9.55\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.5564ms | 0.3649ms | 2.7403 KOps/s | 2.7686 KOps/s | $\color{#d91a1a}-1.02\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 48.4308ms | 46.6602ms | 21.4316 Ops/s | 22.0218 Ops/s | $\color{#d91a1a}-2.68\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.8536ms | 3.0844ms | 324.2146 Ops/s | 318.6338 Ops/s | $\color{#35bf28}+1.75\\%$ | | test_dqn_speed | 1.6687ms | 1.3868ms | 721.0659 Ops/s | 726.8686 Ops/s | $\color{#d91a1a}-0.80\\%$ | | test_ddpg_speed | 3.9067ms | 2.9850ms | 335.0031 Ops/s | 342.4424 Ops/s | $\color{#d91a1a}-2.17\\%$ | | test_sac_speed | 10.6586ms | 9.4549ms | 105.7658 Ops/s | 110.4236 Ops/s | $\color{#d91a1a}-4.22\\%$ | | test_redq_speed | 15.6494ms | 14.0619ms | 71.1139 Ops/s | 70.3905 Ops/s | $\color{#35bf28}+1.03\\%$ | | test_redq_deprec_speed | 16.7540ms | 15.1040ms | 66.2077 Ops/s | 64.6804 Ops/s | $\color{#35bf28}+2.36\\%$ | | test_td3_speed | 9.4796ms | 8.9123ms | 112.2040 Ops/s | 111.8831 Ops/s | $\color{#35bf28}+0.29\\%$ | | test_cql_speed | 39.3945ms | 37.6849ms | 26.5358 Ops/s | 26.6116 Ops/s | $\color{#d91a1a}-0.28\\%$ | | test_a2c_speed | 8.7931ms | 7.7621ms | 128.8306 Ops/s | 128.4020 Ops/s | $\color{#35bf28}+0.33\\%$ | | test_ppo_speed | 9.3472ms | 8.0532ms | 124.1750 Ops/s | 121.0039 Ops/s | $\color{#35bf28}+2.62\\%$ | | test_reinforce_speed | 7.7940ms | 6.9041ms | 144.8406 Ops/s | 142.7647 Ops/s | $\color{#35bf28}+1.45\\%$ | | test_iql_speed | 34.9789ms | 33.4058ms | 29.9349 Ops/s | 29.8611 Ops/s | $\color{#35bf28}+0.25\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.7418ms | 3.8398ms | 260.4331 Ops/s | 271.7949 Ops/s | $\color{#d91a1a}-4.18\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.2064ms | 0.5164ms | 1.9366 KOps/s | 1.9549 KOps/s | $\color{#d91a1a}-0.93\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.8296ms | 0.4958ms | 2.0168 KOps/s | 1.9943 KOps/s | $\color{#35bf28}+1.13\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.3055ms | 3.7863ms | 264.1085 Ops/s | 267.0596 Ops/s | $\color{#d91a1a}-1.11\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8694ms | 0.5089ms | 1.9652 KOps/s | 1.9083 KOps/s | $\color{#35bf28}+2.98\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.8115ms | 0.4933ms | 2.0272 KOps/s | 2.0634 KOps/s | $\color{#d91a1a}-1.76\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.4528ms | 1.7562ms | 569.3971 Ops/s | 583.7801 Ops/s | $\color{#d91a1a}-2.46\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 0.1135s | 1.8668ms | 535.6755 Ops/s | 539.4917 Ops/s | $\color{#d91a1a}-0.71\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.2227ms | 4.1803ms | 239.2172 Ops/s | 247.3397 Ops/s | $\color{#d91a1a}-3.28\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8030ms | 0.6359ms | 1.5725 KOps/s | 1.5685 KOps/s | $\color{#35bf28}+0.26\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 1.2169ms | 0.6159ms | 1.6236 KOps/s | 1.6409 KOps/s | $\color{#d91a1a}-1.05\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.9726ms | 3.9170ms | 255.3003 Ops/s | 266.2821 Ops/s | $\color{#d91a1a}-4.12\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.1089s | 0.6607ms | 1.5136 KOps/s | 1.9426 KOps/s | $\textbf{\color{#d91a1a}-22.09\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6241ms | 0.4883ms | 2.0478 KOps/s | 2.0071 KOps/s | $\color{#35bf28}+2.03\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.6263ms | 3.9169ms | 255.3043 Ops/s | 260.8930 Ops/s | $\color{#d91a1a}-2.14\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6711ms | 0.5214ms | 1.9179 KOps/s | 1.9435 KOps/s | $\color{#d91a1a}-1.32\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 4.1749ms | 0.5085ms | 1.9667 KOps/s | 2.0514 KOps/s | $\color{#d91a1a}-4.13\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.5859ms | 4.0034ms | 249.7878 Ops/s | 250.9764 Ops/s | $\color{#d91a1a}-0.47\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9674ms | 0.6479ms | 1.5434 KOps/s | 1.5530 KOps/s | $\color{#d91a1a}-0.62\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 1.1124ms | 0.6219ms | 1.6079 KOps/s | 1.6107 KOps/s | $\color{#d91a1a}-0.17\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1097s | 5.9489ms | 168.0988 Ops/s | 158.1434 Ops/s | $\textbf{\color{#35bf28}+6.30\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 15.5113ms | 12.9034ms | 77.4988 Ops/s | 76.6938 Ops/s | $\color{#35bf28}+1.05\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 1.7690ms | 1.0795ms | 926.3281 Ops/s | 898.4858 Ops/s | $\color{#35bf28}+3.10\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1114s | 7.9239ms | 126.2003 Ops/s | 125.3807 Ops/s | $\color{#35bf28}+0.65\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 15.8142ms | 12.7990ms | 78.1313 Ops/s | 76.0394 Ops/s | $\color{#35bf28}+2.75\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.5793ms | 1.0770ms | 928.4792 Ops/s | 894.0604 Ops/s | $\color{#35bf28}+3.85\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1068s | 5.9648ms | 167.6510 Ops/s | 152.1662 Ops/s | $\textbf{\color{#35bf28}+10.18\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.4561ms | 13.1827ms | 75.8572 Ops/s | 73.3513 Ops/s | $\color{#35bf28}+3.42\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.7671ms | 1.2379ms | 807.8427 Ops/s | 784.4879 Ops/s | $\color{#35bf28}+2.98\\%$ |
github-actions[bot] commented 1 month ago

$\color{#35bf28}\textsf{\Large\✔\kern{0.2cm}\normalsize OK}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}0$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ---------------------------------- | | test_single | 0.1151s | 0.1146s | 8.7259 Ops/s | 8.6541 Ops/s | $\color{#35bf28}+0.83\\%$ | | test_sync | 0.1041s | 0.1008s | 9.9224 Ops/s | 9.7055 Ops/s | $\color{#35bf28}+2.23\\%$ | | test_async | 0.1944s | 78.3980ms | 12.7554 Ops/s | 13.3184 Ops/s | $\color{#d91a1a}-4.23\\%$ | | test_single_pixels | 0.1279s | 0.1260s | 7.9358 Ops/s | 7.9415 Ops/s | $\color{#d91a1a}-0.07\\%$ | | test_sync_pixels | 85.2765ms | 83.4391ms | 11.9848 Ops/s | 12.2301 Ops/s | $\color{#d91a1a}-2.01\\%$ | | test_async_pixels | 0.1499s | 68.8523ms | 14.5238 Ops/s | 14.7605 Ops/s | $\color{#d91a1a}-1.60\\%$ | | test_simple | 0.8648s | 0.8017s | 1.2473 Ops/s | 1.2389 Ops/s | $\color{#35bf28}+0.68\\%$ | | test_transformed | 1.1328s | 1.0703s | 0.9343 Ops/s | 0.9502 Ops/s | $\color{#d91a1a}-1.67\\%$ | | test_serial | 2.5013s | 2.4427s | 0.4094 Ops/s | 0.4004 Ops/s | $\color{#35bf28}+2.23\\%$ | | test_parallel | 2.4050s | 2.3389s | 0.4276 Ops/s | 0.4237 Ops/s | $\color{#35bf28}+0.92\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 99.4020μs | 32.7338μs | 30.5495 KOps/s | 29.5383 KOps/s | $\color{#35bf28}+3.42\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 45.9210μs | 19.7832μs | 50.5479 KOps/s | 49.4592 KOps/s | $\color{#35bf28}+2.20\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 41.3010μs | 18.8601μs | 53.0219 KOps/s | 51.8727 KOps/s | $\color{#35bf28}+2.22\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 27.7000μs | 11.1807μs | 89.4398 KOps/s | 85.6113 KOps/s | $\color{#35bf28}+4.47\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 53.2610μs | 34.3846μs | 29.0827 KOps/s | 28.1434 KOps/s | $\color{#35bf28}+3.34\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 48.6210μs | 21.6167μs | 46.2605 KOps/s | 45.0905 KOps/s | $\color{#35bf28}+2.59\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 40.0310μs | 20.8743μs | 47.9058 KOps/s | 46.9657 KOps/s | $\color{#35bf28}+2.00\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 39.1010μs | 13.1526μs | 76.0307 KOps/s | 74.7387 KOps/s | $\color{#35bf28}+1.73\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 81.9020μs | 37.0383μs | 26.9991 KOps/s | 26.9452 KOps/s | $\color{#35bf28}+0.20\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 49.1510μs | 23.5274μs | 42.5037 KOps/s | 41.7102 KOps/s | $\color{#35bf28}+1.90\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 44.6810μs | 20.4920μs | 48.7996 KOps/s | 46.7869 KOps/s | $\color{#35bf28}+4.30\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 28.7410μs | 13.0873μs | 76.4098 KOps/s | 72.4913 KOps/s | $\textbf{\color{#35bf28}+5.41\\%}$ | | test_step_mdp_speed[True-False-False-True-True] | 0.1029ms | 38.4342μs | 26.0185 KOps/s | 25.3962 KOps/s | $\color{#35bf28}+2.45\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 42.9810μs | 25.1641μs | 39.7391 KOps/s | 38.0813 KOps/s | $\color{#35bf28}+4.35\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 41.3910μs | 22.6077μs | 44.2327 KOps/s | 43.6522 KOps/s | $\color{#35bf28}+1.33\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 31.1410μs | 14.8034μs | 67.5523 KOps/s | 64.3311 KOps/s | $\textbf{\color{#35bf28}+5.01\\%}$ | | test_step_mdp_speed[False-True-True-True-True] | 59.9720μs | 36.5006μs | 27.3968 KOps/s | 26.0927 KOps/s | $\color{#35bf28}+5.00\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 45.1810μs | 23.7143μs | 42.1687 KOps/s | 40.5208 KOps/s | $\color{#35bf28}+4.07\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 55.2110μs | 25.1253μs | 39.8005 KOps/s | 38.6459 KOps/s | $\color{#35bf28}+2.99\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 41.1510μs | 14.8707μs | 67.2463 KOps/s | 63.3859 KOps/s | $\textbf{\color{#35bf28}+6.09\\%}$ | | test_step_mdp_speed[False-True-False-True-True] | 79.3400μs | 38.3885μs | 26.0495 KOps/s | 25.0937 KOps/s | $\color{#35bf28}+3.81\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 43.7900μs | 24.9849μs | 40.0242 KOps/s | 37.8182 KOps/s | $\textbf{\color{#35bf28}+5.83\\%}$ | | test_step_mdp_speed[False-True-False-False-True] | 47.8910μs | 27.0063μs | 37.0284 KOps/s | 35.4407 KOps/s | $\color{#35bf28}+4.48\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 41.7810μs | 16.5731μs | 60.3386 KOps/s | 56.0296 KOps/s | $\textbf{\color{#35bf28}+7.69\\%}$ | | test_step_mdp_speed[False-False-True-True-True] | 63.6720μs | 40.5849μs | 24.6397 KOps/s | 23.9320 KOps/s | $\color{#35bf28}+2.96\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 54.2510μs | 27.6500μs | 36.1664 KOps/s | 35.6198 KOps/s | $\color{#35bf28}+1.53\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 52.7610μs | 27.0506μs | 36.9678 KOps/s | 35.9235 KOps/s | $\color{#35bf28}+2.91\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 34.0710μs | 16.6919μs | 59.9095 KOps/s | 56.3912 KOps/s | $\textbf{\color{#35bf28}+6.24\\%}$ | | test_step_mdp_speed[False-False-False-True-True] | 69.8910μs | 43.5545μs | 22.9597 KOps/s | 22.7383 KOps/s | $\color{#35bf28}+0.97\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 55.3210μs | 29.0103μs | 34.4706 KOps/s | 32.7222 KOps/s | $\textbf{\color{#35bf28}+5.34\\%}$ | | test_step_mdp_speed[False-False-False-False-True] | 50.3710μs | 28.6428μs | 34.9127 KOps/s | 34.0516 KOps/s | $\color{#35bf28}+2.53\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 36.3210μs | 18.2864μs | 54.6855 KOps/s | 52.3773 KOps/s | $\color{#35bf28}+4.41\\%$ | | test_values[generalized_advantage_estimate-True-True] | 24.3595ms | 23.5025ms | 42.5486 Ops/s | 41.1631 Ops/s | $\color{#35bf28}+3.37\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 90.2655ms | 2.6906ms | 371.6686 Ops/s | 362.5691 Ops/s | $\color{#35bf28}+2.51\\%$ | | test_values[td0_return_estimate-False-False] | 91.0510μs | 64.2077μs | 15.5744 KOps/s | 15.3031 KOps/s | $\color{#35bf28}+1.77\\%$ | | test_values[td1_return_estimate-False-False] | 55.2749ms | 53.7943ms | 18.5893 Ops/s | 18.6336 Ops/s | $\color{#d91a1a}-0.24\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.4292ms | 1.0640ms | 939.8357 Ops/s | 932.4755 Ops/s | $\color{#35bf28}+0.79\\%$ | | test_values[td_lambda_return_estimate-True-False] | 86.7926ms | 85.9165ms | 11.6392 Ops/s | 11.6465 Ops/s | $\color{#d91a1a}-0.06\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.2472ms | 1.0596ms | 943.7532 Ops/s | 935.4968 Ops/s | $\color{#35bf28}+0.88\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 24.5288ms | 24.0528ms | 41.5751 Ops/s | 40.3354 Ops/s | $\color{#35bf28}+3.07\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9311ms | 0.6971ms | 1.4345 KOps/s | 1.4244 KOps/s | $\color{#35bf28}+0.71\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7411ms | 0.6695ms | 1.4937 KOps/s | 1.5317 KOps/s | $\color{#d91a1a}-2.48\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5005ms | 1.4497ms | 689.7750 Ops/s | 685.9658 Ops/s | $\color{#35bf28}+0.56\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7227ms | 0.6702ms | 1.4921 KOps/s | 1.4371 KOps/s | $\color{#35bf28}+3.83\\%$ | | test_dqn_speed | 1.5628ms | 1.4129ms | 707.7788 Ops/s | 705.6953 Ops/s | $\color{#35bf28}+0.30\\%$ | | test_ddpg_speed | 3.0990ms | 2.9223ms | 342.1977 Ops/s | 340.2634 Ops/s | $\color{#35bf28}+0.57\\%$ | | test_sac_speed | 8.9080ms | 8.4240ms | 118.7083 Ops/s | 119.2874 Ops/s | $\color{#d91a1a}-0.49\\%$ | | test_redq_speed | 13.7942ms | 10.7674ms | 92.8733 Ops/s | 92.1681 Ops/s | $\color{#35bf28}+0.77\\%$ | | test_redq_deprec_speed | 12.2627ms | 11.4959ms | 86.9873 Ops/s | 87.5252 Ops/s | $\color{#d91a1a}-0.61\\%$ | | test_td3_speed | 8.4438ms | 8.3141ms | 120.2779 Ops/s | 119.8249 Ops/s | $\color{#35bf28}+0.38\\%$ | | test_cql_speed | 27.6325ms | 25.8423ms | 38.6963 Ops/s | 38.8714 Ops/s | $\color{#d91a1a}-0.45\\%$ | | test_a2c_speed | 5.8398ms | 5.5714ms | 179.4896 Ops/s | 179.6761 Ops/s | $\color{#d91a1a}-0.10\\%$ | | test_ppo_speed | 6.1401ms | 5.8978ms | 169.5556 Ops/s | 169.5033 Ops/s | $\color{#35bf28}+0.03\\%$ | | test_reinforce_speed | 4.7664ms | 4.5588ms | 219.3561 Ops/s | 218.7436 Ops/s | $\color{#35bf28}+0.28\\%$ | | test_iql_speed | 20.3216ms | 19.7108ms | 50.7336 Ops/s | 51.7311 Ops/s | $\color{#d91a1a}-1.93\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.9410ms | 4.7898ms | 208.7789 Ops/s | 200.3873 Ops/s | $\color{#35bf28}+4.19\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.3622ms | 0.5958ms | 1.6785 KOps/s | 1.6721 KOps/s | $\color{#35bf28}+0.38\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7414ms | 0.5691ms | 1.7572 KOps/s | 1.7625 KOps/s | $\color{#d91a1a}-0.30\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.9681ms | 4.7355ms | 211.1691 Ops/s | 204.1724 Ops/s | $\color{#35bf28}+3.43\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.3320ms | 0.5825ms | 1.7167 KOps/s | 1.6997 KOps/s | $\color{#35bf28}+1.00\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7023ms | 0.5593ms | 1.7878 KOps/s | 1.7714 KOps/s | $\color{#35bf28}+0.93\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 4.9310ms | 2.0560ms | 486.3716 Ops/s | 482.5078 Ops/s | $\color{#35bf28}+0.80\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.1187ms | 1.9379ms | 516.0167 Ops/s | 507.8581 Ops/s | $\color{#35bf28}+1.61\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.9903ms | 4.9023ms | 203.9850 Ops/s | 199.0071 Ops/s | $\color{#35bf28}+2.50\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8927ms | 0.7121ms | 1.4042 KOps/s | 1.3966 KOps/s | $\color{#35bf28}+0.55\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.7223ms | 0.6940ms | 1.4408 KOps/s | 1.4408 KOps/s | $+0.00\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.9500ms | 4.7646ms | 209.8793 Ops/s | 199.0190 Ops/s | $\textbf{\color{#35bf28}+5.46\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7018ms | 0.5914ms | 1.6909 KOps/s | 1.6839 KOps/s | $\color{#35bf28}+0.42\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 4.4997ms | 0.5727ms | 1.7460 KOps/s | 1.7328 KOps/s | $\color{#35bf28}+0.76\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.0344ms | 4.7724ms | 209.5396 Ops/s | 203.6169 Ops/s | $\color{#35bf28}+2.91\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.3174ms | 0.5872ms | 1.7030 KOps/s | 1.6966 KOps/s | $\color{#35bf28}+0.37\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7295ms | 0.5654ms | 1.7686 KOps/s | 1.7747 KOps/s | $\color{#d91a1a}-0.34\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.9863ms | 4.8884ms | 204.5659 Ops/s | 198.6160 Ops/s | $\color{#35bf28}+3.00\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8627ms | 0.7154ms | 1.3977 KOps/s | 1.3960 KOps/s | $\color{#35bf28}+0.13\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.6789ms | 0.6964ms | 1.4360 KOps/s | 1.4179 KOps/s | $\color{#35bf28}+1.28\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1147s | 9.3640ms | 106.7925 Ops/s | 106.3452 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.4278ms | 15.9011ms | 62.8886 Ops/s | 61.0306 Ops/s | $\color{#35bf28}+3.04\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.3787ms | 1.3124ms | 761.9549 Ops/s | 758.9753 Ops/s | $\color{#35bf28}+0.39\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1053s | 7.1212ms | 140.4263 Ops/s | 139.7319 Ops/s | $\color{#35bf28}+0.50\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1174s | 17.9834ms | 55.6070 Ops/s | 54.5121 Ops/s | $\color{#35bf28}+2.01\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.5460ms | 1.3177ms | 758.8695 Ops/s | 736.7003 Ops/s | $\color{#35bf28}+3.01\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1049s | 7.2635ms | 137.6756 Ops/s | 136.8311 Ops/s | $\color{#35bf28}+0.62\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 19.0057ms | 16.2173ms | 61.6625 Ops/s | 60.0395 Ops/s | $\color{#35bf28}+2.70\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.7386ms | 1.4927ms | 669.9299 Ops/s | 666.4587 Ops/s | $\color{#35bf28}+0.52\\%$ |