search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.19k stars 289 forks source link

[Feature] Allow users to add random modules for vmap randomness detection #2317

Closed vmoens closed 1 month ago

vmoens commented 1 month ago

Stack from ghstack (oldest at bottom):

pytorch-bot[bot] commented 1 month ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2317

Note: Links to docs will display an error until the docs builds have been completed.

:x: 3 New Failures, 1 Pending, 1 Unrelated Failure

As of commit 400eb94fef6a1bede5b42aead803a8a77451f430 with merge base f840a1a4364bbb0bd33fbff7c4554e75af3ee1db (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2317#27845438622) ([gh](https://github.com/pytorch/rl/actions/runs/10072775637/job/27845438622)) `RuntimeError: Command docker exec -t 287dd08fd7f0ca3ed5516a87bc07d41a43921f4553829ae6a7f8748e604fcfd7 /exec failed with exit code 139` * [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2317#27845497254) ([gh](https://github.com/pytorch/rl/actions/runs/10072775652/job/27845497254)) `AttributeError: module 'torch' has no attribute 'compiler'` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2317#27845408152) ([gh](https://github.com/pytorch/rl/actions/runs/10072775639/job/27845408152)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2317#27845483064) ([gh](https://github.com/pytorch/rl/actions/runs/10072775674/job/27845483064)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/f840a1a4364bbb0bd33fbff7c4554e75af3ee1db#27805197076)) `AttributeError: module 'torch' has no attribute 'compiler'`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 59.5518ms | 57.1750ms | 17.4902 Ops/s | 17.2976 Ops/s | $\color{#35bf28}+1.11\\%$ | | test_sync | 49.9974ms | 32.3411ms | 30.9204 Ops/s | 31.9860 Ops/s | $\color{#d91a1a}-3.33\\%$ | | test_async | 65.3989ms | 30.1459ms | 33.1720 Ops/s | 33.5078 Ops/s | $\color{#d91a1a}-1.00\\%$ | | test_simple | 0.4844s | 0.4089s | 2.4456 Ops/s | 2.4514 Ops/s | $\color{#d91a1a}-0.24\\%$ | | test_transformed | 0.6326s | 0.5633s | 1.7751 Ops/s | 1.7370 Ops/s | $\color{#35bf28}+2.19\\%$ | | test_serial | 1.3222s | 1.2514s | 0.7991 Ops/s | 0.7981 Ops/s | $\color{#35bf28}+0.12\\%$ | | test_parallel | 1.1701s | 1.1007s | 0.9085 Ops/s | 0.9018 Ops/s | $\color{#35bf28}+0.74\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.3304ms | 25.5617μs | 39.1210 KOps/s | 39.2219 KOps/s | $\color{#d91a1a}-0.26\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 47.1570μs | 14.7115μs | 67.9742 KOps/s | 69.3933 KOps/s | $\color{#d91a1a}-2.05\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 44.2220μs | 15.0564μs | 66.4170 KOps/s | 67.7411 KOps/s | $\color{#d91a1a}-1.95\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 36.8190μs | 8.7692μs | 114.0361 KOps/s | 120.0658 KOps/s | $\textbf{\color{#d91a1a}-5.02\\%}$ | | test_step_mdp_speed[True-True-False-True-True] | 58.6080μs | 27.3576μs | 36.5529 KOps/s | 36.5582 KOps/s | $\color{#d91a1a}-0.01\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 46.4560μs | 16.4360μs | 60.8420 KOps/s | 63.1565 KOps/s | $\color{#d91a1a}-3.66\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 46.6870μs | 16.2425μs | 61.5669 KOps/s | 62.1374 KOps/s | $\color{#d91a1a}-0.92\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 0.1266ms | 10.5284μs | 94.9811 KOps/s | 102.1202 KOps/s | $\textbf{\color{#d91a1a}-6.99\\%}$ | | test_step_mdp_speed[True-False-True-True-True] | 59.6400μs | 28.9572μs | 34.5337 KOps/s | 34.7901 KOps/s | $\color{#d91a1a}-0.74\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 60.7030μs | 18.0487μs | 55.4057 KOps/s | 56.3033 KOps/s | $\color{#d91a1a}-1.59\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 54.8620μs | 16.3550μs | 61.1434 KOps/s | 62.4034 KOps/s | $\color{#d91a1a}-2.02\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 35.3650μs | 10.1800μs | 98.2319 KOps/s | 102.4140 KOps/s | $\color{#d91a1a}-4.08\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 61.1040μs | 30.4238μs | 32.8690 KOps/s | 33.3193 KOps/s | $\color{#d91a1a}-1.35\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 47.0770μs | 19.6622μs | 50.8589 KOps/s | 52.1259 KOps/s | $\color{#d91a1a}-2.43\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 59.7910μs | 17.9592μs | 55.6817 KOps/s | 56.9539 KOps/s | $\color{#d91a1a}-2.23\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 0.1517ms | 11.7179μs | 85.3395 KOps/s | 87.7769 KOps/s | $\color{#d91a1a}-2.78\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 0.1143ms | 29.4176μs | 33.9933 KOps/s | 34.9938 KOps/s | $\color{#d91a1a}-2.86\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 47.4780μs | 18.2602μs | 54.7639 KOps/s | 57.2673 KOps/s | $\color{#d91a1a}-4.37\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 46.5860μs | 19.1535μs | 52.2097 KOps/s | 54.7879 KOps/s | $\color{#d91a1a}-4.71\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 28.8340μs | 11.4864μs | 87.0594 KOps/s | 90.7266 KOps/s | $\color{#d91a1a}-4.04\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 60.6320μs | 30.8310μs | 32.4348 KOps/s | 33.0080 KOps/s | $\color{#d91a1a}-1.74\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 69.7300μs | 19.9110μs | 50.2234 KOps/s | 52.4309 KOps/s | $\color{#d91a1a}-4.21\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 74.9500μs | 20.3757μs | 49.0781 KOps/s | 50.1339 KOps/s | $\color{#d91a1a}-2.11\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 35.9660μs | 12.9279μs | 77.3523 KOps/s | 80.3602 KOps/s | $\color{#d91a1a}-3.74\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.3605ms | 31.9685μs | 31.2808 KOps/s | 31.1552 KOps/s | $\color{#35bf28}+0.40\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 52.6380μs | 21.1137μs | 47.3626 KOps/s | 48.7972 KOps/s | $\color{#d91a1a}-2.94\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 45.0940μs | 20.2255μs | 49.4425 KOps/s | 48.7426 KOps/s | $\color{#35bf28}+1.44\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 40.7760μs | 12.8520μs | 77.8087 KOps/s | 78.5592 KOps/s | $\color{#d91a1a}-0.96\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 89.5760μs | 33.5122μs | 29.8399 KOps/s | 30.4230 KOps/s | $\color{#d91a1a}-1.92\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 55.1120μs | 22.4552μs | 44.5330 KOps/s | 45.6817 KOps/s | $\color{#d91a1a}-2.51\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 51.0750μs | 21.5424μs | 46.4200 KOps/s | 46.4667 KOps/s | $\color{#d91a1a}-0.10\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 43.1710μs | 14.3328μs | 69.7699 KOps/s | 71.2891 KOps/s | $\color{#d91a1a}-2.13\\%$ | | test_values[generalized_advantage_estimate-True-True] | 12.9650ms | 9.4570ms | 105.7418 Ops/s | 103.0204 Ops/s | $\color{#35bf28}+2.64\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 38.4340ms | 33.7488ms | 29.6307 Ops/s | 29.7271 Ops/s | $\color{#d91a1a}-0.32\\%$ | | test_values[td0_return_estimate-False-False] | 0.2341ms | 0.1735ms | 5.7646 KOps/s | 5.5656 KOps/s | $\color{#35bf28}+3.58\\%$ | | test_values[td1_return_estimate-False-False] | 25.1820ms | 23.6953ms | 42.2025 Ops/s | 42.1029 Ops/s | $\color{#35bf28}+0.24\\%$ | | test_values[vec_td1_return_estimate-False-False] | 36.1748ms | 33.7539ms | 29.6262 Ops/s | 30.0035 Ops/s | $\color{#d91a1a}-1.26\\%$ | | test_values[td_lambda_return_estimate-True-False] | 36.0512ms | 33.9982ms | 29.4133 Ops/s | 28.8444 Ops/s | $\color{#35bf28}+1.97\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 36.1474ms | 33.7045ms | 29.6696 Ops/s | 29.7681 Ops/s | $\color{#d91a1a}-0.33\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 11.4611ms | 8.2784ms | 120.7957 Ops/s | 117.6861 Ops/s | $\color{#35bf28}+2.64\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.2552ms | 1.9571ms | 510.9560 Ops/s | 501.6656 Ops/s | $\color{#35bf28}+1.85\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4875ms | 0.3544ms | 2.8218 KOps/s | 2.8073 KOps/s | $\color{#35bf28}+0.52\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 42.0851ms | 41.3943ms | 24.1579 Ops/s | 22.3267 Ops/s | $\textbf{\color{#35bf28}+8.20\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.9436ms | 3.0369ms | 329.2811 Ops/s | 330.2093 Ops/s | $\color{#d91a1a}-0.28\\%$ | | test_dqn_speed | 1.4893ms | 1.3451ms | 743.4215 Ops/s | 717.6000 Ops/s | $\color{#35bf28}+3.60\\%$ | | test_ddpg_speed | 3.8229ms | 2.8690ms | 348.5580 Ops/s | 345.0088 Ops/s | $\color{#35bf28}+1.03\\%$ | | test_sac_speed | 9.6222ms | 8.4279ms | 118.6540 Ops/s | 118.3145 Ops/s | $\color{#35bf28}+0.29\\%$ | | test_redq_speed | 16.0763ms | 13.6656ms | 73.1767 Ops/s | 74.7812 Ops/s | $\color{#d91a1a}-2.15\\%$ | | test_redq_deprec_speed | 14.7776ms | 13.5502ms | 73.7998 Ops/s | 76.4193 Ops/s | $\color{#d91a1a}-3.43\\%$ | | test_td3_speed | 8.7407ms | 8.3462ms | 119.8145 Ops/s | 119.4602 Ops/s | $\color{#35bf28}+0.30\\%$ | | test_cql_speed | 38.2634ms | 36.5002ms | 27.3971 Ops/s | 27.7758 Ops/s | $\color{#d91a1a}-1.36\\%$ | | test_a2c_speed | 8.8896ms | 7.3677ms | 135.7269 Ops/s | 134.6299 Ops/s | $\color{#35bf28}+0.81\\%$ | | test_ppo_speed | 8.7211ms | 7.7265ms | 129.4253 Ops/s | 129.6541 Ops/s | $\color{#d91a1a}-0.18\\%$ | | test_reinforce_speed | 7.7399ms | 6.6591ms | 150.1709 Ops/s | 149.7820 Ops/s | $\color{#35bf28}+0.26\\%$ | | test_iql_speed | 34.9967ms | 32.6208ms | 30.6553 Ops/s | 28.2558 Ops/s | $\textbf{\color{#35bf28}+8.49\\%}$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.7307ms | 4.9987ms | 200.0531 Ops/s | 203.5509 Ops/s | $\color{#d91a1a}-1.72\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7129ms | 0.4825ms | 2.0725 KOps/s | 2.1011 KOps/s | $\color{#d91a1a}-1.36\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6606ms | 0.4551ms | 2.1974 KOps/s | 2.1891 KOps/s | $\color{#35bf28}+0.38\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.2124ms | 4.9348ms | 202.6443 Ops/s | 207.0607 Ops/s | $\color{#d91a1a}-2.13\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.1166s | 0.5775ms | 1.7317 KOps/s | 2.1271 KOps/s | $\textbf{\color{#d91a1a}-18.59\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6142ms | 0.4526ms | 2.2094 KOps/s | 2.2100 KOps/s | $\color{#d91a1a}-0.03\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.2656ms | 1.6808ms | 594.9474 Ops/s | 590.4370 Ops/s | $\color{#35bf28}+0.76\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2143ms | 1.5923ms | 628.0397 Ops/s | 620.8564 Ops/s | $\color{#35bf28}+1.16\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.2475ms | 5.0061ms | 199.7546 Ops/s | 203.0788 Ops/s | $\color{#d91a1a}-1.64\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9930ms | 0.6114ms | 1.6357 KOps/s | 1.6317 KOps/s | $\color{#35bf28}+0.24\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8435ms | 0.5881ms | 1.7005 KOps/s | 1.6985 KOps/s | $\color{#35bf28}+0.12\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.6195ms | 4.9343ms | 202.6633 Ops/s | 210.1070 Ops/s | $\color{#d91a1a}-3.54\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.2305ms | 0.4787ms | 2.0889 KOps/s | 2.0902 KOps/s | $\color{#d91a1a}-0.06\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.5881ms | 0.4521ms | 2.2121 KOps/s | 2.1252 KOps/s | $\color{#35bf28}+4.09\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 7.5856ms | 4.7988ms | 208.3863 Ops/s | 211.4132 Ops/s | $\color{#d91a1a}-1.43\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6962ms | 0.4777ms | 2.0935 KOps/s | 2.1230 KOps/s | $\color{#d91a1a}-1.39\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 7.7470ms | 0.5129ms | 1.9496 KOps/s | 2.2224 KOps/s | $\textbf{\color{#d91a1a}-12.27\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.5380ms | 5.1544ms | 194.0105 Ops/s | 205.7212 Ops/s | $\textbf{\color{#d91a1a}-5.69\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9635ms | 0.6171ms | 1.6206 KOps/s | 1.6385 KOps/s | $\color{#d91a1a}-1.09\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7718ms | 0.5960ms | 1.6779 KOps/s | 1.7120 KOps/s | $\color{#d91a1a}-1.99\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1336s | 6.5084ms | 153.6479 Ops/s | 157.8050 Ops/s | $\color{#d91a1a}-2.63\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 16.9079ms | 12.9597ms | 77.1624 Ops/s | 77.0307 Ops/s | $\color{#35bf28}+0.17\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 1.6204ms | 1.0978ms | 910.9143 Ops/s | 833.5878 Ops/s | $\textbf{\color{#35bf28}+9.28\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1190s | 6.1555ms | 162.4568 Ops/s | 165.7120 Ops/s | $\color{#d91a1a}-1.96\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 17.1389ms | 12.8894ms | 77.5830 Ops/s | 66.4121 Ops/s | $\textbf{\color{#35bf28}+16.82\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.5739ms | 1.0789ms | 926.8792 Ops/s | 871.7604 Ops/s | $\textbf{\color{#35bf28}+6.32\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1125s | 8.2964ms | 120.5340 Ops/s | 164.9178 Ops/s | $\textbf{\color{#d91a1a}-26.91\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 17.3312ms | 13.0006ms | 76.9198 Ops/s | 74.8989 Ops/s | $\color{#35bf28}+2.70\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 6.1311ms | 1.3599ms | 735.3540 Ops/s | 750.8343 Ops/s | $\color{#d91a1a}-2.06\\%$ |
github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}9$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1089s | 0.1085s | 9.2137 Ops/s | 8.2992 Ops/s | $\textbf{\color{#35bf28}+11.02\\%}$ | | test_sync | 95.7133ms | 95.2587ms | 10.4977 Ops/s | 10.4964 Ops/s | $\color{#35bf28}+0.01\\%$ | | test_async | 0.2083s | 90.1549ms | 11.0920 Ops/s | 10.9923 Ops/s | $\color{#35bf28}+0.91\\%$ | | test_single_pixels | 0.1189s | 0.1188s | 8.4149 Ops/s | 8.3917 Ops/s | $\color{#35bf28}+0.28\\%$ | | test_sync_pixels | 78.0385ms | 76.0521ms | 13.1489 Ops/s | 12.0645 Ops/s | $\textbf{\color{#35bf28}+8.99\\%}$ | | test_async_pixels | 0.1351s | 69.4183ms | 14.4054 Ops/s | 14.2575 Ops/s | $\color{#35bf28}+1.04\\%$ | | test_simple | 0.8775s | 0.8012s | 1.2481 Ops/s | 1.2354 Ops/s | $\color{#35bf28}+1.03\\%$ | | test_transformed | 1.1004s | 1.0289s | 0.9719 Ops/s | 0.9682 Ops/s | $\color{#35bf28}+0.39\\%$ | | test_serial | 2.3381s | 2.2687s | 0.4408 Ops/s | 0.4210 Ops/s | $\color{#35bf28}+4.70\\%$ | | test_parallel | 2.0518s | 1.9911s | 0.5022 Ops/s | 0.5069 Ops/s | $\color{#d91a1a}-0.92\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 68.4620μs | 37.9022μs | 26.3837 KOps/s | 25.8466 KOps/s | $\color{#35bf28}+2.08\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 48.1510μs | 21.6283μs | 46.2357 KOps/s | 45.8457 KOps/s | $\color{#35bf28}+0.85\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 0.1494ms | 21.3545μs | 46.8286 KOps/s | 46.0532 KOps/s | $\color{#35bf28}+1.68\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 31.1510μs | 12.1197μs | 82.5104 KOps/s | 80.6182 KOps/s | $\color{#35bf28}+2.35\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 62.4810μs | 40.2395μs | 24.8512 KOps/s | 24.0445 KOps/s | $\color{#35bf28}+3.35\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 42.0300μs | 23.9032μs | 41.8355 KOps/s | 41.1337 KOps/s | $\color{#35bf28}+1.71\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 49.0100μs | 23.5056μs | 42.5430 KOps/s | 41.4503 KOps/s | $\color{#35bf28}+2.64\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 36.1300μs | 14.4831μs | 69.0458 KOps/s | 67.7806 KOps/s | $\color{#35bf28}+1.87\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 71.3810μs | 42.7662μs | 23.3830 KOps/s | 23.0572 KOps/s | $\color{#35bf28}+1.41\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 43.2500μs | 26.6713μs | 37.4935 KOps/s | 37.3116 KOps/s | $\color{#35bf28}+0.49\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 51.4610μs | 23.4726μs | 42.6028 KOps/s | 41.6397 KOps/s | $\color{#35bf28}+2.31\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 44.6300μs | 14.5248μs | 68.8476 KOps/s | 68.2537 KOps/s | $\color{#35bf28}+0.87\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 76.4710μs | 44.6992μs | 22.3718 KOps/s | 21.9169 KOps/s | $\color{#35bf28}+2.08\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 48.6200μs | 28.6696μs | 34.8802 KOps/s | 33.9801 KOps/s | $\color{#35bf28}+2.65\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 80.1510μs | 25.7311μs | 38.8634 KOps/s | 38.0098 KOps/s | $\color{#35bf28}+2.25\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 38.3210μs | 16.7287μs | 59.7774 KOps/s | 58.7288 KOps/s | $\color{#35bf28}+1.79\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 67.0810μs | 42.7786μs | 23.3762 KOps/s | 22.8732 KOps/s | $\color{#35bf28}+2.20\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 51.7510μs | 26.6928μs | 37.4633 KOps/s | 36.7561 KOps/s | $\color{#35bf28}+1.92\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 54.2200μs | 28.1436μs | 35.5321 KOps/s | 35.0011 KOps/s | $\color{#35bf28}+1.52\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 34.6400μs | 16.6237μs | 60.1550 KOps/s | 57.3341 KOps/s | $\color{#35bf28}+4.92\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 74.7010μs | 45.0200μs | 22.2124 KOps/s | 21.7946 KOps/s | $\color{#35bf28}+1.92\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 59.4210μs | 28.9013μs | 34.6005 KOps/s | 34.1478 KOps/s | $\color{#35bf28}+1.33\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 58.3200μs | 30.3743μs | 32.9226 KOps/s | 32.1430 KOps/s | $\color{#35bf28}+2.43\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 40.0900μs | 18.7372μs | 53.3697 KOps/s | 51.1882 KOps/s | $\color{#35bf28}+4.26\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.8508ms | 47.3299μs | 21.1283 KOps/s | 20.3536 KOps/s | $\color{#35bf28}+3.81\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 46.7710μs | 31.2969μs | 31.9520 KOps/s | 31.3220 KOps/s | $\color{#35bf28}+2.01\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 57.5510μs | 30.0387μs | 33.2904 KOps/s | 32.3649 KOps/s | $\color{#35bf28}+2.86\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 40.6400μs | 18.5913μs | 53.7886 KOps/s | 51.2036 KOps/s | $\textbf{\color{#35bf28}+5.05\\%}$ | | test_step_mdp_speed[False-False-False-True-True] | 85.9710μs | 49.3670μs | 20.2564 KOps/s | 20.0637 KOps/s | $\color{#35bf28}+0.96\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 58.8810μs | 33.1316μs | 30.1827 KOps/s | 29.4389 KOps/s | $\color{#35bf28}+2.53\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 48.7800μs | 32.3213μs | 30.9393 KOps/s | 30.3102 KOps/s | $\color{#35bf28}+2.08\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 42.6410μs | 20.8656μs | 47.9257 KOps/s | 46.2628 KOps/s | $\color{#35bf28}+3.59\\%$ | | test_values[generalized_advantage_estimate-True-True] | 27.1730ms | 25.6526ms | 38.9824 Ops/s | 39.4785 Ops/s | $\color{#d91a1a}-1.26\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 88.7331ms | 2.6846ms | 372.4881 Ops/s | 370.4780 Ops/s | $\color{#35bf28}+0.54\\%$ | | test_values[td0_return_estimate-False-False] | 90.9120μs | 68.1642μs | 14.6705 KOps/s | 14.8077 KOps/s | $\color{#d91a1a}-0.93\\%$ | | test_values[td1_return_estimate-False-False] | 60.1522ms | 59.5998ms | 16.7786 Ops/s | 17.5206 Ops/s | $\color{#d91a1a}-4.23\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.3687ms | 1.1005ms | 908.6797 Ops/s | 902.5548 Ops/s | $\color{#35bf28}+0.68\\%$ | | test_values[td_lambda_return_estimate-True-False] | 94.2111ms | 91.0607ms | 10.9817 Ops/s | 11.0552 Ops/s | $\color{#d91a1a}-0.66\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.3615ms | 1.0971ms | 911.5342 Ops/s | 908.5981 Ops/s | $\color{#35bf28}+0.32\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 25.6764ms | 25.4271ms | 39.3282 Ops/s | 39.0642 Ops/s | $\color{#35bf28}+0.68\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9593ms | 0.7379ms | 1.3552 KOps/s | 1.3611 KOps/s | $\color{#d91a1a}-0.44\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7790ms | 0.6849ms | 1.4601 KOps/s | 1.4598 KOps/s | $\color{#35bf28}+0.02\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5107ms | 1.4843ms | 673.7363 Ops/s | 674.3519 Ops/s | $\color{#d91a1a}-0.09\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7645ms | 0.6997ms | 1.4293 KOps/s | 1.4285 KOps/s | $\color{#35bf28}+0.06\\%$ | | test_dqn_speed | 7.2791ms | 1.4293ms | 699.6190 Ops/s | 678.6433 Ops/s | $\color{#35bf28}+3.09\\%$ | | test_ddpg_speed | 3.2364ms | 2.9424ms | 339.8623 Ops/s | 332.9524 Ops/s | $\color{#35bf28}+2.08\\%$ | | test_sac_speed | 0.1013s | 9.1860ms | 108.8615 Ops/s | 117.1757 Ops/s | $\textbf{\color{#d91a1a}-7.10\\%}$ | | test_redq_speed | 11.7926ms | 10.9497ms | 91.3263 Ops/s | 91.1191 Ops/s | $\color{#35bf28}+0.23\\%$ | | test_redq_deprec_speed | 12.2257ms | 11.5714ms | 86.4199 Ops/s | 87.3468 Ops/s | $\color{#d91a1a}-1.06\\%$ | | test_td3_speed | 8.5645ms | 8.3297ms | 120.0529 Ops/s | 118.9321 Ops/s | $\color{#35bf28}+0.94\\%$ | | test_cql_speed | 26.6823ms | 25.6541ms | 38.9802 Ops/s | 38.6530 Ops/s | $\color{#35bf28}+0.85\\%$ | | test_a2c_speed | 5.6934ms | 5.4076ms | 184.9254 Ops/s | 173.1991 Ops/s | $\textbf{\color{#35bf28}+6.77\\%}$ | | test_ppo_speed | 6.1421ms | 5.7494ms | 173.9308 Ops/s | 164.7960 Ops/s | $\textbf{\color{#35bf28}+5.54\\%}$ | | test_reinforce_speed | 5.1626ms | 4.3431ms | 230.2508 Ops/s | 212.5029 Ops/s | $\textbf{\color{#35bf28}+8.35\\%}$ | | test_iql_speed | 19.8201ms | 19.0418ms | 52.5160 Ops/s | 49.6235 Ops/s | $\textbf{\color{#35bf28}+5.83\\%}$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.9185ms | 6.7163ms | 148.8911 Ops/s | 149.2031 Ops/s | $\color{#d91a1a}-0.21\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.1840ms | 0.5125ms | 1.9511 KOps/s | 1.9577 KOps/s | $\color{#d91a1a}-0.34\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6714ms | 0.4891ms | 2.0444 KOps/s | 2.0449 KOps/s | $\color{#d91a1a}-0.02\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.8348ms | 6.6088ms | 151.3130 Ops/s | 151.9627 Ops/s | $\color{#d91a1a}-0.43\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8529ms | 0.5069ms | 1.9729 KOps/s | 1.6034 KOps/s | $\textbf{\color{#35bf28}+23.05\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6562ms | 0.4866ms | 2.0551 KOps/s | 2.0668 KOps/s | $\color{#d91a1a}-0.57\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.2006ms | 1.9745ms | 506.4452 Ops/s | 509.7282 Ops/s | $\color{#d91a1a}-0.64\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.0592ms | 1.8733ms | 533.8300 Ops/s | 537.2121 Ops/s | $\color{#d91a1a}-0.63\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.9471ms | 6.8209ms | 146.6086 Ops/s | 146.5863 Ops/s | $\color{#35bf28}+0.02\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.1727s | 0.8363ms | 1.1957 KOps/s | 1.5162 KOps/s | $\textbf{\color{#d91a1a}-21.14\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8082ms | 0.6438ms | 1.5533 KOps/s | 1.5553 KOps/s | $\color{#d91a1a}-0.13\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.9190ms | 6.6958ms | 149.3471 Ops/s | 149.8391 Ops/s | $\color{#d91a1a}-0.33\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6440ms | 0.5191ms | 1.9265 KOps/s | 1.9517 KOps/s | $\color{#d91a1a}-1.29\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6892ms | 0.4934ms | 2.0269 KOps/s | 2.0399 KOps/s | $\color{#d91a1a}-0.64\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.8262ms | 6.5841ms | 151.8820 Ops/s | 151.6080 Ops/s | $\color{#35bf28}+0.18\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.1300s | 0.6898ms | 1.4497 KOps/s | 1.9611 KOps/s | $\textbf{\color{#d91a1a}-26.08\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6929ms | 0.4884ms | 2.0477 KOps/s | 2.0652 KOps/s | $\color{#d91a1a}-0.85\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.9845ms | 6.8453ms | 146.0853 Ops/s | 147.3414 Ops/s | $\color{#d91a1a}-0.85\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.9823ms | 0.6966ms | 1.4356 KOps/s | 1.4955 KOps/s | $\color{#d91a1a}-4.01\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8660ms | 0.6829ms | 1.4643 KOps/s | 1.5596 KOps/s | $\textbf{\color{#d91a1a}-6.11\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1311s | 7.8568ms | 127.2783 Ops/s | 129.4419 Ops/s | $\color{#d91a1a}-1.67\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 23.0539ms | 16.7732ms | 59.6191 Ops/s | 60.2902 Ops/s | $\color{#d91a1a}-1.11\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 7.0047ms | 1.3673ms | 731.3641 Ops/s | 742.4468 Ops/s | $\color{#d91a1a}-1.49\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1279s | 7.7020ms | 129.8364 Ops/s | 130.9786 Ops/s | $\color{#d91a1a}-0.87\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1379s | 18.6296ms | 53.6781 Ops/s | 60.6826 Ops/s | $\textbf{\color{#d91a1a}-11.54\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 5.9396ms | 1.3391ms | 746.7859 Ops/s | 740.9766 Ops/s | $\color{#35bf28}+0.78\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1282s | 8.0070ms | 124.8909 Ops/s | 126.2739 Ops/s | $\color{#d91a1a}-1.10\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 19.0345ms | 16.3921ms | 61.0051 Ops/s | 60.1037 Ops/s | $\color{#35bf28}+1.50\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.3873ms | 1.3886ms | 720.1290 Ops/s | 656.1795 Ops/s | $\textbf{\color{#35bf28}+9.75\\%}$ |