search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.25k stars 297 forks source link

[BugFix] Fix lib tests #2218

Closed vmoens closed 4 months ago

pytorch-bot[bot] commented 4 months ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2218

Note: Links to docs will display an error until the docs builds have been completed.

:heavy_exclamation_mark: 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

:x: 3 New Failures, 1 Unrelated Failure

As of commit e1128d6c7ec8e6a3fa3134bc4db47420d729c399 with merge base 1bd3814d6efb0104dda86a7fe6f6164abd39fd42 (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2218#25979918130) ([gh](https://github.com/pytorch/rl/actions/runs/9431351737/job/25979918130)) `RuntimeError: Command docker exec -t 3886ccfcff9923c4b539cee0791cf079f4ad53b3b63bb585a556687e309dd5dd /exec failed with exit code 139` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2218#25979918756) ([gh](https://github.com/pytorch/rl/actions/runs/9431351736/job/25979918756)) `RuntimeError: Command docker exec -t dcc367e8b179544303fd0d103d90a591387a01ce33d8ee7a5a6e86bf5b98296c /exec failed with exit code 1` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2218#25979905078) ([gh](https://github.com/pytorch/rl/actions/runs/9431351743/job/25979905078)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2218#25979919079) ([gh](https://github.com/pytorch/rl/actions/runs/9431351736/job/25979919079)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/1bd3814d6efb0104dda86a7fe6f6164abd39fd42#25973885043)) `test/test_transforms.py::TestVecNorm::test_state_dict_vecnorm`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 4 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1166s | 60.1382ms | 16.6284 Ops/s | 17.7781 Ops/s | $\textbf{\color{#d91a1a}-6.47\\%}$ | | test_sync | 36.8459ms | 30.6061ms | 32.6732 Ops/s | 32.2574 Ops/s | $\color{#35bf28}+1.29\\%$ | | test_async | 59.9175ms | 29.3490ms | 34.0727 Ops/s | 36.0236 Ops/s | $\textbf{\color{#d91a1a}-5.42\\%}$ | | test_simple | 0.4500s | 0.3921s | 2.5506 Ops/s | 2.5979 Ops/s | $\color{#d91a1a}-1.82\\%$ | | test_transformed | 0.5269s | 0.5241s | 1.9079 Ops/s | 1.8671 Ops/s | $\color{#35bf28}+2.18\\%$ | | test_serial | 1.3211s | 1.2642s | 0.7910 Ops/s | 0.7856 Ops/s | $\color{#35bf28}+0.69\\%$ | | test_parallel | 1.1415s | 1.0725s | 0.9324 Ops/s | 0.9209 Ops/s | $\color{#35bf28}+1.25\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1825ms | 21.3278μs | 46.8871 KOps/s | 46.7129 KOps/s | $\color{#35bf28}+0.37\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 36.9790μs | 13.0223μs | 76.7911 KOps/s | 76.6919 KOps/s | $\color{#35bf28}+0.13\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 36.6690μs | 12.7546μs | 78.4032 KOps/s | 77.8537 KOps/s | $\color{#35bf28}+0.71\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 29.7450μs | 7.8414μs | 127.5284 KOps/s | 130.3096 KOps/s | $\color{#d91a1a}-2.13\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 51.8370μs | 23.0534μs | 43.3775 KOps/s | 43.5331 KOps/s | $\color{#d91a1a}-0.36\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 42.0790μs | 14.3295μs | 69.7863 KOps/s | 69.7456 KOps/s | $\color{#35bf28}+0.06\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 40.5160μs | 13.9813μs | 71.5243 KOps/s | 70.6761 KOps/s | $\color{#35bf28}+1.20\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 30.8370μs | 9.0465μs | 110.5395 KOps/s | 111.3351 KOps/s | $\color{#d91a1a}-0.71\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 50.9440μs | 24.3304μs | 41.1008 KOps/s | 41.1220 KOps/s | $\color{#d91a1a}-0.05\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 57.8370μs | 15.6504μs | 63.8961 KOps/s | 63.4365 KOps/s | $\color{#35bf28}+0.72\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 57.5470μs | 14.1010μs | 70.9171 KOps/s | 71.7600 KOps/s | $\color{#d91a1a}-1.17\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 34.5550μs | 8.9446μs | 111.7987 KOps/s | 113.2659 KOps/s | $\color{#d91a1a}-1.30\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 64.1900μs | 25.3357μs | 39.4701 KOps/s | 39.7583 KOps/s | $\color{#d91a1a}-0.72\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 44.3630μs | 16.6927μs | 59.9066 KOps/s | 59.6826 KOps/s | $\color{#35bf28}+0.38\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 64.0290μs | 15.5498μs | 64.3094 KOps/s | 65.9369 KOps/s | $\color{#d91a1a}-2.47\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 28.3730μs | 10.0358μs | 99.6432 KOps/s | 98.7782 KOps/s | $\color{#35bf28}+0.88\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 58.6700μs | 24.0617μs | 41.5598 KOps/s | 41.2921 KOps/s | $\color{#35bf28}+0.65\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 43.8020μs | 15.6535μs | 63.8834 KOps/s | 63.8052 KOps/s | $\color{#35bf28}+0.12\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 54.5020μs | 16.0710μs | 62.2237 KOps/s | 61.3261 KOps/s | $\color{#35bf28}+1.46\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 33.9130μs | 10.2397μs | 97.6587 KOps/s | 97.9687 KOps/s | $\color{#d91a1a}-0.32\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 74.1980μs | 25.2687μs | 39.5747 KOps/s | 39.4537 KOps/s | $\color{#35bf28}+0.31\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 38.4420μs | 16.8451μs | 59.3644 KOps/s | 59.4643 KOps/s | $\color{#d91a1a}-0.17\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 54.5020μs | 17.4007μs | 57.4689 KOps/s | 56.6998 KOps/s | $\color{#35bf28}+1.36\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 35.4060μs | 11.4178μs | 87.5828 KOps/s | 86.9946 KOps/s | $\color{#35bf28}+0.68\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 57.0660μs | 26.3144μs | 38.0020 KOps/s | 37.3629 KOps/s | $\color{#35bf28}+1.71\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 43.9710μs | 18.0232μs | 55.4840 KOps/s | 54.6463 KOps/s | $\color{#35bf28}+1.53\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 60.9340μs | 17.4429μs | 57.3299 KOps/s | 57.2736 KOps/s | $\color{#35bf28}+0.10\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 41.7380μs | 11.3447μs | 88.1466 KOps/s | 87.3193 KOps/s | $\color{#35bf28}+0.95\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 40.2350μs | 27.9220μs | 35.8141 KOps/s | 35.2327 KOps/s | $\color{#35bf28}+1.65\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 52.3180μs | 19.0158μs | 52.5880 KOps/s | 51.7879 KOps/s | $\color{#35bf28}+1.54\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 0.1984ms | 18.1966μs | 54.9554 KOps/s | 54.4219 KOps/s | $\color{#35bf28}+0.98\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 76.6730μs | 12.5231μs | 79.8523 KOps/s | 79.9300 KOps/s | $\color{#d91a1a}-0.10\\%$ | | test_values[generalized_advantage_estimate-True-True] | 9.9160ms | 9.5590ms | 104.6140 Ops/s | 104.6407 Ops/s | $\color{#d91a1a}-0.03\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 40.2880ms | 33.7967ms | 29.5887 Ops/s | 28.4425 Ops/s | $\color{#35bf28}+4.03\\%$ | | test_values[td0_return_estimate-False-False] | 0.2611ms | 0.1815ms | 5.5103 KOps/s | 6.0311 KOps/s | $\textbf{\color{#d91a1a}-8.64\\%}$ | | test_values[td1_return_estimate-False-False] | 24.2248ms | 23.7621ms | 42.0838 Ops/s | 41.4177 Ops/s | $\color{#35bf28}+1.61\\%$ | | test_values[vec_td1_return_estimate-False-False] | 34.6973ms | 33.4241ms | 29.9185 Ops/s | 28.2380 Ops/s | $\textbf{\color{#35bf28}+5.95\\%}$ | | test_values[td_lambda_return_estimate-True-False] | 34.3998ms | 33.8685ms | 29.5259 Ops/s | 28.8670 Ops/s | $\color{#35bf28}+2.28\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 34.6181ms | 33.4050ms | 29.9357 Ops/s | 28.2800 Ops/s | $\textbf{\color{#35bf28}+5.85\\%}$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 12.0129ms | 8.3441ms | 119.8453 Ops/s | 116.6746 Ops/s | $\color{#35bf28}+2.72\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 1.8624ms | 1.7781ms | 562.3975 Ops/s | 498.7310 Ops/s | $\textbf{\color{#35bf28}+12.77\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4335ms | 0.3487ms | 2.8677 KOps/s | 2.8113 KOps/s | $\color{#35bf28}+2.01\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 48.3131ms | 40.9037ms | 24.4476 Ops/s | 22.8606 Ops/s | $\textbf{\color{#35bf28}+6.94\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.5367ms | 3.0119ms | 332.0118 Ops/s | 329.7006 Ops/s | $\color{#35bf28}+0.70\\%$ | | test_dqn_speed | 1.8053ms | 1.3393ms | 746.6796 Ops/s | 733.3647 Ops/s | $\color{#35bf28}+1.82\\%$ | | test_ddpg_speed | 3.1903ms | 2.8273ms | 353.6979 Ops/s | 346.2225 Ops/s | $\color{#35bf28}+2.16\\%$ | | test_sac_speed | 10.0164ms | 8.4558ms | 118.2619 Ops/s | 118.0803 Ops/s | $\color{#35bf28}+0.15\\%$ | | test_redq_speed | 14.2966ms | 13.3109ms | 75.1263 Ops/s | 74.0642 Ops/s | $\color{#35bf28}+1.43\\%$ | | test_redq_deprec_speed | 15.3180ms | 13.4820ms | 74.1731 Ops/s | 73.1078 Ops/s | $\color{#35bf28}+1.46\\%$ | | test_td3_speed | 16.2526ms | 8.3463ms | 119.8133 Ops/s | 117.9165 Ops/s | $\color{#35bf28}+1.61\\%$ | | test_cql_speed | 49.7719ms | 37.1911ms | 26.8881 Ops/s | 26.8846 Ops/s | $\color{#35bf28}+0.01\\%$ | | test_a2c_speed | 8.2280ms | 7.4342ms | 134.5127 Ops/s | 129.8808 Ops/s | $\color{#35bf28}+3.57\\%$ | | test_ppo_speed | 8.9974ms | 7.7350ms | 129.2819 Ops/s | 126.8578 Ops/s | $\color{#35bf28}+1.91\\%$ | | test_reinforce_speed | 7.9525ms | 6.7743ms | 147.6176 Ops/s | 146.4660 Ops/s | $\color{#35bf28}+0.79\\%$ | | test_iql_speed | 34.2287ms | 32.7740ms | 30.5120 Ops/s | 30.0529 Ops/s | $\color{#35bf28}+1.53\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.6900ms | 3.4949ms | 286.1286 Ops/s | 276.4820 Ops/s | $\color{#35bf28}+3.49\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.2460ms | 0.4966ms | 2.0137 KOps/s | 1.7723 KOps/s | $\textbf{\color{#35bf28}+13.62\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6826ms | 0.4852ms | 2.0609 KOps/s | 2.0811 KOps/s | $\color{#d91a1a}-0.97\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.0309ms | 3.6415ms | 274.6142 Ops/s | 279.0740 Ops/s | $\color{#d91a1a}-1.60\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7425ms | 0.4923ms | 2.0313 KOps/s | 2.0340 KOps/s | $\color{#d91a1a}-0.13\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6574ms | 0.4760ms | 2.1007 KOps/s | 2.0953 KOps/s | $\color{#35bf28}+0.26\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 1.7836ms | 1.6821ms | 594.5100 Ops/s | 588.5665 Ops/s | $\color{#35bf28}+1.01\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 6.9349ms | 1.6057ms | 622.7722 Ops/s | 618.7163 Ops/s | $\color{#35bf28}+0.66\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.5408ms | 3.5874ms | 278.7563 Ops/s | 268.5704 Ops/s | $\color{#35bf28}+3.79\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8517ms | 0.6376ms | 1.5683 KOps/s | 1.6117 KOps/s | $\color{#d91a1a}-2.69\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.1161s | 0.6943ms | 1.4403 KOps/s | 1.6689 KOps/s | $\textbf{\color{#d91a1a}-13.70\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.0199ms | 3.6801ms | 271.7333 Ops/s | 280.5445 Ops/s | $\color{#d91a1a}-3.14\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6240ms | 0.5135ms | 1.9475 KOps/s | 1.9870 KOps/s | $\color{#d91a1a}-1.99\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7203ms | 0.5034ms | 1.9865 KOps/s | 2.0846 KOps/s | $\color{#d91a1a}-4.71\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.9310ms | 3.6657ms | 272.7993 Ops/s | 270.5127 Ops/s | $\color{#35bf28}+0.85\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.1082s | 0.6475ms | 1.5443 KOps/s | 1.9155 KOps/s | $\textbf{\color{#d91a1a}-19.38\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6294ms | 0.4714ms | 2.1211 KOps/s | 2.1010 KOps/s | $\color{#35bf28}+0.96\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.0525ms | 3.6527ms | 273.7728 Ops/s | 271.0163 Ops/s | $\color{#35bf28}+1.02\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9031ms | 0.6219ms | 1.6081 KOps/s | 1.6198 KOps/s | $\color{#d91a1a}-0.72\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7399ms | 0.5912ms | 1.6916 KOps/s | 1.6807 KOps/s | $\color{#35bf28}+0.65\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1084s | 7.7896ms | 128.3769 Ops/s | 128.5621 Ops/s | $\color{#d91a1a}-0.14\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 14.5000ms | 12.3342ms | 81.0752 Ops/s | 78.6776 Ops/s | $\color{#35bf28}+3.05\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 1.5851ms | 1.0401ms | 961.4550 Ops/s | 947.3686 Ops/s | $\color{#35bf28}+1.49\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1051s | 5.7190ms | 174.8553 Ops/s | 173.6181 Ops/s | $\color{#35bf28}+0.71\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 14.8713ms | 12.4526ms | 80.3045 Ops/s | 79.2669 Ops/s | $\color{#35bf28}+1.31\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.5248ms | 1.0461ms | 955.9589 Ops/s | 950.1361 Ops/s | $\color{#35bf28}+0.61\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1015s | 5.9668ms | 167.5934 Ops/s | 129.0830 Ops/s | $\textbf{\color{#35bf28}+29.83\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 0.1082s | 14.4469ms | 69.2190 Ops/s | 77.0155 Ops/s | $\textbf{\color{#d91a1a}-10.12\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 3.9211ms | 1.2183ms | 820.8156 Ops/s | 832.8310 Ops/s | $\color{#d91a1a}-1.44\\%$ |
github-actions[bot] commented 4 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1200s | 0.1193s | 8.3814 Ops/s | 8.4021 Ops/s | $\color{#d91a1a}-0.25\\%$ | | test_sync | 0.1070s | 0.1060s | 9.4325 Ops/s | 9.6129 Ops/s | $\color{#d91a1a}-1.88\\%$ | | test_async | 0.2000s | 96.7500ms | 10.3359 Ops/s | 12.2348 Ops/s | $\textbf{\color{#d91a1a}-15.52\\%}$ | | test_single_pixels | 0.1300s | 0.1296s | 7.7190 Ops/s | 7.6658 Ops/s | $\color{#35bf28}+0.69\\%$ | | test_sync_pixels | 85.7669ms | 82.2344ms | 12.1604 Ops/s | 11.9589 Ops/s | $\color{#35bf28}+1.68\\%$ | | test_async_pixels | 0.1654s | 70.6496ms | 14.1544 Ops/s | 14.3439 Ops/s | $\color{#d91a1a}-1.32\\%$ | | test_simple | 0.8910s | 0.8278s | 1.2081 Ops/s | 1.2234 Ops/s | $\color{#d91a1a}-1.25\\%$ | | test_transformed | 1.1458s | 1.0853s | 0.9214 Ops/s | 0.9203 Ops/s | $\color{#35bf28}+0.12\\%$ | | test_serial | 2.5769s | 2.5152s | 0.3976 Ops/s | 0.3964 Ops/s | $\color{#35bf28}+0.30\\%$ | | test_parallel | 2.4411s | 2.3770s | 0.4207 Ops/s | 0.4179 Ops/s | $\color{#35bf28}+0.68\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 94.2220μs | 33.7291μs | 29.6480 KOps/s | 29.3090 KOps/s | $\color{#35bf28}+1.16\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 45.9010μs | 19.7848μs | 50.5439 KOps/s | 49.7303 KOps/s | $\color{#35bf28}+1.64\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 43.1110μs | 19.3521μs | 51.6741 KOps/s | 51.2202 KOps/s | $\color{#35bf28}+0.89\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 28.1110μs | 11.4120μs | 87.6274 KOps/s | 86.4548 KOps/s | $\color{#35bf28}+1.36\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 57.5910μs | 35.3568μs | 28.2831 KOps/s | 27.2969 KOps/s | $\color{#35bf28}+3.61\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 42.9310μs | 21.7359μs | 46.0067 KOps/s | 45.0771 KOps/s | $\color{#35bf28}+2.06\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 41.1700μs | 21.2968μs | 46.9554 KOps/s | 46.7574 KOps/s | $\color{#35bf28}+0.42\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 34.5400μs | 13.3935μs | 74.6630 KOps/s | 74.5175 KOps/s | $\color{#35bf28}+0.20\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 62.9910μs | 36.8699μs | 27.1224 KOps/s | 27.1031 KOps/s | $\color{#35bf28}+0.07\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 45.5610μs | 23.7088μs | 42.1784 KOps/s | 42.4026 KOps/s | $\color{#d91a1a}-0.53\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 41.9810μs | 21.0945μs | 47.4056 KOps/s | 47.4841 KOps/s | $\color{#d91a1a}-0.17\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 35.1910μs | 13.4719μs | 74.2283 KOps/s | 73.5251 KOps/s | $\color{#35bf28}+0.96\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 72.9310μs | 39.0524μs | 25.6066 KOps/s | 25.0093 KOps/s | $\color{#35bf28}+2.39\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 48.2010μs | 25.6774μs | 38.9448 KOps/s | 38.9988 KOps/s | $\color{#d91a1a}-0.14\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 42.5610μs | 22.7320μs | 43.9908 KOps/s | 44.0891 KOps/s | $\color{#d91a1a}-0.22\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 37.0210μs | 15.3886μs | 64.9831 KOps/s | 66.2197 KOps/s | $\color{#d91a1a}-1.87\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 62.4810μs | 37.8347μs | 26.4307 KOps/s | 26.4965 KOps/s | $\color{#d91a1a}-0.25\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 45.1200μs | 24.1520μs | 41.4045 KOps/s | 42.5309 KOps/s | $\color{#d91a1a}-2.65\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 52.9700μs | 25.2616μs | 39.5858 KOps/s | 39.7570 KOps/s | $\color{#d91a1a}-0.43\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 34.4100μs | 15.4534μs | 64.7106 KOps/s | 66.5483 KOps/s | $\color{#d91a1a}-2.76\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 67.8220μs | 39.5194μs | 25.3040 KOps/s | 25.7402 KOps/s | $\color{#d91a1a}-1.69\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 56.3910μs | 25.7119μs | 38.8926 KOps/s | 39.1385 KOps/s | $\color{#d91a1a}-0.63\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 47.8910μs | 26.7056μs | 37.4453 KOps/s | 36.8069 KOps/s | $\color{#35bf28}+1.73\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 43.0810μs | 17.0128μs | 58.7794 KOps/s | 58.4633 KOps/s | $\color{#35bf28}+0.54\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 81.6010μs | 40.5681μs | 24.6499 KOps/s | 24.2152 KOps/s | $\color{#35bf28}+1.79\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 44.8210μs | 27.3815μs | 36.5211 KOps/s | 36.5777 KOps/s | $\color{#d91a1a}-0.15\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 42.1700μs | 26.8683μs | 37.2186 KOps/s | 35.6953 KOps/s | $\color{#35bf28}+4.27\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 37.3710μs | 16.9572μs | 58.9721 KOps/s | 58.6449 KOps/s | $\color{#35bf28}+0.56\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 78.0810μs | 42.9633μs | 23.2757 KOps/s | 22.2339 KOps/s | $\color{#35bf28}+4.69\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 61.9710μs | 29.8565μs | 33.4936 KOps/s | 33.5080 KOps/s | $\color{#d91a1a}-0.04\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 55.0610μs | 28.3520μs | 35.2708 KOps/s | 34.3706 KOps/s | $\color{#35bf28}+2.62\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 38.2500μs | 18.9257μs | 52.8383 KOps/s | 52.0651 KOps/s | $\color{#35bf28}+1.49\\%$ | | test_values[generalized_advantage_estimate-True-True] | 25.5266ms | 25.0787ms | 39.8745 Ops/s | 39.3215 Ops/s | $\color{#35bf28}+1.41\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 88.3593ms | 2.6752ms | 373.8057 Ops/s | 367.4463 Ops/s | $\color{#35bf28}+1.73\\%$ | | test_values[td0_return_estimate-False-False] | 92.3610μs | 67.6260μs | 14.7872 KOps/s | 14.9860 KOps/s | $\color{#d91a1a}-1.33\\%$ | | test_values[td1_return_estimate-False-False] | 57.2123ms | 56.2160ms | 17.7885 Ops/s | 17.9015 Ops/s | $\color{#d91a1a}-0.63\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.4189ms | 1.1006ms | 908.6011 Ops/s | 907.6022 Ops/s | $\color{#35bf28}+0.11\\%$ | | test_values[td_lambda_return_estimate-True-False] | 90.0255ms | 88.6813ms | 11.2763 Ops/s | 11.4403 Ops/s | $\color{#d91a1a}-1.43\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.4263ms | 1.1005ms | 908.6859 Ops/s | 919.4338 Ops/s | $\color{#d91a1a}-1.17\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 25.9161ms | 25.5785ms | 39.0954 Ops/s | 39.7269 Ops/s | $\color{#d91a1a}-1.59\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9676ms | 0.7330ms | 1.3643 KOps/s | 1.3773 KOps/s | $\color{#d91a1a}-0.95\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7755ms | 0.6896ms | 1.4501 KOps/s | 1.4725 KOps/s | $\color{#d91a1a}-1.52\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5355ms | 1.4848ms | 673.4794 Ops/s | 677.6192 Ops/s | $\color{#d91a1a}-0.61\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7516ms | 0.7032ms | 1.4221 KOps/s | 1.4388 KOps/s | $\color{#d91a1a}-1.16\\%$ | | test_dqn_speed | 2.7772ms | 1.4944ms | 669.1791 Ops/s | 651.6699 Ops/s | $\color{#35bf28}+2.69\\%$ | | test_ddpg_speed | 3.4405ms | 3.0807ms | 324.6066 Ops/s | 321.6381 Ops/s | $\color{#35bf28}+0.92\\%$ | | test_sac_speed | 9.0069ms | 8.6292ms | 115.8862 Ops/s | 114.5757 Ops/s | $\color{#35bf28}+1.14\\%$ | | test_redq_speed | 11.9933ms | 10.8832ms | 91.8849 Ops/s | 83.0230 Ops/s | $\textbf{\color{#35bf28}+10.67\\%}$ | | test_redq_deprec_speed | 12.8999ms | 12.0038ms | 83.3072 Ops/s | 80.8237 Ops/s | $\color{#35bf28}+3.07\\%$ | | test_td3_speed | 17.8657ms | 8.6872ms | 115.1116 Ops/s | 116.2024 Ops/s | $\color{#d91a1a}-0.94\\%$ | | test_cql_speed | 27.4571ms | 26.4991ms | 37.7371 Ops/s | 37.1315 Ops/s | $\color{#35bf28}+1.63\\%$ | | test_a2c_speed | 6.0927ms | 5.8196ms | 171.8337 Ops/s | 172.6850 Ops/s | $\color{#d91a1a}-0.49\\%$ | | test_ppo_speed | 6.5719ms | 6.1879ms | 161.6063 Ops/s | 163.8475 Ops/s | $\color{#d91a1a}-1.37\\%$ | | test_reinforce_speed | 5.1094ms | 4.7662ms | 209.8099 Ops/s | 207.6182 Ops/s | $\color{#35bf28}+1.06\\%$ | | test_iql_speed | 21.8247ms | 20.8731ms | 47.9087 Ops/s | 48.4954 Ops/s | $\color{#d91a1a}-1.21\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.1167ms | 4.8286ms | 207.0981 Ops/s | 208.7711 Ops/s | $\color{#d91a1a}-0.80\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.7829ms | 0.6335ms | 1.5786 KOps/s | 1.5913 KOps/s | $\color{#d91a1a}-0.80\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 1.0918ms | 0.6177ms | 1.6188 KOps/s | 1.6104 KOps/s | $\color{#35bf28}+0.53\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.0117ms | 4.7498ms | 210.5355 Ops/s | 209.6574 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.4783ms | 0.6210ms | 1.6104 KOps/s | 1.6150 KOps/s | $\color{#d91a1a}-0.28\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7550ms | 0.6042ms | 1.6550 KOps/s | 1.6326 KOps/s | $\color{#35bf28}+1.37\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 5.1844ms | 2.2130ms | 451.8811 Ops/s | 452.8737 Ops/s | $\color{#d91a1a}-0.22\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2812ms | 2.0776ms | 481.3264 Ops/s | 480.8920 Ops/s | $\color{#35bf28}+0.09\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0328ms | 4.8250ms | 207.2536 Ops/s | 207.5260 Ops/s | $\color{#d91a1a}-0.13\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8346ms | 0.7274ms | 1.3748 KOps/s | 1.1262 KOps/s | $\textbf{\color{#35bf28}+22.07\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8761ms | 0.7144ms | 1.3997 KOps/s | 1.3457 KOps/s | $\color{#35bf28}+4.02\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.8380ms | 4.7068ms | 212.4566 Ops/s | 209.4683 Ops/s | $\color{#35bf28}+1.43\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7445ms | 0.6209ms | 1.6106 KOps/s | 1.5960 KOps/s | $\color{#35bf28}+0.92\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 4.7210ms | 0.6050ms | 1.6530 KOps/s | 1.6390 KOps/s | $\color{#35bf28}+0.85\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.9478ms | 4.6844ms | 213.4743 Ops/s | 213.2538 Ops/s | $\color{#35bf28}+0.10\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.6293ms | 0.6126ms | 1.6325 KOps/s | 1.6230 KOps/s | $\color{#35bf28}+0.58\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7370ms | 0.5954ms | 1.6794 KOps/s | 1.6639 KOps/s | $\color{#35bf28}+0.93\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.9354ms | 4.7981ms | 208.4140 Ops/s | 203.2645 Ops/s | $\color{#35bf28}+2.53\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8601ms | 0.7345ms | 1.3615 KOps/s | 1.3100 KOps/s | $\color{#35bf28}+3.93\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.7428ms | 0.7209ms | 1.3871 KOps/s | 1.3337 KOps/s | $\color{#35bf28}+4.00\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1240s | 7.3502ms | 136.0514 Ops/s | 127.5616 Ops/s | $\textbf{\color{#35bf28}+6.66\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.3466ms | 15.8354ms | 63.1498 Ops/s | 62.3548 Ops/s | $\color{#35bf28}+1.28\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.3830ms | 1.3371ms | 747.8834 Ops/s | 755.2043 Ops/s | $\color{#d91a1a}-0.97\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1182s | 7.2360ms | 138.1987 Ops/s | 103.8330 Ops/s | $\textbf{\color{#35bf28}+33.10\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1299s | 18.3391ms | 54.5282 Ops/s | 60.6628 Ops/s | $\textbf{\color{#d91a1a}-10.11\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.5596ms | 1.4213ms | 703.5816 Ops/s | 730.1893 Ops/s | $\color{#d91a1a}-3.64\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1198s | 7.5173ms | 133.0272 Ops/s | 131.1233 Ops/s | $\color{#35bf28}+1.45\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.9154ms | 15.9756ms | 62.5956 Ops/s | 60.0093 Ops/s | $\color{#35bf28}+4.31\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.6074ms | 1.4933ms | 669.6359 Ops/s | 595.9731 Ops/s | $\textbf{\color{#35bf28}+12.36\\%}$ |