search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.27k stars 302 forks source link

[Feature] PendulumEnv #2302

Closed vmoens closed 2 months ago

vmoens commented 2 months ago

Stack from ghstack (oldest at bottom):

pytorch-bot[bot] commented 2 months ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2302

Note: Links to docs will display an error until the docs builds have been completed.

:x: 2 New Failures, 1 Pending, 3 Unrelated Failures

As of commit f3cf01762e395cff021cf969457b036d3aa0a8e1 with merge base c771e6e858faa84a9084c7e3993c3d0a134d8ea3 (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2302#27751688538) ([gh](https://github.com/pytorch/rl/actions/runs/10042088028/job/27751688538)) `RuntimeError: Command docker exec -t 940e3a68166270a693d3f36e3bd5e2e2f2ef7c4bbdb87ce19e3def81fe57ba3f /exec failed with exit code 139` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2302#27751697267) ([gh](https://github.com/pytorch/rl/actions/runs/10042088037/job/27751697267)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2302#27751697942) ([gh](https://github.com/pytorch/rl/actions/runs/10042088039/job/27751697942)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/c771e6e858faa84a9084c7e3993c3d0a134d8ea3#27674777898)) `AttributeError: module 'torch' has no attribute 'compiler'` * [Libs Tests on Linux / unittests-robohive (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2302#27751698607) ([gh](https://github.com/pytorch/rl/actions/runs/10042088039/job/27751698607)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/c771e6e858faa84a9084c7e3993c3d0a134d8ea3#27674778892)) `test/test_libs.py::TestRoboHive::test_robohive[franka_slide_random-v3-True-True]` * [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2302#27751695241) ([gh](https://github.com/pytorch/rl/actions/runs/10042088051/job/27751695241)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/c771e6e858faa84a9084c7e3993c3d0a134d8ea3#27674776256)) `AttributeError: module 'torch' has no attribute 'compiler'`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 2 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 65.9668ms | 58.6763ms | 17.0426 Ops/s | 17.0504 Ops/s | $\color{#d91a1a}-0.05\\%$ | | test_sync | 47.0845ms | 33.0467ms | 30.2602 Ops/s | 31.2715 Ops/s | $\color{#d91a1a}-3.23\\%$ | | test_async | 71.0826ms | 30.2565ms | 33.0507 Ops/s | 33.1248 Ops/s | $\color{#d91a1a}-0.22\\%$ | | test_simple | 0.4818s | 0.4107s | 2.4350 Ops/s | 2.3895 Ops/s | $\color{#35bf28}+1.90\\%$ | | test_transformed | 0.6292s | 0.5678s | 1.7610 Ops/s | 1.7200 Ops/s | $\color{#35bf28}+2.39\\%$ | | test_serial | 1.3255s | 1.2639s | 0.7912 Ops/s | 0.7742 Ops/s | $\color{#35bf28}+2.19\\%$ | | test_parallel | 1.1544s | 1.1002s | 0.9089 Ops/s | 0.8992 Ops/s | $\color{#35bf28}+1.08\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 77.7450μs | 24.6590μs | 40.5531 KOps/s | 40.2252 KOps/s | $\color{#35bf28}+0.82\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 57.2890μs | 14.3434μs | 69.7184 KOps/s | 68.9016 KOps/s | $\color{#35bf28}+1.19\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 83.6170μs | 14.1805μs | 70.5192 KOps/s | 69.7382 KOps/s | $\color{#35bf28}+1.12\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 54.9220μs | 8.2942μs | 120.5658 KOps/s | 117.7867 KOps/s | $\color{#35bf28}+2.36\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 60.8430μs | 26.3704μs | 37.9214 KOps/s | 37.4631 KOps/s | $\color{#35bf28}+1.22\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 70.3910μs | 15.9497μs | 62.6969 KOps/s | 61.9387 KOps/s | $\color{#35bf28}+1.22\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 43.7020μs | 15.9238μs | 62.7990 KOps/s | 62.5675 KOps/s | $\color{#35bf28}+0.37\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 73.9960μs | 9.8450μs | 101.5742 KOps/s | 100.1371 KOps/s | $\color{#35bf28}+1.44\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 86.0430μs | 27.9956μs | 35.7199 KOps/s | 35.3386 KOps/s | $\color{#35bf28}+1.08\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 58.9100μs | 17.5477μs | 56.9875 KOps/s | 57.0739 KOps/s | $\color{#d91a1a}-0.15\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 71.1230μs | 15.8029μs | 63.2797 KOps/s | 62.6432 KOps/s | $\color{#35bf28}+1.02\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 33.8730μs | 9.8621μs | 101.3986 KOps/s | 100.6064 KOps/s | $\color{#35bf28}+0.79\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 81.5020μs | 29.5050μs | 33.8925 KOps/s | 33.6444 KOps/s | $\color{#35bf28}+0.74\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 46.0960μs | 19.1348μs | 52.2608 KOps/s | 52.2465 KOps/s | $\color{#35bf28}+0.03\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 62.5870μs | 17.2131μs | 58.0951 KOps/s | 56.9304 KOps/s | $\color{#35bf28}+2.05\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 68.5380μs | 11.3515μs | 88.0941 KOps/s | 88.4014 KOps/s | $\color{#d91a1a}-0.35\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 68.1170μs | 28.2236μs | 35.4313 KOps/s | 35.4012 KOps/s | $\color{#35bf28}+0.09\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 74.1590μs | 17.4133μs | 57.4273 KOps/s | 56.7034 KOps/s | $\color{#35bf28}+1.28\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 49.7930μs | 18.1915μs | 54.9707 KOps/s | 52.8655 KOps/s | $\color{#35bf28}+3.98\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 72.1750μs | 11.0242μs | 90.7091 KOps/s | 89.1005 KOps/s | $\color{#35bf28}+1.81\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 0.1665ms | 30.1830μs | 33.1312 KOps/s | 33.7215 KOps/s | $\color{#d91a1a}-1.75\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 75.2810μs | 19.0366μs | 52.5303 KOps/s | 52.4310 KOps/s | $\color{#35bf28}+0.19\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 64.6710μs | 19.8101μs | 50.4792 KOps/s | 50.0053 KOps/s | $\color{#35bf28}+0.95\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 55.2120μs | 12.4779μs | 80.1415 KOps/s | 77.8139 KOps/s | $\color{#35bf28}+2.99\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.5411ms | 31.3839μs | 31.8634 KOps/s | 31.4860 KOps/s | $\color{#35bf28}+1.20\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 51.1750μs | 20.6790μs | 48.3583 KOps/s | 47.7555 KOps/s | $\color{#35bf28}+1.26\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 63.5990μs | 20.0989μs | 49.7538 KOps/s | 49.4273 KOps/s | $\color{#35bf28}+0.66\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 39.8940μs | 12.6904μs | 78.7996 KOps/s | 78.2156 KOps/s | $\color{#35bf28}+0.75\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 0.2346ms | 32.7174μs | 30.5648 KOps/s | 30.5030 KOps/s | $\color{#35bf28}+0.20\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 72.1150μs | 22.1371μs | 45.1730 KOps/s | 45.2954 KOps/s | $\color{#d91a1a}-0.27\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 49.2120μs | 21.4504μs | 46.6192 KOps/s | 46.8983 KOps/s | $\color{#d91a1a}-0.60\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 49.9940μs | 13.9872μs | 71.4941 KOps/s | 70.8309 KOps/s | $\color{#35bf28}+0.94\\%$ | | test_values[generalized_advantage_estimate-True-True] | 10.1229ms | 9.4741ms | 105.5512 Ops/s | 107.2644 Ops/s | $\color{#d91a1a}-1.60\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 38.9946ms | 35.7688ms | 27.9573 Ops/s | 27.9830 Ops/s | $\color{#d91a1a}-0.09\\%$ | | test_values[td0_return_estimate-False-False] | 0.2313ms | 0.1696ms | 5.8965 KOps/s | 5.9347 KOps/s | $\color{#d91a1a}-0.64\\%$ | | test_values[td1_return_estimate-False-False] | 24.3421ms | 23.8526ms | 41.9242 Ops/s | 42.5168 Ops/s | $\color{#d91a1a}-1.39\\%$ | | test_values[vec_td1_return_estimate-False-False] | 38.1823ms | 35.7452ms | 27.9758 Ops/s | 27.9451 Ops/s | $\color{#35bf28}+0.11\\%$ | | test_values[td_lambda_return_estimate-True-False] | 38.1340ms | 34.4960ms | 28.9888 Ops/s | 29.3068 Ops/s | $\color{#d91a1a}-1.08\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 37.6989ms | 35.8706ms | 27.8780 Ops/s | 27.9412 Ops/s | $\color{#d91a1a}-0.23\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 11.6228ms | 8.4203ms | 118.7605 Ops/s | 121.7395 Ops/s | $\color{#d91a1a}-2.45\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.9779ms | 2.0653ms | 484.1890 Ops/s | 559.6990 Ops/s | $\textbf{\color{#d91a1a}-13.49\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4580ms | 0.3588ms | 2.7869 KOps/s | 2.7879 KOps/s | $\color{#d91a1a}-0.04\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 50.2721ms | 47.0181ms | 21.2684 Ops/s | 21.4117 Ops/s | $\color{#d91a1a}-0.67\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 4.1558ms | 3.1285ms | 319.6372 Ops/s | 329.2942 Ops/s | $\color{#d91a1a}-2.93\\%$ | | test_dqn_speed | 1.9657ms | 1.3814ms | 723.9197 Ops/s | 718.4789 Ops/s | $\color{#35bf28}+0.76\\%$ | | test_ddpg_speed | 3.2526ms | 2.8981ms | 345.0516 Ops/s | 329.3607 Ops/s | $\color{#35bf28}+4.76\\%$ | | test_sac_speed | 10.9086ms | 8.6159ms | 116.0639 Ops/s | 116.4375 Ops/s | $\color{#d91a1a}-0.32\\%$ | | test_redq_speed | 14.7319ms | 13.4489ms | 74.3555 Ops/s | 74.1058 Ops/s | $\color{#35bf28}+0.34\\%$ | | test_redq_deprec_speed | 15.1386ms | 13.3146ms | 75.1053 Ops/s | 75.0482 Ops/s | $\color{#35bf28}+0.08\\%$ | | test_td3_speed | 8.9003ms | 8.5594ms | 116.8306 Ops/s | 115.8584 Ops/s | $\color{#35bf28}+0.84\\%$ | | test_cql_speed | 38.2140ms | 36.6732ms | 27.2679 Ops/s | 27.2165 Ops/s | $\color{#35bf28}+0.19\\%$ | | test_a2c_speed | 9.3497ms | 7.5423ms | 132.5857 Ops/s | 133.3334 Ops/s | $\color{#d91a1a}-0.56\\%$ | | test_ppo_speed | 8.5334ms | 7.8069ms | 128.0924 Ops/s | 127.8311 Ops/s | $\color{#35bf28}+0.20\\%$ | | test_reinforce_speed | 8.1963ms | 6.6869ms | 149.5463 Ops/s | 149.3196 Ops/s | $\color{#35bf28}+0.15\\%$ | | test_iql_speed | 34.1773ms | 32.9455ms | 30.3532 Ops/s | 30.3038 Ops/s | $\color{#35bf28}+0.16\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.6533ms | 4.3958ms | 227.4894 Ops/s | 228.9615 Ops/s | $\color{#d91a1a}-0.64\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6817ms | 0.4837ms | 2.0674 KOps/s | 2.0698 KOps/s | $\color{#d91a1a}-0.12\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 1.5735ms | 0.4579ms | 2.1837 KOps/s | 2.1594 KOps/s | $\color{#35bf28}+1.13\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.5756ms | 4.3174ms | 231.6197 Ops/s | 228.3727 Ops/s | $\color{#35bf28}+1.42\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.6334ms | 0.4772ms | 2.0957 KOps/s | 2.0644 KOps/s | $\color{#35bf28}+1.52\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.8033ms | 0.4537ms | 2.2043 KOps/s | 2.1260 KOps/s | $\color{#35bf28}+3.68\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.2529ms | 1.7156ms | 582.8835 Ops/s | 588.6058 Ops/s | $\color{#d91a1a}-0.97\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.7482ms | 1.6249ms | 615.4231 Ops/s | 617.7361 Ops/s | $\color{#d91a1a}-0.37\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 7.0022ms | 4.4374ms | 225.3552 Ops/s | 220.5002 Ops/s | $\color{#35bf28}+2.20\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.7827ms | 0.6231ms | 1.6048 KOps/s | 1.3838 KOps/s | $\textbf{\color{#35bf28}+15.97\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 1.4504ms | 0.5929ms | 1.6867 KOps/s | 1.6682 KOps/s | $\color{#35bf28}+1.11\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.4237ms | 4.2541ms | 235.0665 Ops/s | 228.0468 Ops/s | $\color{#35bf28}+3.08\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.6599ms | 0.6574ms | 1.5211 KOps/s | 2.0771 KOps/s | $\textbf{\color{#d91a1a}-26.77\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 1.6294ms | 0.5369ms | 1.8625 KOps/s | 2.1482 KOps/s | $\textbf{\color{#d91a1a}-13.30\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.6828ms | 4.3091ms | 232.0667 Ops/s | 229.4411 Ops/s | $\color{#35bf28}+1.14\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.6620ms | 0.6059ms | 1.6505 KOps/s | 2.0435 KOps/s | $\textbf{\color{#d91a1a}-19.23\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 7.4497ms | 0.5849ms | 1.7098 KOps/s | 2.1476 KOps/s | $\textbf{\color{#d91a1a}-20.38\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.7513ms | 4.5165ms | 221.4102 Ops/s | 220.0870 Ops/s | $\color{#35bf28}+0.60\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.8063ms | 0.9230ms | 1.0834 KOps/s | 1.6014 KOps/s | $\textbf{\color{#d91a1a}-32.35\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 7.6835ms | 0.7299ms | 1.3701 KOps/s | 1.6771 KOps/s | $\textbf{\color{#d91a1a}-18.30\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1164s | 8.1640ms | 122.4883 Ops/s | 153.9727 Ops/s | $\textbf{\color{#d91a1a}-20.45\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 16.9770ms | 12.9893ms | 76.9862 Ops/s | 77.0694 Ops/s | $\color{#d91a1a}-0.11\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 5.5078ms | 1.1767ms | 849.8244 Ops/s | 825.8662 Ops/s | $\color{#35bf28}+2.90\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1104s | 5.9079ms | 169.2637 Ops/s | 121.1530 Ops/s | $\textbf{\color{#35bf28}+39.71\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 16.9796ms | 13.0040ms | 76.8995 Ops/s | 76.9001 Ops/s | $-0.00\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.7477ms | 1.0717ms | 933.1358 Ops/s | 866.3885 Ops/s | $\textbf{\color{#35bf28}+7.70\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1141s | 6.1052ms | 163.7952 Ops/s | 162.2095 Ops/s | $\color{#35bf28}+0.98\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 24.4680ms | 13.5153ms | 73.9902 Ops/s | 76.2726 Ops/s | $\color{#d91a1a}-2.99\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 5.3812ms | 1.3119ms | 762.2647 Ops/s | 791.0355 Ops/s | $\color{#d91a1a}-3.64\\%$ |
github-actions[bot] commented 2 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1075s | 0.1074s | 9.3137 Ops/s | 8.4374 Ops/s | $\textbf{\color{#35bf28}+10.39\\%}$ | | test_sync | 95.2952ms | 94.4331ms | 10.5895 Ops/s | 10.6316 Ops/s | $\color{#d91a1a}-0.40\\%$ | | test_async | 0.1737s | 89.7864ms | 11.1375 Ops/s | 11.0617 Ops/s | $\color{#35bf28}+0.69\\%$ | | test_single_pixels | 0.1193s | 0.1177s | 8.4926 Ops/s | 8.4691 Ops/s | $\color{#35bf28}+0.28\\%$ | | test_sync_pixels | 76.0976ms | 73.1467ms | 13.6712 Ops/s | 13.5872 Ops/s | $\color{#35bf28}+0.62\\%$ | | test_async_pixels | 0.1328s | 69.1004ms | 14.4717 Ops/s | 14.2863 Ops/s | $\color{#35bf28}+1.30\\%$ | | test_simple | 0.8727s | 0.7954s | 1.2572 Ops/s | 1.2838 Ops/s | $\color{#d91a1a}-2.07\\%$ | | test_transformed | 1.0996s | 1.0283s | 0.9724 Ops/s | 0.9878 Ops/s | $\color{#d91a1a}-1.55\\%$ | | test_serial | 2.3321s | 2.2589s | 0.4427 Ops/s | 0.4459 Ops/s | $\color{#d91a1a}-0.71\\%$ | | test_parallel | 2.0502s | 1.9879s | 0.5030 Ops/s | 0.4956 Ops/s | $\color{#35bf28}+1.49\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1050ms | 38.3611μs | 26.0681 KOps/s | 26.6086 KOps/s | $\color{#d91a1a}-2.03\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 46.3000μs | 21.5635μs | 46.3746 KOps/s | 45.9931 KOps/s | $\color{#35bf28}+0.83\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 0.1289ms | 21.7187μs | 46.0432 KOps/s | 46.8879 KOps/s | $\color{#d91a1a}-1.80\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 41.8610μs | 12.3705μs | 80.8374 KOps/s | 81.1112 KOps/s | $\color{#d91a1a}-0.34\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 0.2321ms | 40.2304μs | 24.8568 KOps/s | 25.3106 KOps/s | $\color{#d91a1a}-1.79\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 0.2213ms | 23.8885μs | 41.8612 KOps/s | 42.1414 KOps/s | $\color{#d91a1a}-0.66\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 0.2206ms | 23.7569μs | 42.0931 KOps/s | 43.5888 KOps/s | $\color{#d91a1a}-3.43\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 0.1933ms | 14.5603μs | 68.6798 KOps/s | 67.9058 KOps/s | $\color{#35bf28}+1.14\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 71.2710μs | 43.0378μs | 23.2354 KOps/s | 23.7067 KOps/s | $\color{#d91a1a}-1.99\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 0.1088ms | 26.5286μs | 37.6952 KOps/s | 38.7802 KOps/s | $\color{#d91a1a}-2.80\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 42.5710μs | 23.8643μs | 41.9035 KOps/s | 43.2353 KOps/s | $\color{#d91a1a}-3.08\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 37.1610μs | 14.6342μs | 68.3329 KOps/s | 68.8469 KOps/s | $\color{#d91a1a}-0.75\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 63.6510μs | 44.6515μs | 22.3956 KOps/s | 22.8729 KOps/s | $\color{#d91a1a}-2.09\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 51.4810μs | 29.0032μs | 34.4790 KOps/s | 35.6165 KOps/s | $\color{#d91a1a}-3.19\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 52.3710μs | 25.8774μs | 38.6437 KOps/s | 39.3382 KOps/s | $\color{#d91a1a}-1.77\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 34.1010μs | 17.0062μs | 58.8022 KOps/s | 58.6709 KOps/s | $\color{#35bf28}+0.22\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 62.7910μs | 43.3077μs | 23.0906 KOps/s | 23.8995 KOps/s | $\color{#d91a1a}-3.38\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 48.1010μs | 26.5819μs | 37.6196 KOps/s | 38.4182 KOps/s | $\color{#d91a1a}-2.08\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 47.3710μs | 28.8051μs | 34.7161 KOps/s | 35.8439 KOps/s | $\color{#d91a1a}-3.15\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 45.8510μs | 16.7012μs | 59.8758 KOps/s | 60.5762 KOps/s | $\color{#d91a1a}-1.16\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 88.1110μs | 44.5883μs | 22.4274 KOps/s | 23.2563 KOps/s | $\color{#d91a1a}-3.56\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 48.9900μs | 28.7711μs | 34.7571 KOps/s | 35.3042 KOps/s | $\color{#d91a1a}-1.55\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 53.9910μs | 30.6908μs | 32.5831 KOps/s | 33.4637 KOps/s | $\color{#d91a1a}-2.63\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 39.3810μs | 19.1308μs | 52.2717 KOps/s | 52.5194 KOps/s | $\color{#d91a1a}-0.47\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 4.0387ms | 48.3690μs | 20.6744 KOps/s | 21.1022 KOps/s | $\color{#d91a1a}-2.03\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 49.0000μs | 31.3367μs | 31.9115 KOps/s | 32.9299 KOps/s | $\color{#d91a1a}-3.09\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 47.9710μs | 30.8514μs | 32.4134 KOps/s | 32.9827 KOps/s | $\color{#d91a1a}-1.73\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 36.4600μs | 19.2013μs | 52.0797 KOps/s | 52.2425 KOps/s | $\color{#d91a1a}-0.31\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 72.5310μs | 49.9864μs | 20.0054 KOps/s | 20.6475 KOps/s | $\color{#d91a1a}-3.11\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 59.4910μs | 33.8348μs | 29.5554 KOps/s | 30.4795 KOps/s | $\color{#d91a1a}-3.03\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 51.1910μs | 32.8565μs | 30.4354 KOps/s | 31.2237 KOps/s | $\color{#d91a1a}-2.52\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 38.8010μs | 21.2699μs | 47.0148 KOps/s | 47.9145 KOps/s | $\color{#d91a1a}-1.88\\%$ | | test_values[generalized_advantage_estimate-True-True] | 25.3744ms | 24.8624ms | 40.2213 Ops/s | 40.2358 Ops/s | $\color{#d91a1a}-0.04\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 90.2781ms | 2.7062ms | 369.5249 Ops/s | 369.9229 Ops/s | $\color{#d91a1a}-0.11\\%$ | | test_values[td0_return_estimate-False-False] | 95.9020μs | 67.1025μs | 14.9026 KOps/s | 14.7876 KOps/s | $\color{#35bf28}+0.78\\%$ | | test_values[td1_return_estimate-False-False] | 56.3456ms | 55.9587ms | 17.8703 Ops/s | 17.8512 Ops/s | $\color{#35bf28}+0.11\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.4086ms | 1.0915ms | 916.1535 Ops/s | 913.9958 Ops/s | $\color{#35bf28}+0.24\\%$ | | test_values[td_lambda_return_estimate-True-False] | 93.5418ms | 89.7604ms | 11.1408 Ops/s | 11.2830 Ops/s | $\color{#d91a1a}-1.26\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.4038ms | 1.0956ms | 912.7436 Ops/s | 915.2998 Ops/s | $\color{#d91a1a}-0.28\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 25.2228ms | 24.9201ms | 40.1283 Ops/s | 40.2526 Ops/s | $\color{#d91a1a}-0.31\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 1.1121ms | 0.7320ms | 1.3661 KOps/s | 1.3737 KOps/s | $\color{#d91a1a}-0.56\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.8210ms | 0.6753ms | 1.4809 KOps/s | 1.4691 KOps/s | $\color{#35bf28}+0.81\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.6383ms | 1.4733ms | 678.7480 Ops/s | 677.5053 Ops/s | $\color{#35bf28}+0.18\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.8880ms | 0.7074ms | 1.4136 KOps/s | 1.4399 KOps/s | $\color{#d91a1a}-1.83\\%$ | | test_dqn_speed | 7.2986ms | 1.4829ms | 674.3644 Ops/s | 678.4461 Ops/s | $\color{#d91a1a}-0.60\\%$ | | test_ddpg_speed | 3.2260ms | 2.9928ms | 334.1354 Ops/s | 334.0824 Ops/s | $\color{#35bf28}+0.02\\%$ | | test_sac_speed | 0.1037s | 9.4710ms | 105.5856 Ops/s | 115.9727 Ops/s | $\textbf{\color{#d91a1a}-8.96\\%}$ | | test_redq_speed | 12.2649ms | 10.8732ms | 91.9696 Ops/s | 91.0767 Ops/s | $\color{#35bf28}+0.98\\%$ | | test_redq_deprec_speed | 12.5411ms | 11.7593ms | 85.0390 Ops/s | 86.4781 Ops/s | $\color{#d91a1a}-1.66\\%$ | | test_td3_speed | 8.8560ms | 8.5565ms | 116.8698 Ops/s | 115.6915 Ops/s | $\color{#35bf28}+1.02\\%$ | | test_cql_speed | 27.1407ms | 26.4018ms | 37.8763 Ops/s | 37.2705 Ops/s | $\color{#35bf28}+1.63\\%$ | | test_a2c_speed | 6.8234ms | 5.9463ms | 168.1708 Ops/s | 170.8511 Ops/s | $\color{#d91a1a}-1.57\\%$ | | test_ppo_speed | 6.5750ms | 6.1976ms | 161.3535 Ops/s | 161.1523 Ops/s | $\color{#35bf28}+0.12\\%$ | | test_reinforce_speed | 4.9939ms | 4.7835ms | 209.0527 Ops/s | 208.7170 Ops/s | $\color{#35bf28}+0.16\\%$ | | test_iql_speed | 20.9833ms | 20.2717ms | 49.3298 Ops/s | 48.9586 Ops/s | $\color{#35bf28}+0.76\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.0571ms | 5.9034ms | 169.3948 Ops/s | 169.3495 Ops/s | $\color{#35bf28}+0.03\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.1122s | 0.6035ms | 1.6571 KOps/s | 1.9280 KOps/s | $\textbf{\color{#d91a1a}-14.05\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6701ms | 0.4956ms | 2.0177 KOps/s | 2.0209 KOps/s | $\color{#d91a1a}-0.16\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.1241ms | 5.8155ms | 171.9556 Ops/s | 170.9884 Ops/s | $\color{#35bf28}+0.57\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.9565ms | 0.5114ms | 1.9556 KOps/s | 1.9471 KOps/s | $\color{#35bf28}+0.43\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6479ms | 0.4880ms | 2.0494 KOps/s | 2.0312 KOps/s | $\color{#35bf28}+0.89\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.1951ms | 1.9908ms | 502.2989 Ops/s | 504.4379 Ops/s | $\color{#d91a1a}-0.42\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.0633ms | 1.8755ms | 533.1904 Ops/s | 532.4668 Ops/s | $\color{#35bf28}+0.14\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.2135ms | 5.9909ms | 166.9207 Ops/s | 164.8871 Ops/s | $\color{#35bf28}+1.23\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.1302s | 0.8282ms | 1.2075 KOps/s | 1.4844 KOps/s | $\textbf{\color{#d91a1a}-18.66\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8241ms | 0.6393ms | 1.5643 KOps/s | 1.4683 KOps/s | $\textbf{\color{#35bf28}+6.54\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.0282ms | 5.8699ms | 170.3612 Ops/s | 168.6020 Ops/s | $\color{#35bf28}+1.04\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7743ms | 0.5188ms | 1.9275 KOps/s | 1.4475 KOps/s | $\textbf{\color{#35bf28}+33.17\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6586ms | 0.4975ms | 2.0101 KOps/s | 2.0131 KOps/s | $\color{#d91a1a}-0.15\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.1261ms | 5.8394ms | 171.2511 Ops/s | 170.9279 Ops/s | $\color{#35bf28}+0.19\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.0120ms | 0.5115ms | 1.9550 KOps/s | 1.9439 KOps/s | $\color{#35bf28}+0.57\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6713ms | 0.4894ms | 2.0433 KOps/s | 2.0306 KOps/s | $\color{#35bf28}+0.63\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.1997ms | 6.0149ms | 166.2542 Ops/s | 165.8309 Ops/s | $\color{#35bf28}+0.26\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.6707ms | 0.6676ms | 1.4979 KOps/s | 1.1811 KOps/s | $\textbf{\color{#35bf28}+26.82\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8174ms | 0.6418ms | 1.5582 KOps/s | 1.5505 KOps/s | $\color{#35bf28}+0.49\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1466s | 8.0675ms | 123.9545 Ops/s | 129.6030 Ops/s | $\color{#d91a1a}-4.36\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 19.0842ms | 16.2554ms | 61.5179 Ops/s | 61.6520 Ops/s | $\color{#d91a1a}-0.22\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 6.8424ms | 1.3620ms | 734.2110 Ops/s | 746.6415 Ops/s | $\color{#d91a1a}-1.66\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1260s | 7.6636ms | 130.4878 Ops/s | 130.7271 Ops/s | $\color{#d91a1a}-0.18\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1373s | 18.6874ms | 53.5121 Ops/s | 61.6959 Ops/s | $\textbf{\color{#d91a1a}-13.26\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.9917ms | 1.1247ms | 889.1308 Ops/s | 788.5663 Ops/s | $\textbf{\color{#35bf28}+12.75\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1269s | 7.8549ms | 127.3098 Ops/s | 126.5668 Ops/s | $\color{#35bf28}+0.59\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.9753ms | 16.4923ms | 60.6344 Ops/s | 61.1665 Ops/s | $\color{#d91a1a}-0.87\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 6.3778ms | 1.5046ms | 664.6245 Ops/s | 700.0381 Ops/s | $\textbf{\color{#d91a1a}-5.06\\%}$ |