search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.01k stars 269 forks source link

[BugFix] Fix typo in weight assignment in PRB #2241

Closed vmoens closed 2 weeks ago

vmoens commented 2 weeks ago

Relates to https://github.com/pytorch/rl/pull/2239#discussion_r1647603264

cc @wertyuilife2

pytorch-bot[bot] commented 2 weeks ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2241

Note: Links to docs will display an error until the docs builds have been completed.

:heavy_exclamation_mark: 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

:x: 8 New Failures, 2 Unrelated Failures

As of commit 98c98692d76cee156a62fca869e6db96d14bac29 with merge base 9b1ebb2f63438def890cfb083cf02cdd79daac50 (image):

NEW FAILURES - The following jobs have failed:

* [Examples Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2241#26478881458) ([gh](https://github.com/pytorch/rl/actions/runs/9601097836/job/26478881458)) `curl: (22) The requested URL returned error:` * [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2241#26478881041) ([gh](https://github.com/pytorch/rl/actions/runs/9601097843/job/26478881041)) `curl: (22) The requested URL returned error:` * [Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2241#26478888812) ([gh](https://github.com/pytorch/rl/actions/runs/9601097849/job/26478888812)) `curl: (22) The requested URL returned error:` * [RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2241#26478881463) ([gh](https://github.com/pytorch/rl/actions/runs/9601097844/job/26478881463)) `curl: (22) The requested URL returned error:` * [Unit-tests on Linux / tests-cpu (3.10) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2241#26478881816) ([gh](https://github.com/pytorch/rl/actions/runs/9601097841/job/26478881816)) `curl: (22) The requested URL returned error:` * [Unit-tests on Linux / tests-cpu (3.9) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2241#26478883777) ([gh](https://github.com/pytorch/rl/actions/runs/9601097841/job/26478883777)) `curl: (22) The requested URL returned error:` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2241#26478885756) ([gh](https://github.com/pytorch/rl/actions/runs/9601097841/job/26478885756)) `curl: (22) The requested URL returned error:` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2241#26478880896) ([gh](https://github.com/pytorch/rl/actions/runs/9601097840/job/26478880896)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2241#26478888537) ([gh](https://github.com/pytorch/rl/actions/runs/9601097849/job/26478888537)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/9b1ebb2f63438def890cfb083cf02cdd79daac50#26466937431)) `##[error]The operation was canceled.` * [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2241#26478885251) ([gh](https://github.com/pytorch/rl/actions/runs/9601097841/job/26478885251)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/9b1ebb2f63438def890cfb083cf02cdd79daac50#26466936410)) `##[error]The operation was canceled.`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 2 weeks ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1096s | 57.6220ms | 17.3545 Ops/s | 17.7822 Ops/s | $\color{#d91a1a}-2.41\\%$ | | test_sync | 35.7560ms | 30.4917ms | 32.7958 Ops/s | 32.9476 Ops/s | $\color{#d91a1a}-0.46\\%$ | | test_async | 54.5722ms | 27.8440ms | 35.9143 Ops/s | 34.7015 Ops/s | $\color{#35bf28}+3.50\\%$ | | test_simple | 0.3737s | 0.3714s | 2.6925 Ops/s | 2.6400 Ops/s | $\color{#35bf28}+1.99\\%$ | | test_transformed | 0.5231s | 0.5206s | 1.9208 Ops/s | 1.8807 Ops/s | $\color{#35bf28}+2.13\\%$ | | test_serial | 1.3087s | 1.2518s | 0.7988 Ops/s | 0.7823 Ops/s | $\color{#35bf28}+2.11\\%$ | | test_parallel | 1.1334s | 1.0694s | 0.9351 Ops/s | 0.9337 Ops/s | $\color{#35bf28}+0.15\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1512ms | 22.9554μs | 43.5627 KOps/s | 44.9673 KOps/s | $\color{#d91a1a}-3.12\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 44.1720μs | 13.0594μs | 76.5732 KOps/s | 76.4908 KOps/s | $\color{#35bf28}+0.11\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 34.0630μs | 13.1479μs | 76.0577 KOps/s | 76.8165 KOps/s | $\color{#d91a1a}-0.99\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 33.4420μs | 7.6553μs | 130.6287 KOps/s | 129.1267 KOps/s | $\color{#35bf28}+1.16\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 55.2330μs | 23.7453μs | 42.1136 KOps/s | 42.5508 KOps/s | $\color{#d91a1a}-1.03\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 37.0690μs | 14.3389μs | 69.7402 KOps/s | 70.4638 KOps/s | $\color{#d91a1a}-1.03\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 49.6730μs | 14.3665μs | 69.6065 KOps/s | 70.4076 KOps/s | $\color{#d91a1a}-1.14\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 27.1300μs | 8.8777μs | 112.6412 KOps/s | 112.9021 KOps/s | $\color{#d91a1a}-0.23\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 57.5580μs | 25.0740μs | 39.8819 KOps/s | 40.9309 KOps/s | $\color{#d91a1a}-2.56\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 48.9710μs | 15.5907μs | 64.1407 KOps/s | 64.4602 KOps/s | $\color{#d91a1a}-0.50\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 46.0260μs | 14.4293μs | 69.3033 KOps/s | 70.1842 KOps/s | $\color{#d91a1a}-1.26\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 26.9900μs | 8.9246μs | 112.0492 KOps/s | 112.3416 KOps/s | $\color{#d91a1a}-0.26\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 78.7470μs | 26.5526μs | 37.6611 KOps/s | 38.7602 KOps/s | $\color{#d91a1a}-2.84\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 46.3660μs | 16.8618μs | 59.3058 KOps/s | 59.5883 KOps/s | $\color{#d91a1a}-0.47\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 47.4290μs | 15.5064μs | 64.4893 KOps/s | 65.5827 KOps/s | $\color{#d91a1a}-1.67\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 33.6820μs | 10.1700μs | 98.3284 KOps/s | 98.9431 KOps/s | $\color{#d91a1a}-0.62\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 56.1940μs | 25.3154μs | 39.5017 KOps/s | 40.7859 KOps/s | $\color{#d91a1a}-3.15\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 42.5090μs | 15.7502μs | 63.4914 KOps/s | 63.7212 KOps/s | $\color{#d91a1a}-0.36\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 36.5880μs | 16.7325μs | 59.7638 KOps/s | 60.6006 KOps/s | $\color{#d91a1a}-1.38\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 33.3620μs | 10.1509μs | 98.5133 KOps/s | 99.7196 KOps/s | $\color{#d91a1a}-1.21\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 57.1170μs | 26.2098μs | 38.1537 KOps/s | 38.6331 KOps/s | $\color{#d91a1a}-1.24\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 44.8930μs | 16.9150μs | 59.1193 KOps/s | 59.7443 KOps/s | $\color{#d91a1a}-1.05\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 57.2770μs | 17.7833μs | 56.2326 KOps/s | 56.7780 KOps/s | $\color{#d91a1a}-0.96\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 40.6740μs | 11.3034μs | 88.4691 KOps/s | 89.0631 KOps/s | $\color{#d91a1a}-0.67\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 68.5560μs | 27.6601μs | 36.1532 KOps/s | 36.7346 KOps/s | $\color{#d91a1a}-1.58\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 45.3730μs | 18.2791μs | 54.7074 KOps/s | 55.5825 KOps/s | $\color{#d91a1a}-1.57\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 51.1950μs | 17.7689μs | 56.2782 KOps/s | 57.2464 KOps/s | $\color{#d91a1a}-1.69\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 44.7440μs | 11.3508μs | 88.0992 KOps/s | 88.5829 KOps/s | $\color{#d91a1a}-0.55\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 41.2870μs | 28.5912μs | 34.9758 KOps/s | 34.6318 KOps/s | $\color{#35bf28}+0.99\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 53.8300μs | 19.2224μs | 52.0225 KOps/s | 52.1123 KOps/s | $\color{#d91a1a}-0.17\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 42.5690μs | 18.7227μs | 53.4111 KOps/s | 54.7963 KOps/s | $\color{#d91a1a}-2.53\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 62.1160μs | 12.4525μs | 80.3049 KOps/s | 80.9877 KOps/s | $\color{#d91a1a}-0.84\\%$ | | test_values[generalized_advantage_estimate-True-True] | 14.1833ms | 11.5151ms | 86.8424 Ops/s | 107.3906 Ops/s | $\textbf{\color{#d91a1a}-19.13\\%}$ | | test_values[vec_generalized_advantage_estimate-True-True] | 47.3474ms | 35.4567ms | 28.2034 Ops/s | 28.4435 Ops/s | $\color{#d91a1a}-0.84\\%$ | | test_values[td0_return_estimate-False-False] | 0.2240ms | 0.1684ms | 5.9381 KOps/s | 6.0558 KOps/s | $\color{#d91a1a}-1.94\\%$ | | test_values[td1_return_estimate-False-False] | 35.2612ms | 24.3505ms | 41.0670 Ops/s | 43.5728 Ops/s | $\textbf{\color{#d91a1a}-5.75\\%}$ | | test_values[vec_td1_return_estimate-False-False] | 37.1439ms | 35.2422ms | 28.3751 Ops/s | 28.3353 Ops/s | $\color{#35bf28}+0.14\\%$ | | test_values[td_lambda_return_estimate-True-False] | 34.9684ms | 34.6790ms | 28.8359 Ops/s | 30.1634 Ops/s | $\color{#d91a1a}-4.40\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 36.3784ms | 35.1761ms | 28.4284 Ops/s | 28.3369 Ops/s | $\color{#35bf28}+0.32\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 10.9587ms | 8.5725ms | 116.6517 Ops/s | 121.9971 Ops/s | $\color{#d91a1a}-4.38\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.1983ms | 1.9587ms | 510.5420 Ops/s | 523.6447 Ops/s | $\color{#d91a1a}-2.50\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.5764ms | 0.3619ms | 2.7635 KOps/s | 2.8708 KOps/s | $\color{#d91a1a}-3.74\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 51.4793ms | 46.1917ms | 21.6489 Ops/s | 22.5335 Ops/s | $\color{#d91a1a}-3.93\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.5374ms | 3.0245ms | 330.6374 Ops/s | 331.9288 Ops/s | $\color{#d91a1a}-0.39\\%$ | | test_dqn_speed | 6.9642ms | 1.3580ms | 736.4007 Ops/s | 727.7495 Ops/s | $\color{#35bf28}+1.19\\%$ | | test_ddpg_speed | 3.1862ms | 2.8606ms | 349.5718 Ops/s | 346.6168 Ops/s | $\color{#35bf28}+0.85\\%$ | | test_sac_speed | 9.9452ms | 8.4369ms | 118.5269 Ops/s | 118.1959 Ops/s | $\color{#35bf28}+0.28\\%$ | | test_redq_speed | 14.5866ms | 13.5529ms | 73.7852 Ops/s | 73.8247 Ops/s | $\color{#d91a1a}-0.05\\%$ | | test_redq_deprec_speed | 98.1315ms | 14.8557ms | 67.3142 Ops/s | 73.9091 Ops/s | $\textbf{\color{#d91a1a}-8.92\\%}$ | | test_td3_speed | 10.1937ms | 8.4152ms | 118.8322 Ops/s | 118.0654 Ops/s | $\color{#35bf28}+0.65\\%$ | | test_cql_speed | 42.5067ms | 36.9949ms | 27.0307 Ops/s | 27.2906 Ops/s | $\color{#d91a1a}-0.95\\%$ | | test_a2c_speed | 8.5547ms | 7.4982ms | 133.3647 Ops/s | 133.6829 Ops/s | $\color{#d91a1a}-0.24\\%$ | | test_ppo_speed | 8.7575ms | 7.7551ms | 128.9472 Ops/s | 129.1948 Ops/s | $\color{#d91a1a}-0.19\\%$ | | test_reinforce_speed | 7.8780ms | 6.6385ms | 150.6375 Ops/s | 150.4552 Ops/s | $\color{#35bf28}+0.12\\%$ | | test_iql_speed | 33.9324ms | 32.9743ms | 30.3266 Ops/s | 30.2336 Ops/s | $\color{#35bf28}+0.31\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.3292ms | 3.4961ms | 286.0333 Ops/s | 284.9079 Ops/s | $\color{#35bf28}+0.40\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7465ms | 0.4978ms | 2.0090 KOps/s | 2.0174 KOps/s | $\color{#d91a1a}-0.42\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7716ms | 0.4746ms | 2.1073 KOps/s | 2.1052 KOps/s | $\color{#35bf28}+0.10\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.5496ms | 3.5103ms | 284.8720 Ops/s | 284.4200 Ops/s | $\color{#35bf28}+0.16\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7982ms | 0.4886ms | 2.0465 KOps/s | 2.0403 KOps/s | $\color{#35bf28}+0.31\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7663ms | 0.4705ms | 2.1256 KOps/s | 2.1183 KOps/s | $\color{#35bf28}+0.34\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.4459ms | 1.7504ms | 571.3142 Ops/s | 584.6933 Ops/s | $\color{#d91a1a}-2.29\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.3086ms | 1.6590ms | 602.7775 Ops/s | 611.4049 Ops/s | $\color{#d91a1a}-1.41\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.1019ms | 3.5991ms | 277.8465 Ops/s | 277.8646 Ops/s | $-0.01\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.1886ms | 0.6326ms | 1.5807 KOps/s | 1.3829 KOps/s | $\textbf{\color{#35bf28}+14.30\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 1.0657ms | 0.6118ms | 1.6345 KOps/s | 1.6311 KOps/s | $\color{#35bf28}+0.21\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.0578ms | 3.4818ms | 287.2100 Ops/s | 283.7138 Ops/s | $\color{#35bf28}+1.23\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7616ms | 0.5028ms | 1.9890 KOps/s | 1.9908 KOps/s | $\color{#d91a1a}-0.09\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.5216ms | 0.4798ms | 2.0843 KOps/s | 2.1064 KOps/s | $\color{#d91a1a}-1.05\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.1973ms | 3.5363ms | 282.7800 Ops/s | 279.0532 Ops/s | $\color{#35bf28}+1.34\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7612ms | 0.5011ms | 1.9956 KOps/s | 1.9955 KOps/s | $+0.00\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6340ms | 0.4701ms | 2.1274 KOps/s | 2.0949 KOps/s | $\color{#35bf28}+1.55\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.4617ms | 3.6292ms | 275.5425 Ops/s | 272.4072 Ops/s | $\color{#35bf28}+1.15\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.2052ms | 0.6353ms | 1.5740 KOps/s | 1.5876 KOps/s | $\color{#d91a1a}-0.86\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8755ms | 0.6180ms | 1.6182 KOps/s | 1.6186 KOps/s | $\color{#d91a1a}-0.02\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1193s | 8.1196ms | 123.1585 Ops/s | 165.9817 Ops/s | $\textbf{\color{#d91a1a}-25.80\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 15.6738ms | 12.7972ms | 78.1420 Ops/s | 68.6709 Ops/s | $\textbf{\color{#35bf28}+13.79\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 4.5423ms | 1.1751ms | 850.9827 Ops/s | 952.8801 Ops/s | $\textbf{\color{#d91a1a}-10.69\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1075s | 5.7666ms | 173.4123 Ops/s | 173.6895 Ops/s | $\color{#d91a1a}-0.16\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 15.7500ms | 12.8272ms | 77.9594 Ops/s | 80.6650 Ops/s | $\color{#d91a1a}-3.35\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 3.5786ms | 1.1071ms | 903.2577 Ops/s | 951.5607 Ops/s | $\textbf{\color{#d91a1a}-5.08\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1085s | 5.9440ms | 168.2375 Ops/s | 168.6770 Ops/s | $\color{#d91a1a}-0.26\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 15.5264ms | 12.9081ms | 77.4707 Ops/s | 79.7328 Ops/s | $\color{#d91a1a}-2.84\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.6660ms | 1.2184ms | 820.7628 Ops/s | 806.4349 Ops/s | $\color{#35bf28}+1.78\\%$ |
github-actions[bot] commented 2 weeks ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1174s | 0.1171s | 8.5369 Ops/s | 8.5931 Ops/s | $\color{#d91a1a}-0.65\\%$ | | test_sync | 0.1056s | 0.1032s | 9.6914 Ops/s | 9.6655 Ops/s | $\color{#35bf28}+0.27\\%$ | | test_async | 0.1957s | 98.5080ms | 10.1515 Ops/s | 10.3562 Ops/s | $\color{#d91a1a}-1.98\\%$ | | test_single_pixels | 0.1270s | 0.1246s | 8.0251 Ops/s | 8.0804 Ops/s | $\color{#d91a1a}-0.68\\%$ | | test_sync_pixels | 83.2901ms | 79.3338ms | 12.6050 Ops/s | 12.4252 Ops/s | $\color{#35bf28}+1.45\\%$ | | test_async_pixels | 0.1539s | 66.5512ms | 15.0260 Ops/s | 15.2601 Ops/s | $\color{#d91a1a}-1.53\\%$ | | test_simple | 0.8886s | 0.8321s | 1.2018 Ops/s | 1.1810 Ops/s | $\color{#35bf28}+1.76\\%$ | | test_transformed | 1.1365s | 1.0802s | 0.9257 Ops/s | 0.9343 Ops/s | $\color{#d91a1a}-0.91\\%$ | | test_serial | 2.5783s | 2.5378s | 0.3940 Ops/s | 0.3855 Ops/s | $\color{#35bf28}+2.23\\%$ | | test_parallel | 2.4771s | 2.4354s | 0.4106 Ops/s | 0.4184 Ops/s | $\color{#d91a1a}-1.86\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.2305ms | 32.2891μs | 30.9702 KOps/s | 31.7186 KOps/s | $\color{#d91a1a}-2.36\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 0.1523ms | 18.6207μs | 53.7036 KOps/s | 53.4509 KOps/s | $\color{#35bf28}+0.47\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 0.1398ms | 18.5857μs | 53.8049 KOps/s | 55.1575 KOps/s | $\color{#d91a1a}-2.45\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 41.8910μs | 10.6788μs | 93.6434 KOps/s | 94.5134 KOps/s | $\color{#d91a1a}-0.92\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 60.5910μs | 34.0581μs | 29.3616 KOps/s | 30.5720 KOps/s | $\color{#d91a1a}-3.96\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 46.2110μs | 20.3081μs | 49.2415 KOps/s | 50.4283 KOps/s | $\color{#d91a1a}-2.35\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 0.2141ms | 20.0746μs | 49.8142 KOps/s | 50.5338 KOps/s | $\color{#d91a1a}-1.42\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 0.2149ms | 12.4630μs | 80.2375 KOps/s | 81.3574 KOps/s | $\color{#d91a1a}-1.38\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 67.3410μs | 35.9226μs | 27.8376 KOps/s | 28.6326 KOps/s | $\color{#d91a1a}-2.78\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 0.2221ms | 21.9800μs | 45.4960 KOps/s | 46.7881 KOps/s | $\color{#d91a1a}-2.76\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 0.2146ms | 20.0974μs | 49.7576 KOps/s | 50.6391 KOps/s | $\color{#d91a1a}-1.74\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 0.1084ms | 12.5661μs | 79.5792 KOps/s | 81.3098 KOps/s | $\color{#d91a1a}-2.13\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 86.9720μs | 37.4393μs | 26.7099 KOps/s | 26.9074 KOps/s | $\color{#d91a1a}-0.73\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 52.3910μs | 23.8525μs | 41.9244 KOps/s | 41.7069 KOps/s | $\color{#35bf28}+0.52\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 55.6310μs | 21.8273μs | 45.8142 KOps/s | 46.8503 KOps/s | $\color{#d91a1a}-2.21\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 0.1461ms | 14.2464μs | 70.1934 KOps/s | 70.7368 KOps/s | $\color{#d91a1a}-0.77\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 90.7020μs | 36.2138μs | 27.6138 KOps/s | 27.9335 KOps/s | $\color{#d91a1a}-1.14\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 55.4110μs | 22.2679μs | 44.9077 KOps/s | 44.8776 KOps/s | $\color{#35bf28}+0.07\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 49.1710μs | 23.8602μs | 41.9108 KOps/s | 42.2896 KOps/s | $\color{#d91a1a}-0.90\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 0.1517ms | 14.3244μs | 69.8109 KOps/s | 71.0407 KOps/s | $\color{#d91a1a}-1.73\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 71.7020μs | 37.2127μs | 26.8726 KOps/s | 27.2732 KOps/s | $\color{#d91a1a}-1.47\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 59.7210μs | 23.8468μs | 41.9343 KOps/s | 42.4037 KOps/s | $\color{#d91a1a}-1.11\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 56.3820μs | 25.2417μs | 39.6170 KOps/s | 40.5222 KOps/s | $\color{#d91a1a}-2.23\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 42.7610μs | 15.9527μs | 62.6854 KOps/s | 63.3811 KOps/s | $\color{#d91a1a}-1.10\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 70.5510μs | 39.0072μs | 25.6363 KOps/s | 26.3240 KOps/s | $\color{#d91a1a}-2.61\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 62.7820μs | 25.2422μs | 39.6163 KOps/s | 39.9991 KOps/s | $\color{#d91a1a}-0.96\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 52.6810μs | 25.3010μs | 39.5241 KOps/s | 40.2559 KOps/s | $\color{#d91a1a}-1.82\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 43.1210μs | 16.0338μs | 62.3682 KOps/s | 63.8437 KOps/s | $\color{#d91a1a}-2.31\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 68.0110μs | 41.4909μs | 24.1017 KOps/s | 24.2152 KOps/s | $\color{#d91a1a}-0.47\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 0.1637ms | 27.2512μs | 36.6956 KOps/s | 36.6610 KOps/s | $\color{#35bf28}+0.09\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 57.1410μs | 26.5006μs | 37.7350 KOps/s | 38.1336 KOps/s | $\color{#d91a1a}-1.05\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 87.8420μs | 17.6090μs | 56.7890 KOps/s | 57.7007 KOps/s | $\color{#d91a1a}-1.58\\%$ | | test_values[generalized_advantage_estimate-True-True] | 28.7325ms | 27.7257ms | 36.0676 Ops/s | 34.2221 Ops/s | $\textbf{\color{#35bf28}+5.39\\%}$ | | test_values[vec_generalized_advantage_estimate-True-True] | 98.8086ms | 2.9136ms | 343.2127 Ops/s | 342.0711 Ops/s | $\color{#35bf28}+0.33\\%$ | | test_values[td0_return_estimate-False-False] | 0.1106ms | 70.0563μs | 14.2742 KOps/s | 13.8970 KOps/s | $\color{#35bf28}+2.71\\%$ | | test_values[td1_return_estimate-False-False] | 64.2604ms | 60.7477ms | 16.4615 Ops/s | 15.8243 Ops/s | $\color{#35bf28}+4.03\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.4007ms | 1.1368ms | 879.6514 Ops/s | 872.6100 Ops/s | $\color{#35bf28}+0.81\\%$ | | test_values[td_lambda_return_estimate-True-False] | 0.1018s | 99.3754ms | 10.0629 Ops/s | 10.1380 Ops/s | $\color{#d91a1a}-0.74\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.3926ms | 1.1303ms | 884.6847 Ops/s | 876.1722 Ops/s | $\color{#35bf28}+0.97\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 28.0535ms | 27.5798ms | 36.2584 Ops/s | 36.3147 Ops/s | $\color{#d91a1a}-0.16\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 1.0105ms | 0.7771ms | 1.2869 KOps/s | 1.2791 KOps/s | $\color{#35bf28}+0.61\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.9440ms | 0.7258ms | 1.3779 KOps/s | 1.3679 KOps/s | $\color{#35bf28}+0.73\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.7129ms | 1.5148ms | 660.1418 Ops/s | 657.3908 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.8952ms | 0.7374ms | 1.3560 KOps/s | 1.3216 KOps/s | $\color{#35bf28}+2.61\\%$ | | test_dqn_speed | 2.1959ms | 1.5212ms | 657.3743 Ops/s | 682.8644 Ops/s | $\color{#d91a1a}-3.73\\%$ | | test_ddpg_speed | 3.4450ms | 3.0293ms | 330.1132 Ops/s | 332.2942 Ops/s | $\color{#d91a1a}-0.66\\%$ | | test_sac_speed | 9.3670ms | 8.7300ms | 114.5470 Ops/s | 117.2552 Ops/s | $\color{#d91a1a}-2.31\\%$ | | test_redq_speed | 0.1015s | 11.8445ms | 84.4271 Ops/s | 90.4854 Ops/s | $\textbf{\color{#d91a1a}-6.70\\%}$ | | test_redq_deprec_speed | 12.1894ms | 11.6948ms | 85.5083 Ops/s | 85.6446 Ops/s | $\color{#d91a1a}-0.16\\%$ | | test_td3_speed | 9.4881ms | 8.6695ms | 115.3474 Ops/s | 117.2001 Ops/s | $\color{#d91a1a}-1.58\\%$ | | test_cql_speed | 29.4741ms | 26.7245ms | 37.4188 Ops/s | 37.9916 Ops/s | $\color{#d91a1a}-1.51\\%$ | | test_a2c_speed | 6.2021ms | 5.8224ms | 171.7504 Ops/s | 177.6807 Ops/s | $\color{#d91a1a}-3.34\\%$ | | test_ppo_speed | 6.4449ms | 6.0939ms | 164.0980 Ops/s | 167.1344 Ops/s | $\color{#d91a1a}-1.82\\%$ | | test_reinforce_speed | 4.9882ms | 4.6271ms | 216.1201 Ops/s | 217.2341 Ops/s | $\color{#d91a1a}-0.51\\%$ | | test_iql_speed | 20.6450ms | 20.1430ms | 49.6451 Ops/s | 51.0678 Ops/s | $\color{#d91a1a}-2.79\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.0474ms | 4.6305ms | 215.9590 Ops/s | 212.7323 Ops/s | $\color{#35bf28}+1.52\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.1021s | 0.3894ms | 2.5680 KOps/s | 3.1513 KOps/s | $\textbf{\color{#d91a1a}-18.51\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.4858ms | 0.2984ms | 3.3507 KOps/s | 3.3701 KOps/s | $\color{#d91a1a}-0.57\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.0404ms | 4.6198ms | 216.4581 Ops/s | 214.2293 Ops/s | $\color{#35bf28}+1.04\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.5078ms | 0.3227ms | 3.0991 KOps/s | 3.1736 KOps/s | $\color{#d91a1a}-2.35\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.5089ms | 0.3058ms | 3.2705 KOps/s | 3.3838 KOps/s | $\color{#d91a1a}-3.35\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.1124ms | 1.7015ms | 587.7202 Ops/s | 617.6923 Ops/s | $\color{#d91a1a}-4.85\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 1.9322ms | 1.5781ms | 633.6631 Ops/s | 658.3072 Ops/s | $\color{#d91a1a}-3.74\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.1047ms | 4.7939ms | 208.5992 Ops/s | 207.7116 Ops/s | $\color{#35bf28}+0.43\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9463ms | 0.5643ms | 1.7720 KOps/s | 1.8005 KOps/s | $\color{#d91a1a}-1.58\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9689ms | 0.5481ms | 1.8243 KOps/s | 2.0667 KOps/s | $\textbf{\color{#d91a1a}-11.73\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.9558ms | 4.6590ms | 214.6387 Ops/s | 214.5014 Ops/s | $\color{#35bf28}+0.06\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.2244ms | 0.4272ms | 2.3406 KOps/s | 3.0560 KOps/s | $\textbf{\color{#d91a1a}-23.41\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.5265ms | 0.2942ms | 3.3990 KOps/s | 3.3521 KOps/s | $\color{#35bf28}+1.40\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.9844ms | 4.6259ms | 216.1726 Ops/s | 214.7442 Ops/s | $\color{#35bf28}+0.67\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7206ms | 0.3187ms | 3.1373 KOps/s | 3.0814 KOps/s | $\color{#35bf28}+1.82\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 9.4858ms | 0.3037ms | 3.2930 KOps/s | 3.3610 KOps/s | $\color{#d91a1a}-2.02\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0973ms | 4.7994ms | 208.3611 Ops/s | 207.2956 Ops/s | $\color{#35bf28}+0.51\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.7162ms | 0.4728ms | 2.1151 KOps/s | 1.7964 KOps/s | $\textbf{\color{#35bf28}+17.74\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8227ms | 0.4473ms | 2.2355 KOps/s | 1.8286 KOps/s | $\textbf{\color{#35bf28}+22.25\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1502s | 8.2560ms | 121.1245 Ops/s | 123.3842 Ops/s | $\color{#d91a1a}-1.83\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 22.0504ms | 16.9874ms | 58.8673 Ops/s | 57.7941 Ops/s | $\color{#35bf28}+1.86\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.1420ms | 0.9410ms | 1.0627 KOps/s | 1.0291 KOps/s | $\color{#35bf28}+3.27\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1220s | 10.0640ms | 99.3642 Ops/s | 126.7061 Ops/s | $\textbf{\color{#d91a1a}-21.58\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 22.7527ms | 17.4623ms | 57.2661 Ops/s | 57.2842 Ops/s | $\color{#d91a1a}-0.03\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.0293ms | 0.9590ms | 1.0428 KOps/s | 1.0019 KOps/s | $\color{#35bf28}+4.08\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1215s | 7.8085ms | 128.0656 Ops/s | 96.4375 Ops/s | $\textbf{\color{#35bf28}+32.80\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 22.3896ms | 17.1844ms | 58.1923 Ops/s | 57.3999 Ops/s | $\color{#35bf28}+1.38\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 8.3926ms | 1.2336ms | 810.6468 Ops/s | 899.2847 Ops/s | $\textbf{\color{#d91a1a}-9.86\\%}$ |