search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.24k stars 296 forks source link

[Quality] Fix low/high in SOTA implementations #2266

Closed vmoens closed 3 months ago

pytorch-bot[bot] commented 3 months ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2266

Note: Links to docs will display an error until the docs builds have been completed.

:x: 2 New Failures, 15 Unrelated Failures

As of commit 69bc78c8e7e1132699c5ffff13a2309249bcac7b with merge base 79fa8bfc908b9aff6d858b4c801e633841bc5f68 (image):

NEW FAILURES - The following jobs have failed:

* [Generate documentation / build-docs (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985820769) ([gh](https://github.com/pytorch/rl/actions/runs/9775436433/job/26985820769)) `No files were found with the provided path: /home/ec2-user/actions-runner/_work/_temp/artifacts/. No artifacts will be uploaded.` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985818076) ([gh](https://github.com/pytorch/rl/actions/runs/9775436428/job/26985818076)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

* [Examples Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985819753) ([gh](https://github.com/pytorch/rl/actions/runs/9775436437/job/26985819753)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985818781) ([gh](https://github.com/pytorch/rl/actions/runs/9775436419/job/26985818781)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985828807) ([gh](https://github.com/pytorch/rl/actions/runs/9775436435/job/26985828807)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985829155) ([gh](https://github.com/pytorch/rl/actions/runs/9775436435/job/26985829155)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Lint / c-source / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985818060) ([gh](https://github.com/pytorch/rl/actions/runs/9775436425/job/26985818060)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Lint / python-source-and-configs / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985818760) ([gh](https://github.com/pytorch/rl/actions/runs/9775436425/job/26985818760)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985819774) ([gh](https://github.com/pytorch/rl/actions/runs/9775436452/job/26985819774)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.10) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985822142) ([gh](https://github.com/pytorch/rl/actions/runs/9775436441/job/26985822142)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.11) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985820764) ([gh](https://github.com/pytorch/rl/actions/runs/9775436441/job/26985820764)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.8) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985821727) ([gh](https://github.com/pytorch/rl/actions/runs/9775436441/job/26985821727)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.9) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985823262) ([gh](https://github.com/pytorch/rl/actions/runs/9775436441/job/26985823262)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-gpu (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985824187) ([gh](https://github.com/pytorch/rl/actions/runs/9775436441/job/26985824187)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985824594) ([gh](https://github.com/pytorch/rl/actions/runs/9775436441/job/26985824594)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985825348) ([gh](https://github.com/pytorch/rl/actions/runs/9775436441/job/26985825348)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2266#26985825682) ([gh](https://github.com/pytorch/rl/actions/runs/9775436441/job/26985825682)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 3 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1060s | 57.9263ms | 17.2633 Ops/s | 17.3684 Ops/s | $\color{#d91a1a}-0.61\\%$ | | test_sync | 38.2455ms | 31.5407ms | 31.7050 Ops/s | 32.0785 Ops/s | $\color{#d91a1a}-1.16\\%$ | | test_async | 55.0582ms | 28.3717ms | 35.2464 Ops/s | 32.7458 Ops/s | $\textbf{\color{#35bf28}+7.64\\%}$ | | test_simple | 0.3828s | 0.3811s | 2.6240 Ops/s | 2.5803 Ops/s | $\color{#35bf28}+1.69\\%$ | | test_transformed | 0.5444s | 0.5404s | 1.8505 Ops/s | 1.7691 Ops/s | $\color{#35bf28}+4.60\\%$ | | test_serial | 1.3430s | 1.2923s | 0.7738 Ops/s | 0.7685 Ops/s | $\color{#35bf28}+0.70\\%$ | | test_parallel | 1.1859s | 1.1122s | 0.8991 Ops/s | 0.9026 Ops/s | $\color{#d91a1a}-0.39\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1584ms | 22.5401μs | 44.3655 KOps/s | 44.2439 KOps/s | $\color{#35bf28}+0.27\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 41.1870μs | 13.4029μs | 74.6110 KOps/s | 74.8742 KOps/s | $\color{#d91a1a}-0.35\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 38.1710μs | 13.0818μs | 76.4422 KOps/s | 77.1639 KOps/s | $\color{#d91a1a}-0.94\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 26.8700μs | 7.7026μs | 129.8268 KOps/s | 129.7279 KOps/s | $\color{#35bf28}+0.08\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 86.8400μs | 24.1528μs | 41.4031 KOps/s | 41.4811 KOps/s | $\color{#d91a1a}-0.19\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 56.5750μs | 14.6421μs | 68.2962 KOps/s | 68.2761 KOps/s | $\color{#35bf28}+0.03\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 0.1418ms | 14.5576μs | 68.6925 KOps/s | 69.1795 KOps/s | $\color{#d91a1a}-0.70\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 38.8830μs | 9.0264μs | 110.7855 KOps/s | 111.9998 KOps/s | $\color{#d91a1a}-1.08\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 0.1395ms | 26.4881μs | 37.7527 KOps/s | 39.7027 KOps/s | $\color{#d91a1a}-4.91\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 42.1090μs | 15.9154μs | 62.8321 KOps/s | 62.6103 KOps/s | $\color{#35bf28}+0.35\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 50.4340μs | 14.4159μs | 69.3680 KOps/s | 70.3992 KOps/s | $\color{#d91a1a}-1.46\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 40.9870μs | 9.0449μs | 110.5593 KOps/s | 111.4919 KOps/s | $\color{#d91a1a}-0.84\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 0.1194ms | 26.7213μs | 37.4233 KOps/s | 37.5602 KOps/s | $\color{#d91a1a}-0.36\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 44.3330μs | 16.9356μs | 59.0471 KOps/s | 58.2271 KOps/s | $\color{#35bf28}+1.41\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 42.1780μs | 15.5750μs | 64.2055 KOps/s | 64.1707 KOps/s | $\color{#35bf28}+0.05\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 38.2510μs | 10.1553μs | 98.4706 KOps/s | 98.3273 KOps/s | $\color{#35bf28}+0.15\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 0.1344ms | 25.7214μs | 38.8781 KOps/s | 39.1606 KOps/s | $\color{#d91a1a}-0.72\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 47.8990μs | 15.9266μs | 62.7881 KOps/s | 62.5866 KOps/s | $\color{#35bf28}+0.32\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 39.6630μs | 16.6584μs | 60.0297 KOps/s | 59.3461 KOps/s | $\color{#35bf28}+1.15\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 51.3760μs | 10.1987μs | 98.0512 KOps/s | 97.2108 KOps/s | $\color{#35bf28}+0.86\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 70.3520μs | 26.7681μs | 37.3578 KOps/s | 37.2602 KOps/s | $\color{#35bf28}+0.26\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 39.4630μs | 17.1619μs | 58.2686 KOps/s | 58.2628 KOps/s | $+0.01\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 96.1900μs | 17.9751μs | 55.6327 KOps/s | 55.0501 KOps/s | $\color{#35bf28}+1.06\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 52.9880μs | 11.4164μs | 87.5934 KOps/s | 87.2169 KOps/s | $\color{#35bf28}+0.43\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 77.8050μs | 28.1849μs | 35.4800 KOps/s | 35.4495 KOps/s | $\color{#35bf28}+0.09\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 57.0760μs | 18.5447μs | 53.9239 KOps/s | 53.7456 KOps/s | $\color{#35bf28}+0.33\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 50.2530μs | 18.1337μs | 55.1460 KOps/s | 55.5665 KOps/s | $\color{#d91a1a}-0.76\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 34.2930μs | 11.3381μs | 88.1983 KOps/s | 86.3080 KOps/s | $\color{#35bf28}+2.19\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 64.0100μs | 29.8004μs | 33.5566 KOps/s | 33.6672 KOps/s | $\color{#d91a1a}-0.33\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 58.7400μs | 19.4395μs | 51.4415 KOps/s | 50.2046 KOps/s | $\color{#35bf28}+2.46\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 47.2180μs | 19.0416μs | 52.5166 KOps/s | 52.0999 KOps/s | $\color{#35bf28}+0.80\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 47.8090μs | 12.4636μs | 80.2336 KOps/s | 79.1503 KOps/s | $\color{#35bf28}+1.37\\%$ | | test_values[generalized_advantage_estimate-True-True] | 12.8231ms | 9.5752ms | 104.4368 Ops/s | 102.5045 Ops/s | $\color{#35bf28}+1.88\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 37.8465ms | 35.4262ms | 28.2277 Ops/s | 27.8154 Ops/s | $\color{#35bf28}+1.48\\%$ | | test_values[td0_return_estimate-False-False] | 0.2316ms | 0.1717ms | 5.8244 KOps/s | 5.5310 KOps/s | $\textbf{\color{#35bf28}+5.30\\%}$ | | test_values[td1_return_estimate-False-False] | 24.8588ms | 23.6524ms | 42.2791 Ops/s | 41.1889 Ops/s | $\color{#35bf28}+2.65\\%$ | | test_values[vec_td1_return_estimate-False-False] | 37.0527ms | 35.4729ms | 28.1905 Ops/s | 27.6241 Ops/s | $\color{#35bf28}+2.05\\%$ | | test_values[td_lambda_return_estimate-True-False] | 36.1851ms | 33.8418ms | 29.5492 Ops/s | 28.5114 Ops/s | $\color{#35bf28}+3.64\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 36.6349ms | 35.4076ms | 28.2426 Ops/s | 27.7065 Ops/s | $\color{#35bf28}+1.93\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 11.5302ms | 8.3275ms | 120.0839 Ops/s | 116.6752 Ops/s | $\color{#35bf28}+2.92\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.3288ms | 1.9920ms | 502.0148 Ops/s | 492.8632 Ops/s | $\color{#35bf28}+1.86\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4371ms | 0.3567ms | 2.8038 KOps/s | 2.7454 KOps/s | $\color{#35bf28}+2.13\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 46.0418ms | 44.9796ms | 22.2323 Ops/s | 22.8017 Ops/s | $\color{#d91a1a}-2.50\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.5602ms | 3.0519ms | 327.6684 Ops/s | 329.2988 Ops/s | $\color{#d91a1a}-0.50\\%$ | | test_dqn_speed | 1.7879ms | 1.3509ms | 740.2226 Ops/s | 739.7488 Ops/s | $\color{#35bf28}+0.06\\%$ | | test_ddpg_speed | 3.6211ms | 2.9036ms | 344.3961 Ops/s | 350.6380 Ops/s | $\color{#d91a1a}-1.78\\%$ | | test_sac_speed | 8.9649ms | 8.5337ms | 117.1830 Ops/s | 116.4628 Ops/s | $\color{#35bf28}+0.62\\%$ | | test_redq_speed | 18.7012ms | 13.6959ms | 73.0147 Ops/s | 65.0030 Ops/s | $\textbf{\color{#35bf28}+12.33\\%}$ | | test_redq_deprec_speed | 15.5349ms | 13.8950ms | 71.9684 Ops/s | 71.3037 Ops/s | $\color{#35bf28}+0.93\\%$ | | test_td3_speed | 18.3474ms | 8.6452ms | 115.6717 Ops/s | 116.5557 Ops/s | $\color{#d91a1a}-0.76\\%$ | | test_cql_speed | 39.0918ms | 37.0558ms | 26.9863 Ops/s | 26.9247 Ops/s | $\color{#35bf28}+0.23\\%$ | | test_a2c_speed | 8.2765ms | 7.5804ms | 131.9188 Ops/s | 130.3190 Ops/s | $\color{#35bf28}+1.23\\%$ | | test_ppo_speed | 9.2731ms | 7.9743ms | 125.4026 Ops/s | 125.5146 Ops/s | $\color{#d91a1a}-0.09\\%$ | | test_reinforce_speed | 7.4269ms | 6.7399ms | 148.3694 Ops/s | 145.6503 Ops/s | $\color{#35bf28}+1.87\\%$ | | test_iql_speed | 33.7068ms | 33.0634ms | 30.2449 Ops/s | 29.9722 Ops/s | $\color{#35bf28}+0.91\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.9753ms | 3.6434ms | 274.4712 Ops/s | 265.6351 Ops/s | $\color{#35bf28}+3.33\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.8005ms | 0.5020ms | 1.9921 KOps/s | 1.9588 KOps/s | $\color{#35bf28}+1.70\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.5976ms | 0.4861ms | 2.0573 KOps/s | 2.0661 KOps/s | $\color{#d91a1a}-0.43\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.7184ms | 3.6922ms | 270.8428 Ops/s | 269.1497 Ops/s | $\color{#35bf28}+0.63\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8628ms | 0.5231ms | 1.9116 KOps/s | 2.0153 KOps/s | $\textbf{\color{#d91a1a}-5.15\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6829ms | 0.4744ms | 2.1079 KOps/s | 2.0737 KOps/s | $\color{#35bf28}+1.65\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.1635ms | 1.7322ms | 577.2981 Ops/s | 565.0912 Ops/s | $\color{#35bf28}+2.16\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2574ms | 1.6787ms | 595.6992 Ops/s | 595.9123 Ops/s | $\color{#d91a1a}-0.04\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.2041ms | 3.7267ms | 268.3314 Ops/s | 266.2594 Ops/s | $\color{#35bf28}+0.78\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.1123s | 0.7306ms | 1.3687 KOps/s | 1.5657 KOps/s | $\textbf{\color{#d91a1a}-12.58\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9730ms | 0.6133ms | 1.6306 KOps/s | 1.6170 KOps/s | $\color{#35bf28}+0.84\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.9871ms | 3.5361ms | 282.7978 Ops/s | 275.2691 Ops/s | $\color{#35bf28}+2.74\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6120ms | 0.5026ms | 1.9896 KOps/s | 1.9708 KOps/s | $\color{#35bf28}+0.95\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.6809ms | 0.4864ms | 2.0559 KOps/s | 2.0841 KOps/s | $\color{#d91a1a}-1.35\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.5558ms | 3.5378ms | 282.6628 Ops/s | 278.8240 Ops/s | $\color{#35bf28}+1.38\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6430ms | 0.4964ms | 2.0145 KOps/s | 2.0003 KOps/s | $\color{#35bf28}+0.71\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6279ms | 0.4785ms | 2.0899 KOps/s | 2.0913 KOps/s | $\color{#d91a1a}-0.07\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.4856ms | 3.8156ms | 262.0836 Ops/s | 261.3492 Ops/s | $\color{#35bf28}+0.28\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.2254ms | 0.6380ms | 1.5673 KOps/s | 1.5487 KOps/s | $\color{#35bf28}+1.20\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7873ms | 0.6102ms | 1.6389 KOps/s | 1.6072 KOps/s | $\color{#35bf28}+1.97\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1326s | 6.3051ms | 158.6021 Ops/s | 119.5084 Ops/s | $\textbf{\color{#35bf28}+32.71\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 15.3807ms | 12.6658ms | 78.9527 Ops/s | 77.9557 Ops/s | $\color{#35bf28}+1.28\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 4.3544ms | 1.1644ms | 858.8367 Ops/s | 919.7869 Ops/s | $\textbf{\color{#d91a1a}-6.63\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1126s | 8.0010ms | 124.9842 Ops/s | 163.9941 Ops/s | $\textbf{\color{#d91a1a}-23.79\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 15.2117ms | 12.6899ms | 78.8029 Ops/s | 76.4831 Ops/s | $\color{#35bf28}+3.03\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.5509ms | 1.0666ms | 937.5492 Ops/s | 921.2021 Ops/s | $\color{#35bf28}+1.77\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1126s | 6.1444ms | 162.7498 Ops/s | 152.4614 Ops/s | $\textbf{\color{#35bf28}+6.75\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 16.2135ms | 12.8537ms | 77.7988 Ops/s | 77.4300 Ops/s | $\color{#35bf28}+0.48\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.9145ms | 1.2183ms | 820.8459 Ops/s | 772.8600 Ops/s | $\textbf{\color{#35bf28}+6.21\\%}$ |
github-actions[bot] commented 3 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1636s | 0.1107s | 9.0333 Ops/s | 9.5137 Ops/s | $\textbf{\color{#d91a1a}-5.05\\%}$ | | test_sync | 94.6730ms | 92.2509ms | 10.8400 Ops/s | 11.0643 Ops/s | $\color{#d91a1a}-2.03\\%$ | | test_async | 0.1876s | 93.3271ms | 10.7150 Ops/s | 11.0700 Ops/s | $\color{#d91a1a}-3.21\\%$ | | test_single_pixels | 0.1155s | 0.1154s | 8.6677 Ops/s | 8.6835 Ops/s | $\color{#d91a1a}-0.18\\%$ | | test_sync_pixels | 70.7128ms | 69.4123ms | 14.4067 Ops/s | 14.6318 Ops/s | $\color{#d91a1a}-1.54\\%$ | | test_async_pixels | 0.1253s | 62.8604ms | 15.9083 Ops/s | 17.0982 Ops/s | $\textbf{\color{#d91a1a}-6.96\\%}$ | | test_simple | 0.8148s | 0.7566s | 1.3216 Ops/s | 1.3090 Ops/s | $\color{#35bf28}+0.96\\%$ | | test_transformed | 1.0487s | 0.9872s | 1.0130 Ops/s | 1.0123 Ops/s | $\color{#35bf28}+0.07\\%$ | | test_serial | 2.2596s | 2.2026s | 0.4540 Ops/s | 0.4533 Ops/s | $\color{#35bf28}+0.16\\%$ | | test_parallel | 1.9007s | 1.8616s | 0.5372 Ops/s | 0.5311 Ops/s | $\color{#35bf28}+1.14\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1623ms | 34.5976μs | 28.9037 KOps/s | 29.4969 KOps/s | $\color{#d91a1a}-2.01\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 41.3410μs | 19.8390μs | 50.4057 KOps/s | 51.3339 KOps/s | $\color{#d91a1a}-1.81\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 43.8600μs | 19.4719μs | 51.3561 KOps/s | 52.5529 KOps/s | $\color{#d91a1a}-2.28\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 28.6100μs | 11.4665μs | 87.2106 KOps/s | 89.9214 KOps/s | $\color{#d91a1a}-3.01\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 60.8010μs | 36.7477μs | 27.2126 KOps/s | 28.2269 KOps/s | $\color{#d91a1a}-3.59\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 66.6710μs | 21.8051μs | 45.8609 KOps/s | 47.0066 KOps/s | $\color{#d91a1a}-2.44\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 50.0410μs | 21.6195μs | 46.2545 KOps/s | 47.9736 KOps/s | $\color{#d91a1a}-3.58\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 33.4810μs | 13.3773μs | 74.7536 KOps/s | 76.3659 KOps/s | $\color{#d91a1a}-2.11\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 79.7820μs | 38.4914μs | 25.9798 KOps/s | 26.5526 KOps/s | $\color{#d91a1a}-2.16\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 41.0410μs | 23.6806μs | 42.2286 KOps/s | 42.6890 KOps/s | $\color{#d91a1a}-1.08\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 40.7810μs | 21.0289μs | 47.5537 KOps/s | 47.5684 KOps/s | $\color{#d91a1a}-0.03\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 52.6810μs | 13.3686μs | 74.8020 KOps/s | 76.4931 KOps/s | $\color{#d91a1a}-2.21\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 66.9210μs | 40.4724μs | 24.7082 KOps/s | 25.4315 KOps/s | $\color{#d91a1a}-2.84\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 55.9110μs | 25.5191μs | 39.1863 KOps/s | 39.4748 KOps/s | $\color{#d91a1a}-0.73\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 42.0300μs | 23.0476μs | 43.3884 KOps/s | 44.2644 KOps/s | $\color{#d91a1a}-1.98\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 30.3210μs | 15.2484μs | 65.5806 KOps/s | 67.8494 KOps/s | $\color{#d91a1a}-3.34\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 62.6110μs | 38.1176μs | 26.2346 KOps/s | 26.7160 KOps/s | $\color{#d91a1a}-1.80\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 59.0010μs | 23.8035μs | 42.0106 KOps/s | 43.0608 KOps/s | $\color{#d91a1a}-2.44\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 66.8310μs | 25.8610μs | 38.6682 KOps/s | 39.5017 KOps/s | $\color{#d91a1a}-2.11\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 31.3000μs | 14.9446μs | 66.9140 KOps/s | 67.0684 KOps/s | $\color{#d91a1a}-0.23\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 63.0310μs | 40.4741μs | 24.7072 KOps/s | 25.5626 KOps/s | $\color{#d91a1a}-3.35\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 68.5910μs | 25.4602μs | 39.2770 KOps/s | 39.5135 KOps/s | $\color{#d91a1a}-0.60\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 57.1510μs | 28.1613μs | 35.5097 KOps/s | 37.7278 KOps/s | $\textbf{\color{#d91a1a}-5.88\\%}$ | | test_step_mdp_speed[False-True-False-False-False] | 43.4410μs | 16.9865μs | 58.8704 KOps/s | 58.9608 KOps/s | $\color{#d91a1a}-0.15\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 62.6610μs | 42.2112μs | 23.6904 KOps/s | 24.4005 KOps/s | $\color{#d91a1a}-2.91\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 48.9410μs | 27.5654μs | 36.2774 KOps/s | 37.2029 KOps/s | $\color{#d91a1a}-2.49\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 47.0710μs | 27.8902μs | 35.8549 KOps/s | 37.4365 KOps/s | $\color{#d91a1a}-4.22\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 38.5610μs | 16.8687μs | 59.2813 KOps/s | 60.4520 KOps/s | $\color{#d91a1a}-1.94\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 59.4200μs | 44.5601μs | 22.4416 KOps/s | 23.3125 KOps/s | $\color{#d91a1a}-3.74\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 51.5810μs | 29.6310μs | 33.7484 KOps/s | 34.5904 KOps/s | $\color{#d91a1a}-2.43\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 61.1810μs | 29.4041μs | 34.0089 KOps/s | 35.6595 KOps/s | $\color{#d91a1a}-4.63\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 39.5720μs | 19.0257μs | 52.5605 KOps/s | 54.4787 KOps/s | $\color{#d91a1a}-3.52\\%$ | | test_values[generalized_advantage_estimate-True-True] | 24.8788ms | 24.4266ms | 40.9389 Ops/s | 39.9912 Ops/s | $\color{#35bf28}+2.37\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 88.3704ms | 2.6703ms | 374.4894 Ops/s | 369.0527 Ops/s | $\color{#35bf28}+1.47\\%$ | | test_values[td0_return_estimate-False-False] | 93.7910μs | 66.1864μs | 15.1088 KOps/s | 14.4063 KOps/s | $\color{#35bf28}+4.88\\%$ | | test_values[td1_return_estimate-False-False] | 54.9340ms | 54.4616ms | 18.3616 Ops/s | 17.4780 Ops/s | $\textbf{\color{#35bf28}+5.06\\%}$ | | test_values[vec_td1_return_estimate-False-False] | 1.3383ms | 1.0897ms | 917.6640 Ops/s | 907.9996 Ops/s | $\color{#35bf28}+1.06\\%$ | | test_values[td_lambda_return_estimate-True-False] | 89.0490ms | 86.9677ms | 11.4985 Ops/s | 10.9694 Ops/s | $\color{#35bf28}+4.82\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.4351ms | 1.0888ms | 918.4269 Ops/s | 913.0890 Ops/s | $\color{#35bf28}+0.58\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 24.9372ms | 24.6345ms | 40.5935 Ops/s | 40.0111 Ops/s | $\color{#35bf28}+1.46\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9538ms | 0.7227ms | 1.3837 KOps/s | 1.3610 KOps/s | $\color{#35bf28}+1.67\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7585ms | 0.6759ms | 1.4796 KOps/s | 1.4666 KOps/s | $\color{#35bf28}+0.89\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.6020ms | 1.4715ms | 679.5627 Ops/s | 671.9093 Ops/s | $\color{#35bf28}+1.14\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.8333ms | 0.6909ms | 1.4475 KOps/s | 1.4387 KOps/s | $\color{#35bf28}+0.61\\%$ | | test_dqn_speed | 7.4719ms | 1.4895ms | 671.3792 Ops/s | 686.7592 Ops/s | $\color{#d91a1a}-2.24\\%$ | | test_ddpg_speed | 3.2474ms | 3.0016ms | 333.1595 Ops/s | 333.2210 Ops/s | $\color{#d91a1a}-0.02\\%$ | | test_sac_speed | 8.9577ms | 8.5653ms | 116.7506 Ops/s | 115.9797 Ops/s | $\color{#35bf28}+0.66\\%$ | | test_redq_speed | 0.1045s | 11.9761ms | 83.4997 Ops/s | 90.8191 Ops/s | $\textbf{\color{#d91a1a}-8.06\\%}$ | | test_redq_deprec_speed | 12.4548ms | 11.7406ms | 85.1745 Ops/s | 76.7461 Ops/s | $\textbf{\color{#35bf28}+10.98\\%}$ | | test_td3_speed | 8.6071ms | 8.4755ms | 117.9876 Ops/s | 117.9117 Ops/s | $\color{#35bf28}+0.06\\%$ | | test_cql_speed | 26.6786ms | 26.0791ms | 38.3449 Ops/s | 37.6730 Ops/s | $\color{#35bf28}+1.78\\%$ | | test_a2c_speed | 5.9551ms | 5.7843ms | 172.8821 Ops/s | 170.5620 Ops/s | $\color{#35bf28}+1.36\\%$ | | test_ppo_speed | 6.3634ms | 6.1136ms | 163.5694 Ops/s | 161.7375 Ops/s | $\color{#35bf28}+1.13\\%$ | | test_reinforce_speed | 5.5796ms | 4.7706ms | 209.6153 Ops/s | 208.1190 Ops/s | $\color{#35bf28}+0.72\\%$ | | test_iql_speed | 20.2696ms | 19.8204ms | 50.4531 Ops/s | 49.4709 Ops/s | $\color{#35bf28}+1.99\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.8145ms | 4.6590ms | 214.6379 Ops/s | 216.3426 Ops/s | $\color{#d91a1a}-0.79\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.1098s | 0.6369ms | 1.5702 KOps/s | 1.8149 KOps/s | $\textbf{\color{#d91a1a}-13.48\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7439ms | 0.5281ms | 1.8938 KOps/s | 1.8939 KOps/s | $-0.01\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.9091ms | 4.6172ms | 216.5829 Ops/s | 218.4392 Ops/s | $\color{#d91a1a}-0.85\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.3981ms | 0.5408ms | 1.8492 KOps/s | 1.8463 KOps/s | $\color{#35bf28}+0.16\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7005ms | 0.5223ms | 1.9144 KOps/s | 1.9216 KOps/s | $\color{#d91a1a}-0.38\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 4.4477ms | 1.9854ms | 503.6853 Ops/s | 497.0547 Ops/s | $\color{#35bf28}+1.33\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.5945ms | 2.0588ms | 485.7155 Ops/s | 522.2105 Ops/s | $\textbf{\color{#d91a1a}-6.99\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.9370ms | 4.8128ms | 207.7773 Ops/s | 212.9622 Ops/s | $\color{#d91a1a}-2.43\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8727ms | 0.6943ms | 1.4404 KOps/s | 1.3695 KOps/s | $\textbf{\color{#35bf28}+5.17\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.6098ms | 0.6785ms | 1.4737 KOps/s | 1.3543 KOps/s | $\textbf{\color{#35bf28}+8.82\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.8359ms | 4.6543ms | 214.8569 Ops/s | 216.6637 Ops/s | $\color{#d91a1a}-0.83\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6848ms | 0.5501ms | 1.8179 KOps/s | 1.8233 KOps/s | $\color{#d91a1a}-0.30\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 4.4066ms | 0.5358ms | 1.8662 KOps/s | 1.8786 KOps/s | $\color{#d91a1a}-0.66\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.9547ms | 4.6379ms | 215.6162 Ops/s | 217.1178 Ops/s | $\color{#d91a1a}-0.69\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.1381s | 0.7405ms | 1.3504 KOps/s | 1.8480 KOps/s | $\textbf{\color{#d91a1a}-26.93\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6852ms | 0.5245ms | 1.9067 KOps/s | 1.9151 KOps/s | $\color{#d91a1a}-0.44\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0495ms | 4.8155ms | 207.6625 Ops/s | 210.9781 Ops/s | $\color{#d91a1a}-1.57\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8610ms | 0.7047ms | 1.4191 KOps/s | 1.4360 KOps/s | $\color{#d91a1a}-1.18\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.5886ms | 0.6915ms | 1.4462 KOps/s | 1.4645 KOps/s | $\color{#d91a1a}-1.25\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1269s | 7.3305ms | 136.4169 Ops/s | 137.3304 Ops/s | $\color{#d91a1a}-0.67\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 17.7883ms | 15.3996ms | 64.9366 Ops/s | 64.1747 Ops/s | $\color{#35bf28}+1.19\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.1989ms | 1.1536ms | 866.8800 Ops/s | 862.3144 Ops/s | $\color{#35bf28}+0.53\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1270s | 7.2360ms | 138.1972 Ops/s | 136.7615 Ops/s | $\color{#35bf28}+1.05\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1388s | 17.8612ms | 55.9872 Ops/s | 55.0354 Ops/s | $\color{#35bf28}+1.73\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.2393ms | 1.1834ms | 844.9999 Ops/s | 845.2233 Ops/s | $\color{#d91a1a}-0.03\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1265s | 7.4539ms | 134.1571 Ops/s | 132.8641 Ops/s | $\color{#35bf28}+0.97\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 17.8227ms | 15.5960ms | 64.1192 Ops/s | 63.6152 Ops/s | $\color{#35bf28}+0.79\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.4421ms | 1.3398ms | 746.4002 Ops/s | 749.6888 Ops/s | $\color{#d91a1a}-0.44\\%$ |