search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.19k stars 289 forks source link

[Feature] Construct loss by providing lists of modules #2311

Closed vmoens closed 1 month ago

vmoens commented 1 month ago

Stack from ghstack (oldest at bottom):

pytorch-bot[bot] commented 1 month ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2311

Note: Links to docs will display an error until the docs builds have been completed.

:x: 1 New Failure, 1 Pending, 13 Unrelated Failures

As of commit 0cb0d40febe66673fab410dd0fd023a15b2f309a with merge base 59c3374162efb9f3436ec1b8e9b2c76a03b2a7ad (image):

NEW FAILURE - The following job has failed:

* [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2311#27805173974) ([gh](https://github.com/pytorch/rl/actions/runs/10059593917/job/27805173974)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

* [Examples Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2311#27805172056) ([gh](https://github.com/pytorch/rl/actions/runs/10059593957/job/27805172056)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2311#27805172037) ([gh](https://github.com/pytorch/rl/actions/runs/10059593939/job/27805172037)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2311#27805178847) ([gh](https://github.com/pytorch/rl/actions/runs/10059593938/job/27805178847)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2311#27805179223) ([gh](https://github.com/pytorch/rl/actions/runs/10059593938/job/27805179223)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2311#27805171442) ([gh](https://github.com/pytorch/rl/actions/runs/10059593955/job/27805171442)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.10) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2311#27805211685) ([gh](https://github.com/pytorch/rl/actions/runs/10059593979/job/27805211685)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.11) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2311#27805211031) ([gh](https://github.com/pytorch/rl/actions/runs/10059593979/job/27805211031)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.8) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2311#27805210602) ([gh](https://github.com/pytorch/rl/actions/runs/10059593979/job/27805210602)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.9) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2311#27805212008) ([gh](https://github.com/pytorch/rl/actions/runs/10059593979/job/27805212008)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-gpu (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2311#27805212367) ([gh](https://github.com/pytorch/rl/actions/runs/10059593979/job/27805212367)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2311#27805212697) ([gh](https://github.com/pytorch/rl/actions/runs/10059593979/job/27805212697)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2311#27805213384) ([gh](https://github.com/pytorch/rl/actions/runs/10059593979/job/27805213384)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2311#27805213033) ([gh](https://github.com/pytorch/rl/actions/runs/10059593979/job/27805213033)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 59.8789ms | 57.7394ms | 17.3192 Ops/s | 16.9649 Ops/s | $\color{#35bf28}+2.09\\%$ | | test_sync | 38.3675ms | 32.0204ms | 31.2301 Ops/s | 31.0512 Ops/s | $\color{#35bf28}+0.58\\%$ | | test_async | 52.0907ms | 29.9091ms | 33.4346 Ops/s | 32.7333 Ops/s | $\color{#35bf28}+2.14\\%$ | | test_simple | 0.4887s | 0.4146s | 2.4119 Ops/s | 2.4278 Ops/s | $\color{#d91a1a}-0.66\\%$ | | test_transformed | 0.6428s | 0.5741s | 1.7418 Ops/s | 1.7689 Ops/s | $\color{#d91a1a}-1.53\\%$ | | test_serial | 1.3553s | 1.2763s | 0.7835 Ops/s | 0.7806 Ops/s | $\color{#35bf28}+0.37\\%$ | | test_parallel | 1.1809s | 1.1133s | 0.8982 Ops/s | 0.8813 Ops/s | $\color{#35bf28}+1.92\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.2083ms | 24.7873μs | 40.3432 KOps/s | 40.3558 KOps/s | $\color{#d91a1a}-0.03\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 42.3400μs | 14.5768μs | 68.6023 KOps/s | 69.2075 KOps/s | $\color{#d91a1a}-0.87\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 43.5020μs | 14.2643μs | 70.1052 KOps/s | 69.5988 KOps/s | $\color{#35bf28}+0.73\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 44.9140μs | 8.3704μs | 119.4687 KOps/s | 117.7444 KOps/s | $\color{#35bf28}+1.46\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 0.3847ms | 27.1628μs | 36.8151 KOps/s | 37.7326 KOps/s | $\color{#d91a1a}-2.43\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 57.0070μs | 16.0638μs | 62.2519 KOps/s | 62.7476 KOps/s | $\color{#d91a1a}-0.79\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 63.4390μs | 16.1603μs | 61.8799 KOps/s | 63.4396 KOps/s | $\color{#d91a1a}-2.46\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 35.8180μs | 9.9731μs | 100.2702 KOps/s | 101.4054 KOps/s | $\color{#d91a1a}-1.12\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 61.2950μs | 28.2405μs | 35.4101 KOps/s | 35.5046 KOps/s | $\color{#d91a1a}-0.27\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 43.7020μs | 17.9008μs | 55.8633 KOps/s | 56.7927 KOps/s | $\color{#d91a1a}-1.64\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 72.4560μs | 16.0409μs | 62.3405 KOps/s | 62.8048 KOps/s | $\color{#d91a1a}-0.74\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 0.1968ms | 10.0846μs | 99.1608 KOps/s | 100.5393 KOps/s | $\color{#d91a1a}-1.37\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 64.9820μs | 29.9612μs | 33.3765 KOps/s | 33.9834 KOps/s | $\color{#d91a1a}-1.79\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 68.6490μs | 19.4944μs | 51.2967 KOps/s | 52.5224 KOps/s | $\color{#d91a1a}-2.33\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 59.4110μs | 17.4895μs | 57.1772 KOps/s | 57.7870 KOps/s | $\color{#d91a1a}-1.06\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 51.4770μs | 11.4602μs | 87.2583 KOps/s | 87.9424 KOps/s | $\color{#d91a1a}-0.78\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 78.9580μs | 28.6321μs | 34.9259 KOps/s | 35.5059 KOps/s | $\color{#d91a1a}-1.63\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 54.7130μs | 17.9431μs | 55.7317 KOps/s | 56.7647 KOps/s | $\color{#d91a1a}-1.82\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 91.7220μs | 18.6420μs | 53.6422 KOps/s | 52.2580 KOps/s | $\color{#35bf28}+2.65\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 60.7340μs | 11.2380μs | 88.9836 KOps/s | 90.8272 KOps/s | $\color{#d91a1a}-2.03\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 66.3650μs | 29.4297μs | 33.9793 KOps/s | 34.0306 KOps/s | $\color{#d91a1a}-0.15\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 59.7830μs | 19.3552μs | 51.6657 KOps/s | 52.4393 KOps/s | $\color{#d91a1a}-1.48\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 58.5500μs | 20.3246μs | 49.2015 KOps/s | 51.4814 KOps/s | $\color{#d91a1a}-4.43\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 66.0020μs | 12.7618μs | 78.3588 KOps/s | 80.7667 KOps/s | $\color{#d91a1a}-2.98\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.7114ms | 31.0096μs | 32.2480 KOps/s | 32.1461 KOps/s | $\color{#35bf28}+0.32\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 54.0410μs | 20.7112μs | 48.2831 KOps/s | 48.8096 KOps/s | $\color{#d91a1a}-1.08\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 72.2870μs | 19.5261μs | 51.2135 KOps/s | 50.8805 KOps/s | $\color{#35bf28}+0.65\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 61.7360μs | 12.6662μs | 78.9503 KOps/s | 80.3186 KOps/s | $\color{#d91a1a}-1.70\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 83.1560μs | 32.6332μs | 30.6436 KOps/s | 30.9674 KOps/s | $\color{#d91a1a}-1.05\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 58.2590μs | 21.9979μs | 45.4588 KOps/s | 45.2328 KOps/s | $\color{#35bf28}+0.50\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 54.8020μs | 20.9639μs | 47.7011 KOps/s | 48.0024 KOps/s | $\color{#d91a1a}-0.63\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 49.9640μs | 13.9478μs | 71.6961 KOps/s | 72.0629 KOps/s | $\color{#d91a1a}-0.51\\%$ | | test_values[generalized_advantage_estimate-True-True] | 11.1893ms | 9.4631ms | 105.6740 Ops/s | 108.6256 Ops/s | $\color{#d91a1a}-2.72\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 38.4336ms | 33.7583ms | 29.6224 Ops/s | 27.5026 Ops/s | $\textbf{\color{#35bf28}+7.71\\%}$ | | test_values[td0_return_estimate-False-False] | 0.2331ms | 0.1673ms | 5.9785 KOps/s | 6.4356 KOps/s | $\textbf{\color{#d91a1a}-7.10\\%}$ | | test_values[td1_return_estimate-False-False] | 24.2562ms | 23.5175ms | 42.5215 Ops/s | 43.5727 Ops/s | $\color{#d91a1a}-2.41\\%$ | | test_values[vec_td1_return_estimate-False-False] | 35.7683ms | 33.7124ms | 29.6627 Ops/s | 27.3898 Ops/s | $\textbf{\color{#35bf28}+8.30\\%}$ | | test_values[td_lambda_return_estimate-True-False] | 37.7402ms | 34.1084ms | 29.3183 Ops/s | 30.1687 Ops/s | $\color{#d91a1a}-2.82\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 35.8095ms | 33.7522ms | 29.6277 Ops/s | 27.2068 Ops/s | $\textbf{\color{#35bf28}+8.90\\%}$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 11.7119ms | 8.2817ms | 120.7475 Ops/s | 125.1234 Ops/s | $\color{#d91a1a}-3.50\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.2271ms | 1.9591ms | 510.4326 Ops/s | 502.9047 Ops/s | $\color{#35bf28}+1.50\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.6561ms | 0.3597ms | 2.7805 KOps/s | 2.8121 KOps/s | $\color{#d91a1a}-1.13\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 41.9795ms | 40.6826ms | 24.5806 Ops/s | 21.6825 Ops/s | $\textbf{\color{#35bf28}+13.37\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.8381ms | 3.0262ms | 330.4484 Ops/s | 332.2761 Ops/s | $\color{#d91a1a}-0.55\\%$ | | test_dqn_speed | 1.8484ms | 1.3828ms | 723.1460 Ops/s | 715.9157 Ops/s | $\color{#35bf28}+1.01\\%$ | | test_ddpg_speed | 3.7873ms | 2.9386ms | 340.2928 Ops/s | 343.4403 Ops/s | $\color{#d91a1a}-0.92\\%$ | | test_sac_speed | 10.0926ms | 8.6281ms | 115.9001 Ops/s | 117.8942 Ops/s | $\color{#d91a1a}-1.69\\%$ | | test_redq_speed | 14.6131ms | 13.4923ms | 74.1162 Ops/s | 71.7569 Ops/s | $\color{#35bf28}+3.29\\%$ | | test_redq_deprec_speed | 14.8832ms | 13.7015ms | 72.9845 Ops/s | 75.6199 Ops/s | $\color{#d91a1a}-3.49\\%$ | | test_td3_speed | 8.7545ms | 8.4531ms | 118.2997 Ops/s | 118.0804 Ops/s | $\color{#35bf28}+0.19\\%$ | | test_cql_speed | 38.5134ms | 36.5597ms | 27.3525 Ops/s | 27.7987 Ops/s | $\color{#d91a1a}-1.61\\%$ | | test_a2c_speed | 8.5651ms | 7.4220ms | 134.7338 Ops/s | 132.4915 Ops/s | $\color{#35bf28}+1.69\\%$ | | test_ppo_speed | 8.8511ms | 7.7291ms | 129.3813 Ops/s | 125.6152 Ops/s | $\color{#35bf28}+3.00\\%$ | | test_reinforce_speed | 8.0128ms | 6.5979ms | 151.5629 Ops/s | 151.9280 Ops/s | $\color{#d91a1a}-0.24\\%$ | | test_iql_speed | 34.5665ms | 32.3574ms | 30.9048 Ops/s | 30.6730 Ops/s | $\color{#35bf28}+0.76\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 7.4905ms | 4.9076ms | 203.7664 Ops/s | 204.0627 Ops/s | $\color{#d91a1a}-0.15\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6914ms | 0.4846ms | 2.0635 KOps/s | 2.0357 KOps/s | $\color{#35bf28}+1.37\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6339ms | 0.4597ms | 2.1755 KOps/s | 2.1593 KOps/s | $\color{#35bf28}+0.75\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.9931ms | 4.8759ms | 205.0887 Ops/s | 204.4616 Ops/s | $\color{#35bf28}+0.31\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.1206s | 0.5680ms | 1.7605 KOps/s | 2.0685 KOps/s | $\textbf{\color{#d91a1a}-14.89\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6761ms | 0.4558ms | 2.1939 KOps/s | 2.1433 KOps/s | $\color{#35bf28}+2.36\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 3.4936ms | 1.7511ms | 571.0859 Ops/s | 571.4235 Ops/s | $\color{#d91a1a}-0.06\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.1654ms | 1.6139ms | 619.6301 Ops/s | 625.4639 Ops/s | $\color{#d91a1a}-0.93\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.3544ms | 5.1093ms | 195.7214 Ops/s | 198.5567 Ops/s | $\color{#d91a1a}-1.43\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9088ms | 0.6335ms | 1.5785 KOps/s | 1.5992 KOps/s | $\color{#d91a1a}-1.30\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 1.1586ms | 0.6095ms | 1.6406 KOps/s | 1.6690 KOps/s | $\color{#d91a1a}-1.70\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.1010ms | 4.9377ms | 202.5229 Ops/s | 203.4601 Ops/s | $\color{#d91a1a}-0.46\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.2182ms | 0.4850ms | 2.0617 KOps/s | 2.0723 KOps/s | $\color{#d91a1a}-0.51\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7601ms | 0.4655ms | 2.1482 KOps/s | 2.1260 KOps/s | $\color{#35bf28}+1.04\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.9882ms | 5.0342ms | 198.6396 Ops/s | 200.6013 Ops/s | $\color{#d91a1a}-0.98\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8194ms | 0.4939ms | 2.0247 KOps/s | 2.0808 KOps/s | $\color{#d91a1a}-2.70\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7086ms | 0.4707ms | 2.1244 KOps/s | 2.1848 KOps/s | $\color{#d91a1a}-2.76\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 7.3931ms | 5.1024ms | 195.9862 Ops/s | 199.3332 Ops/s | $\color{#d91a1a}-1.68\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.7526ms | 0.6217ms | 1.6084 KOps/s | 1.5845 KOps/s | $\color{#35bf28}+1.51\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.6928ms | 0.6019ms | 1.6614 KOps/s | 1.6427 KOps/s | $\color{#35bf28}+1.14\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1303s | 6.3276ms | 158.0388 Ops/s | 110.0774 Ops/s | $\textbf{\color{#35bf28}+43.57\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 17.5011ms | 12.9355ms | 77.3065 Ops/s | 78.0222 Ops/s | $\color{#d91a1a}-0.92\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 4.7633ms | 1.1963ms | 835.8797 Ops/s | 846.8311 Ops/s | $\color{#d91a1a}-1.29\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1293s | 6.2625ms | 159.6796 Ops/s | 163.9864 Ops/s | $\color{#d91a1a}-2.63\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 17.4211ms | 13.0336ms | 76.7248 Ops/s | 77.1361 Ops/s | $\color{#d91a1a}-0.53\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.8618ms | 1.1433ms | 874.6638 Ops/s | 842.5641 Ops/s | $\color{#35bf28}+3.81\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1243s | 8.7459ms | 114.3399 Ops/s | 148.0767 Ops/s | $\textbf{\color{#d91a1a}-22.78\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.0665ms | 13.3333ms | 75.0003 Ops/s | 75.7352 Ops/s | $\color{#d91a1a}-0.97\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.7612ms | 1.2479ms | 801.3191 Ops/s | 835.7314 Ops/s | $\color{#d91a1a}-4.12\\%$ |
github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1069s | 0.1054s | 9.4858 Ops/s | 8.6395 Ops/s | $\textbf{\color{#35bf28}+9.80\\%}$ | | test_sync | 95.1722ms | 93.2641ms | 10.7222 Ops/s | 10.5230 Ops/s | $\color{#35bf28}+1.89\\%$ | | test_async | 0.1788s | 89.7218ms | 11.1456 Ops/s | 11.2618 Ops/s | $\color{#d91a1a}-1.03\\%$ | | test_single_pixels | 0.1158s | 0.1155s | 8.6577 Ops/s | 8.6060 Ops/s | $\color{#35bf28}+0.60\\%$ | | test_sync_pixels | 76.4797ms | 75.0574ms | 13.3231 Ops/s | 13.5416 Ops/s | $\color{#d91a1a}-1.61\\%$ | | test_async_pixels | 0.1396s | 68.2166ms | 14.6592 Ops/s | 14.4204 Ops/s | $\color{#35bf28}+1.66\\%$ | | test_simple | 0.8600s | 0.7838s | 1.2759 Ops/s | 1.2404 Ops/s | $\color{#35bf28}+2.86\\%$ | | test_transformed | 1.0813s | 1.0067s | 0.9934 Ops/s | 0.9814 Ops/s | $\color{#35bf28}+1.22\\%$ | | test_serial | 2.2684s | 2.1977s | 0.4550 Ops/s | 0.4415 Ops/s | $\color{#35bf28}+3.07\\%$ | | test_parallel | 2.0260s | 1.9647s | 0.5090 Ops/s | 0.5093 Ops/s | $\color{#d91a1a}-0.06\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1115ms | 36.9381μs | 27.0723 KOps/s | 26.0178 KOps/s | $\color{#35bf28}+4.05\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 46.3030μs | 21.4017μs | 46.7253 KOps/s | 46.8724 KOps/s | $\color{#d91a1a}-0.31\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 0.1166ms | 21.4389μs | 46.6441 KOps/s | 46.5876 KOps/s | $\color{#35bf28}+0.12\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 29.9820μs | 12.1187μs | 82.5174 KOps/s | 82.6347 KOps/s | $\color{#d91a1a}-0.14\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 70.1340μs | 39.5470μs | 25.2863 KOps/s | 24.5125 KOps/s | $\color{#35bf28}+3.16\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 44.1630μs | 23.4363μs | 42.6689 KOps/s | 42.3632 KOps/s | $\color{#35bf28}+0.72\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 47.3120μs | 23.7296μs | 42.1414 KOps/s | 41.6255 KOps/s | $\color{#35bf28}+1.24\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 39.7620μs | 14.4291μs | 69.3046 KOps/s | 69.3692 KOps/s | $\color{#d91a1a}-0.09\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 76.5040μs | 42.5370μs | 23.5090 KOps/s | 23.6151 KOps/s | $\color{#d91a1a}-0.45\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 56.1130μs | 25.9725μs | 38.5023 KOps/s | 38.5963 KOps/s | $\color{#d91a1a}-0.24\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 47.4430μs | 23.6667μs | 42.2534 KOps/s | 41.7775 KOps/s | $\color{#35bf28}+1.14\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 36.1120μs | 14.4451μs | 69.2276 KOps/s | 69.8576 KOps/s | $\color{#d91a1a}-0.90\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 71.5440μs | 44.1371μs | 22.6567 KOps/s | 22.1805 KOps/s | $\color{#35bf28}+2.15\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 54.6130μs | 28.0439μs | 35.6583 KOps/s | 35.2815 KOps/s | $\color{#35bf28}+1.07\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 0.2197ms | 25.9139μs | 38.5893 KOps/s | 37.8856 KOps/s | $\color{#35bf28}+1.86\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 0.1281ms | 16.7430μs | 59.7266 KOps/s | 59.5675 KOps/s | $\color{#35bf28}+0.27\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 0.2471ms | 41.8706μs | 23.8831 KOps/s | 23.4197 KOps/s | $\color{#35bf28}+1.98\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 0.1158ms | 25.9442μs | 38.5443 KOps/s | 38.3155 KOps/s | $\color{#35bf28}+0.60\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 61.4240μs | 27.9665μs | 35.7571 KOps/s | 35.5660 KOps/s | $\color{#35bf28}+0.54\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 0.2185ms | 17.0272μs | 58.7294 KOps/s | 60.3081 KOps/s | $\color{#d91a1a}-2.62\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 71.2140μs | 44.0436μs | 22.7048 KOps/s | 22.1819 KOps/s | $\color{#35bf28}+2.36\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 0.1164ms | 28.0234μs | 35.6845 KOps/s | 35.2718 KOps/s | $\color{#35bf28}+1.17\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 56.1330μs | 30.0514μs | 33.2763 KOps/s | 32.7402 KOps/s | $\color{#35bf28}+1.64\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 42.4220μs | 18.7108μs | 53.4451 KOps/s | 53.2745 KOps/s | $\color{#35bf28}+0.32\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 4.1187ms | 47.2059μs | 21.1838 KOps/s | 20.9906 KOps/s | $\color{#35bf28}+0.92\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 59.6430μs | 30.7186μs | 32.5536 KOps/s | 32.3367 KOps/s | $\color{#35bf28}+0.67\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 55.6630μs | 30.0332μs | 33.2965 KOps/s | 32.6825 KOps/s | $\color{#35bf28}+1.88\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 39.9020μs | 18.8351μs | 53.0922 KOps/s | 53.5458 KOps/s | $\color{#d91a1a}-0.85\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 85.8450μs | 48.5958μs | 20.5779 KOps/s | 20.3597 KOps/s | $\color{#35bf28}+1.07\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 59.4230μs | 32.6473μs | 30.6304 KOps/s | 30.4548 KOps/s | $\color{#35bf28}+0.58\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 0.1883ms | 32.1741μs | 31.0809 KOps/s | 31.0069 KOps/s | $\color{#35bf28}+0.24\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 43.6230μs | 20.9533μs | 47.7251 KOps/s | 48.6977 KOps/s | $\color{#d91a1a}-2.00\\%$ | | test_values[generalized_advantage_estimate-True-True] | 23.9387ms | 23.6067ms | 42.3609 Ops/s | 40.9440 Ops/s | $\color{#35bf28}+3.46\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 96.0680ms | 2.8083ms | 356.0914 Ops/s | 366.2589 Ops/s | $\color{#d91a1a}-2.78\\%$ | | test_values[td0_return_estimate-False-False] | 0.1023ms | 63.9518μs | 15.6368 KOps/s | 15.3800 KOps/s | $\color{#35bf28}+1.67\\%$ | | test_values[td1_return_estimate-False-False] | 54.3601ms | 53.3809ms | 18.7333 Ops/s | 18.2502 Ops/s | $\color{#35bf28}+2.65\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.3884ms | 1.0676ms | 936.6648 Ops/s | 928.8654 Ops/s | $\color{#35bf28}+0.84\\%$ | | test_values[td_lambda_return_estimate-True-False] | 87.2138ms | 83.9454ms | 11.9125 Ops/s | 11.0288 Ops/s | $\textbf{\color{#35bf28}+8.01\\%}$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.4313ms | 1.0652ms | 938.8306 Ops/s | 934.0088 Ops/s | $\color{#35bf28}+0.52\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 24.0106ms | 23.6987ms | 42.1964 Ops/s | 38.9062 Ops/s | $\textbf{\color{#35bf28}+8.46\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9233ms | 0.7249ms | 1.3795 KOps/s | 1.4001 KOps/s | $\color{#d91a1a}-1.47\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.8319ms | 0.6748ms | 1.4818 KOps/s | 1.4711 KOps/s | $\color{#35bf28}+0.73\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5999ms | 1.4509ms | 689.2474 Ops/s | 682.9488 Ops/s | $\color{#35bf28}+0.92\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.8169ms | 0.6654ms | 1.5029 KOps/s | 1.4848 KOps/s | $\color{#35bf28}+1.21\\%$ | | test_dqn_speed | 7.4665ms | 1.4312ms | 698.7035 Ops/s | 704.7610 Ops/s | $\color{#d91a1a}-0.86\\%$ | | test_ddpg_speed | 3.2979ms | 2.9220ms | 342.2329 Ops/s | 337.3966 Ops/s | $\color{#35bf28}+1.43\\%$ | | test_sac_speed | 0.1040s | 9.0511ms | 110.4832 Ops/s | 120.1503 Ops/s | $\textbf{\color{#d91a1a}-8.05\\%}$ | | test_redq_speed | 11.8115ms | 10.9706ms | 91.1527 Ops/s | 91.0933 Ops/s | $\color{#35bf28}+0.07\\%$ | | test_redq_deprec_speed | 11.9487ms | 11.0694ms | 90.3389 Ops/s | 85.2064 Ops/s | $\textbf{\color{#35bf28}+6.02\\%}$ | | test_td3_speed | 9.7699ms | 8.2725ms | 120.8827 Ops/s | 120.8605 Ops/s | $\color{#35bf28}+0.02\\%$ | | test_cql_speed | 27.5475ms | 26.3064ms | 38.0135 Ops/s | 34.9076 Ops/s | $\textbf{\color{#35bf28}+8.90\\%}$ | | test_a2c_speed | 5.9209ms | 5.6793ms | 176.0777 Ops/s | 171.4569 Ops/s | $\color{#35bf28}+2.69\\%$ | | test_ppo_speed | 6.5498ms | 5.9902ms | 166.9404 Ops/s | 161.9935 Ops/s | $\color{#35bf28}+3.05\\%$ | | test_reinforce_speed | 6.3261ms | 4.6343ms | 215.7806 Ops/s | 209.0726 Ops/s | $\color{#35bf28}+3.21\\%$ | | test_iql_speed | 20.4878ms | 19.9703ms | 50.0745 Ops/s | 49.0206 Ops/s | $\color{#35bf28}+2.15\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.9489ms | 6.7093ms | 149.0472 Ops/s | 149.9728 Ops/s | $\color{#d91a1a}-0.62\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.9183ms | 0.5197ms | 1.9243 KOps/s | 1.9208 KOps/s | $\color{#35bf28}+0.18\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6952ms | 0.4971ms | 2.0118 KOps/s | 1.9980 KOps/s | $\color{#35bf28}+0.69\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.9969ms | 6.6175ms | 151.1151 Ops/s | 151.8288 Ops/s | $\color{#d91a1a}-0.47\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.7463ms | 0.5138ms | 1.9463 KOps/s | 1.9524 KOps/s | $\color{#d91a1a}-0.31\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6406ms | 0.4924ms | 2.0311 KOps/s | 2.0230 KOps/s | $\color{#35bf28}+0.40\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.1592ms | 1.9369ms | 516.2931 Ops/s | 503.9648 Ops/s | $\color{#35bf28}+2.45\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2055ms | 1.8431ms | 542.5586 Ops/s | 529.3259 Ops/s | $\color{#35bf28}+2.50\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 8.7056ms | 6.9798ms | 143.2705 Ops/s | 146.5347 Ops/s | $\color{#d91a1a}-2.23\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.1475s | 0.7872ms | 1.2702 KOps/s | 1.4942 KOps/s | $\textbf{\color{#d91a1a}-14.99\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8349ms | 0.6376ms | 1.5685 KOps/s | 1.4950 KOps/s | $\color{#35bf28}+4.91\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.8673ms | 6.6922ms | 149.4272 Ops/s | 149.5076 Ops/s | $\color{#d91a1a}-0.05\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.5577ms | 0.5160ms | 1.9379 KOps/s | 1.9314 KOps/s | $\color{#35bf28}+0.34\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7022ms | 0.4968ms | 2.0128 KOps/s | 1.9961 KOps/s | $\color{#35bf28}+0.83\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.9675ms | 6.6138ms | 151.1990 Ops/s | 150.8025 Ops/s | $\color{#35bf28}+0.26\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7001ms | 0.5135ms | 1.9474 KOps/s | 1.9470 KOps/s | $\color{#35bf28}+0.02\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 4.2613ms | 0.4971ms | 2.0119 KOps/s | 2.0324 KOps/s | $\color{#d91a1a}-1.01\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 7.1517ms | 6.8573ms | 145.8297 Ops/s | 146.4733 Ops/s | $\color{#d91a1a}-0.44\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.4212ms | 0.6679ms | 1.4973 KOps/s | 1.1882 KOps/s | $\textbf{\color{#35bf28}+26.01\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8457ms | 0.6415ms | 1.5588 KOps/s | 1.5485 KOps/s | $\color{#35bf28}+0.66\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1483s | 8.0923ms | 123.5745 Ops/s | 130.6899 Ops/s | $\textbf{\color{#d91a1a}-5.44\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 19.3290ms | 16.2900ms | 61.3873 Ops/s | 60.6852 Ops/s | $\color{#35bf28}+1.16\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 6.1800ms | 1.3505ms | 740.4644 Ops/s | 857.8244 Ops/s | $\textbf{\color{#d91a1a}-13.68\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1276s | 7.6639ms | 130.4819 Ops/s | 131.8090 Ops/s | $\color{#d91a1a}-1.01\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 18.7338ms | 16.2771ms | 61.4359 Ops/s | 60.6796 Ops/s | $\color{#35bf28}+1.25\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.4002ms | 1.2694ms | 787.7441 Ops/s | 751.6010 Ops/s | $\color{#35bf28}+4.81\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1299s | 10.4225ms | 95.9459 Ops/s | 128.6270 Ops/s | $\textbf{\color{#d91a1a}-25.41\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 19.2199ms | 16.3671ms | 61.0980 Ops/s | 59.9528 Ops/s | $\color{#35bf28}+1.91\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.6534ms | 1.4289ms | 699.8395 Ops/s | 783.1654 Ops/s | $\textbf{\color{#d91a1a}-10.64\\%}$ |