search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.05k stars 273 forks source link

[BugFix] Fix brax wrapping #2190

Closed vmoens closed 1 month ago

vmoens commented 1 month ago

Solves #2183

pytorch-bot[bot] commented 1 month ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2190

Note: Links to docs will display an error until the docs builds have been completed.

:heavy_exclamation_mark: 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

:x: 2 New Failures, 9 Unrelated Failures

As of commit 958a303943acd0279851f303fc620b1496b9af3f with merge base c0c32a03761f1c20317f7e898888f76b4a73facc (image):

NEW FAILURES - The following jobs have failed:

* [Generate documentation / build-docs (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2190#25646967965) ([gh](https://github.com/pytorch/rl/actions/runs/9317181881/job/25646967965)) `No files were found with the provided path: /home/ec2-user/actions-runner/_work/_temp/artifacts/. No artifacts will be uploaded.` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2190#25646967127) ([gh](https://github.com/pytorch/rl/actions/runs/9317181875/job/25646967127)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

* [Examples Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2190#25646967957) ([gh](https://github.com/pytorch/rl/actions/runs/9317181866/job/25646967957)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2190#25646968003) ([gh](https://github.com/pytorch/rl/actions/runs/9317181868/job/25646968003)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2190#25646974714) ([gh](https://github.com/pytorch/rl/actions/runs/9317181874/job/25646974714)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2190#25646975003) ([gh](https://github.com/pytorch/rl/actions/runs/9317181874/job/25646975003)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2190#25646967973) ([gh](https://github.com/pytorch/rl/actions/runs/9317181885/job/25646967973)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-gpu (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2190#25646971356) ([gh](https://github.com/pytorch/rl/actions/runs/9317181872/job/25646971356)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2190#25646972322) ([gh](https://github.com/pytorch/rl/actions/runs/9317181872/job/25646972322)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2190#25646972000) ([gh](https://github.com/pytorch/rl/actions/runs/9317181872/job/25646972000)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2190#25646972569) ([gh](https://github.com/pytorch/rl/actions/runs/9317181872/job/25646972569)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 month ago

$\color{#35bf28}\textsf{\Large\✔\kern{0.2cm}\normalsize OK}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}0$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1077s | 57.5602ms | 17.3731 Ops/s | 18.1198 Ops/s | $\color{#d91a1a}-4.12\\%$ | | test_sync | 36.2467ms | 30.0820ms | 33.2425 Ops/s | 33.6638 Ops/s | $\color{#d91a1a}-1.25\\%$ | | test_async | 71.1936ms | 29.4534ms | 33.9520 Ops/s | 35.6613 Ops/s | $\color{#d91a1a}-4.79\\%$ | | test_simple | 0.4268s | 0.3698s | 2.7040 Ops/s | 2.7971 Ops/s | $\color{#d91a1a}-3.33\\%$ | | test_transformed | 0.5689s | 0.5207s | 1.9207 Ops/s | 1.9322 Ops/s | $\color{#d91a1a}-0.59\\%$ | | test_serial | 1.2838s | 1.2454s | 0.8030 Ops/s | 0.7938 Ops/s | $\color{#35bf28}+1.16\\%$ | | test_parallel | 1.0996s | 1.0224s | 0.9781 Ops/s | 0.9290 Ops/s | $\textbf{\color{#35bf28}+5.29\\%}$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1121ms | 21.4803μs | 46.5542 KOps/s | 46.8077 KOps/s | $\color{#d91a1a}-0.54\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 32.9720μs | 13.0001μs | 76.9222 KOps/s | 76.8006 KOps/s | $\color{#35bf28}+0.16\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 37.2700μs | 12.5934μs | 79.4064 KOps/s | 78.5122 KOps/s | $\color{#35bf28}+1.14\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 26.9910μs | 7.6528μs | 130.6712 KOps/s | 130.3841 KOps/s | $\color{#35bf28}+0.22\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 70.7220μs | 22.8090μs | 43.8424 KOps/s | 43.5447 KOps/s | $\color{#35bf28}+0.68\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 37.1700μs | 14.4693μs | 69.1119 KOps/s | 70.0132 KOps/s | $\color{#d91a1a}-1.29\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 35.5270μs | 13.8682μs | 72.1076 KOps/s | 71.5234 KOps/s | $\color{#35bf28}+0.82\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 31.4990μs | 8.9848μs | 111.2996 KOps/s | 112.0640 KOps/s | $\color{#d91a1a}-0.68\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 53.4500μs | 24.4167μs | 40.9556 KOps/s | 40.7236 KOps/s | $\color{#35bf28}+0.57\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 39.8250μs | 15.9177μs | 62.8230 KOps/s | 63.1943 KOps/s | $\color{#d91a1a}-0.59\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 41.8080μs | 14.0915μs | 70.9646 KOps/s | 71.8642 KOps/s | $\color{#d91a1a}-1.25\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 34.5350μs | 8.9822μs | 111.3311 KOps/s | 110.5863 KOps/s | $\color{#35bf28}+0.67\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 58.3300μs | 25.5732μs | 39.1034 KOps/s | 38.9871 KOps/s | $\color{#35bf28}+0.30\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 60.2930μs | 16.7697μs | 59.6315 KOps/s | 58.6024 KOps/s | $\color{#35bf28}+1.76\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 43.2310μs | 15.1644μs | 65.9440 KOps/s | 65.8241 KOps/s | $\color{#35bf28}+0.18\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 30.7670μs | 10.0725μs | 99.2807 KOps/s | 97.6594 KOps/s | $\color{#35bf28}+1.66\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 91.6320μs | 24.2230μs | 41.2830 KOps/s | 41.0911 KOps/s | $\color{#35bf28}+0.47\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 45.6460μs | 15.6593μs | 63.8599 KOps/s | 63.3340 KOps/s | $\color{#35bf28}+0.83\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 38.8620μs | 16.3210μs | 61.2706 KOps/s | 61.6542 KOps/s | $\color{#d91a1a}-0.62\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 32.8920μs | 10.0543μs | 99.4597 KOps/s | 99.4292 KOps/s | $\color{#35bf28}+0.03\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 51.8770μs | 24.9946μs | 40.0086 KOps/s | 39.3629 KOps/s | $\color{#35bf28}+1.64\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 43.1510μs | 16.8304μs | 59.4162 KOps/s | 58.6518 KOps/s | $\color{#35bf28}+1.30\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 39.5350μs | 17.1505μs | 58.3075 KOps/s | 56.6600 KOps/s | $\color{#35bf28}+2.91\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 39.3130μs | 11.2596μs | 88.8130 KOps/s | 87.9645 KOps/s | $\color{#35bf28}+0.96\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 61.1050μs | 26.2546μs | 38.0886 KOps/s | 37.3633 KOps/s | $\color{#35bf28}+1.94\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 43.1310μs | 18.1201μs | 55.1874 KOps/s | 55.2296 KOps/s | $\color{#d91a1a}-0.08\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 42.8810μs | 17.3209μs | 57.7338 KOps/s | 57.4269 KOps/s | $\color{#35bf28}+0.53\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 38.1520μs | 11.2857μs | 88.6076 KOps/s | 88.9760 KOps/s | $\color{#d91a1a}-0.41\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 44.7240μs | 28.6284μs | 34.9304 KOps/s | 35.5432 KOps/s | $\color{#d91a1a}-1.72\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 47.5590μs | 19.2583μs | 51.9257 KOps/s | 52.7992 KOps/s | $\color{#d91a1a}-1.65\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 48.6920μs | 18.4938μs | 54.0721 KOps/s | 54.2926 KOps/s | $\color{#d91a1a}-0.41\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 39.2140μs | 12.4891μs | 80.0695 KOps/s | 80.7879 KOps/s | $\color{#d91a1a}-0.89\\%$ | | test_values[generalized_advantage_estimate-True-True] | 12.1753ms | 9.4437ms | 105.8903 Ops/s | 104.4091 Ops/s | $\color{#35bf28}+1.42\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 36.3308ms | 35.0819ms | 28.5047 Ops/s | 28.6459 Ops/s | $\color{#d91a1a}-0.49\\%$ | | test_values[td0_return_estimate-False-False] | 0.2279ms | 0.1603ms | 6.2364 KOps/s | 6.0159 KOps/s | $\color{#35bf28}+3.66\\%$ | | test_values[td1_return_estimate-False-False] | 25.2890ms | 22.8203ms | 43.8206 Ops/s | 41.5494 Ops/s | $\textbf{\color{#35bf28}+5.47\\%}$ | | test_values[vec_td1_return_estimate-False-False] | 37.5431ms | 35.1195ms | 28.4742 Ops/s | 28.5922 Ops/s | $\color{#d91a1a}-0.41\\%$ | | test_values[td_lambda_return_estimate-True-False] | 35.9340ms | 33.0252ms | 30.2799 Ops/s | 29.0594 Ops/s | $\color{#35bf28}+4.20\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 36.0681ms | 34.9657ms | 28.5994 Ops/s | 28.5285 Ops/s | $\color{#35bf28}+0.25\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 11.4003ms | 8.1616ms | 122.5257 Ops/s | 118.2178 Ops/s | $\color{#35bf28}+3.64\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.2124ms | 1.7854ms | 560.1134 Ops/s | 512.1558 Ops/s | $\textbf{\color{#35bf28}+9.36\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.6190ms | 0.3486ms | 2.8686 KOps/s | 2.7855 KOps/s | $\color{#35bf28}+2.99\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 46.6674ms | 44.8678ms | 22.2877 Ops/s | 23.3323 Ops/s | $\color{#d91a1a}-4.48\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.0937ms | 3.0039ms | 332.9013 Ops/s | 330.2009 Ops/s | $\color{#35bf28}+0.82\\%$ | | test_dqn_speed | 2.1180ms | 1.3453ms | 743.3184 Ops/s | 737.1243 Ops/s | $\color{#35bf28}+0.84\\%$ | | test_ddpg_speed | 3.4402ms | 2.8344ms | 352.8022 Ops/s | 348.4801 Ops/s | $\color{#35bf28}+1.24\\%$ | | test_sac_speed | 9.8178ms | 8.3855ms | 119.2530 Ops/s | 118.0034 Ops/s | $\color{#35bf28}+1.06\\%$ | | test_redq_speed | 15.2173ms | 13.1781ms | 75.8833 Ops/s | 73.6227 Ops/s | $\color{#35bf28}+3.07\\%$ | | test_redq_deprec_speed | 13.9952ms | 13.2255ms | 75.6117 Ops/s | 73.9102 Ops/s | $\color{#35bf28}+2.30\\%$ | | test_td3_speed | 8.8066ms | 8.2841ms | 120.7133 Ops/s | 118.6262 Ops/s | $\color{#35bf28}+1.76\\%$ | | test_cql_speed | 38.2035ms | 36.6838ms | 27.2600 Ops/s | 27.1839 Ops/s | $\color{#35bf28}+0.28\\%$ | | test_a2c_speed | 8.8224ms | 7.3841ms | 135.4253 Ops/s | 131.8990 Ops/s | $\color{#35bf28}+2.67\\%$ | | test_ppo_speed | 8.2433ms | 7.6554ms | 130.6270 Ops/s | 122.2776 Ops/s | $\textbf{\color{#35bf28}+6.83\\%}$ | | test_reinforce_speed | 7.8414ms | 6.5971ms | 151.5825 Ops/s | 146.2782 Ops/s | $\color{#35bf28}+3.63\\%$ | | test_iql_speed | 33.9598ms | 32.4223ms | 30.8430 Ops/s | 30.5207 Ops/s | $\color{#35bf28}+1.06\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.1642ms | 3.4431ms | 290.4375 Ops/s | 273.6078 Ops/s | $\textbf{\color{#35bf28}+6.15\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.1568ms | 0.5030ms | 1.9882 KOps/s | 1.9288 KOps/s | $\color{#35bf28}+3.08\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.8327ms | 0.5045ms | 1.9822 KOps/s | 2.0262 KOps/s | $\color{#d91a1a}-2.17\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.0855ms | 3.5364ms | 282.7769 Ops/s | 277.0558 Ops/s | $\color{#35bf28}+2.06\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7741ms | 0.4922ms | 2.0318 KOps/s | 1.9639 KOps/s | $\color{#35bf28}+3.46\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.9667ms | 0.4753ms | 2.1039 KOps/s | 2.0632 KOps/s | $\color{#35bf28}+1.97\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 1.9812ms | 1.2368ms | 808.5250 Ops/s | 779.1291 Ops/s | $\color{#35bf28}+3.77\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 4.2164ms | 1.1711ms | 853.9081 Ops/s | 838.4266 Ops/s | $\color{#35bf28}+1.85\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.8368ms | 3.5643ms | 280.5567 Ops/s | 271.8015 Ops/s | $\color{#35bf28}+3.22\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.1720ms | 0.6143ms | 1.6279 KOps/s | 1.6179 KOps/s | $\color{#35bf28}+0.61\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9368ms | 0.5856ms | 1.7076 KOps/s | 1.6941 KOps/s | $\color{#35bf28}+0.80\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.0646ms | 3.4228ms | 292.1559 Ops/s | 282.0154 Ops/s | $\color{#35bf28}+3.60\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 91.9701ms | 0.5999ms | 1.6669 KOps/s | 1.6340 KOps/s | $\color{#35bf28}+2.01\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6108ms | 0.4692ms | 2.1313 KOps/s | 2.0789 KOps/s | $\color{#35bf28}+2.52\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.1371ms | 3.4341ms | 291.2008 Ops/s | 284.5323 Ops/s | $\color{#35bf28}+2.34\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6614ms | 0.4872ms | 2.0527 KOps/s | 2.0178 KOps/s | $\color{#35bf28}+1.73\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 3.5779ms | 0.4721ms | 2.1181 KOps/s | 2.1107 KOps/s | $\color{#35bf28}+0.35\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.2414ms | 3.5299ms | 283.2944 Ops/s | 276.1872 Ops/s | $\color{#35bf28}+2.57\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.6897ms | 0.6074ms | 1.6464 KOps/s | 1.3645 KOps/s | $\textbf{\color{#35bf28}+20.66\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8335ms | 0.5899ms | 1.6952 KOps/s | 1.6954 KOps/s | $-0.01\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1026s | 5.7004ms | 175.4271 Ops/s | 180.4598 Ops/s | $\color{#d91a1a}-2.79\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 14.4893ms | 12.3201ms | 81.1685 Ops/s | 78.8026 Ops/s | $\color{#35bf28}+3.00\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 1.5774ms | 1.0572ms | 945.8848 Ops/s | 946.8460 Ops/s | $\color{#d91a1a}-0.10\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 92.9668ms | 7.1214ms | 140.4222 Ops/s | 137.7568 Ops/s | $\color{#35bf28}+1.93\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 18.0508ms | 12.3416ms | 81.0265 Ops/s | 78.7258 Ops/s | $\color{#35bf28}+2.92\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.5360ms | 1.0591ms | 944.2313 Ops/s | 903.8041 Ops/s | $\color{#35bf28}+4.47\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 89.9219ms | 5.6855ms | 175.8861 Ops/s | 169.5231 Ops/s | $\color{#35bf28}+3.75\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 17.1629ms | 13.0431ms | 76.6692 Ops/s | 76.9653 Ops/s | $\color{#d91a1a}-0.38\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.0733ms | 1.4693ms | 680.5978 Ops/s | 685.2914 Ops/s | $\color{#d91a1a}-0.68\\%$ |
github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1626s | 0.1081s | 9.2498 Ops/s | 9.8107 Ops/s | $\textbf{\color{#d91a1a}-5.72\\%}$ | | test_sync | 94.8201ms | 94.5150ms | 10.5803 Ops/s | 10.5294 Ops/s | $\color{#35bf28}+0.48\\%$ | | test_async | 0.1771s | 89.5747ms | 11.1639 Ops/s | 11.1286 Ops/s | $\color{#35bf28}+0.32\\%$ | | test_single_pixels | 0.1177s | 0.1166s | 8.5794 Ops/s | 8.7650 Ops/s | $\color{#d91a1a}-2.12\\%$ | | test_sync_pixels | 83.0086ms | 77.2804ms | 12.9399 Ops/s | 13.0194 Ops/s | $\color{#d91a1a}-0.61\\%$ | | test_async_pixels | 0.1548s | 66.4950ms | 15.0387 Ops/s | 16.3042 Ops/s | $\textbf{\color{#d91a1a}-7.76\\%}$ | | test_simple | 0.7048s | 0.7014s | 1.4257 Ops/s | 1.3918 Ops/s | $\color{#35bf28}+2.44\\%$ | | test_transformed | 0.9231s | 0.9209s | 1.0860 Ops/s | 1.0638 Ops/s | $\color{#35bf28}+2.08\\%$ | | test_serial | 2.2268s | 2.1671s | 0.4614 Ops/s | 0.4577 Ops/s | $\color{#35bf28}+0.81\\%$ | | test_parallel | 2.3055s | 2.2403s | 0.4464 Ops/s | 0.4515 Ops/s | $\color{#d91a1a}-1.14\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1062ms | 26.4352μs | 37.8283 KOps/s | 38.2000 KOps/s | $\color{#d91a1a}-0.97\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 0.1298ms | 16.2280μs | 61.6220 KOps/s | 62.3375 KOps/s | $\color{#d91a1a}-1.15\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 0.1310ms | 15.2252μs | 65.6804 KOps/s | 65.7270 KOps/s | $\color{#d91a1a}-0.07\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 26.4010μs | 9.3475μs | 106.9803 KOps/s | 107.3935 KOps/s | $\color{#d91a1a}-0.38\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 0.1383ms | 28.0782μs | 35.6148 KOps/s | 36.3211 KOps/s | $\color{#d91a1a}-1.94\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 0.1224ms | 17.6190μs | 56.7569 KOps/s | 57.2792 KOps/s | $\color{#d91a1a}-0.91\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 38.1600μs | 16.9402μs | 59.0310 KOps/s | 60.5684 KOps/s | $\color{#d91a1a}-2.54\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 0.1245ms | 10.8803μs | 91.9096 KOps/s | 92.2574 KOps/s | $\color{#d91a1a}-0.38\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 0.1415ms | 29.4441μs | 33.9626 KOps/s | 34.2576 KOps/s | $\color{#d91a1a}-0.86\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 0.1253ms | 19.3127μs | 51.7794 KOps/s | 52.7370 KOps/s | $\color{#d91a1a}-1.82\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 34.2910μs | 16.7049μs | 59.8628 KOps/s | 60.1457 KOps/s | $\color{#d91a1a}-0.47\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 0.1265ms | 10.8839μs | 91.8786 KOps/s | 91.6220 KOps/s | $\color{#35bf28}+0.28\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 0.1401ms | 31.1344μs | 32.1188 KOps/s | 32.8582 KOps/s | $\color{#d91a1a}-2.25\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 55.2010μs | 20.8662μs | 47.9243 KOps/s | 48.8070 KOps/s | $\color{#d91a1a}-1.81\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 0.1272ms | 18.2958μs | 54.6573 KOps/s | 54.9765 KOps/s | $\color{#d91a1a}-0.58\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 0.1185ms | 12.4600μs | 80.2568 KOps/s | 80.1388 KOps/s | $\color{#35bf28}+0.15\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 65.8010μs | 29.3139μs | 34.1135 KOps/s | 34.5279 KOps/s | $\color{#d91a1a}-1.20\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 0.1290ms | 19.2958μs | 51.8248 KOps/s | 52.3467 KOps/s | $\color{#d91a1a}-1.00\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 0.1300ms | 19.9102μs | 50.2255 KOps/s | 49.1270 KOps/s | $\color{#35bf28}+2.24\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 39.9810μs | 12.3585μs | 80.9161 KOps/s | 81.7806 KOps/s | $\color{#d91a1a}-1.06\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 0.1441ms | 31.3678μs | 31.8798 KOps/s | 32.5792 KOps/s | $\color{#d91a1a}-2.15\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 0.1285ms | 20.7316μs | 48.2356 KOps/s | 48.3824 KOps/s | $\color{#d91a1a}-0.30\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 46.9610μs | 21.7169μs | 46.0470 KOps/s | 46.0235 KOps/s | $\color{#35bf28}+0.05\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 33.8110μs | 13.9277μs | 71.7994 KOps/s | 72.5987 KOps/s | $\color{#d91a1a}-1.10\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 0.1310ms | 32.4808μs | 30.7874 KOps/s | 30.9851 KOps/s | $\color{#d91a1a}-0.64\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 0.1316ms | 22.5570μs | 44.3322 KOps/s | 44.8177 KOps/s | $\color{#d91a1a}-1.08\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 49.7200μs | 21.8190μs | 45.8316 KOps/s | 46.9379 KOps/s | $\color{#d91a1a}-2.36\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 0.1329ms | 13.9598μs | 71.6344 KOps/s | 72.2002 KOps/s | $\color{#d91a1a}-0.78\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 0.1395ms | 34.2527μs | 29.1948 KOps/s | 29.8078 KOps/s | $\color{#d91a1a}-2.06\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 48.9710μs | 24.2382μs | 41.2572 KOps/s | 42.3642 KOps/s | $\color{#d91a1a}-2.61\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 0.1277ms | 22.6006μs | 44.2466 KOps/s | 44.7402 KOps/s | $\color{#d91a1a}-1.10\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 0.1278ms | 15.4702μs | 64.6405 KOps/s | 65.5679 KOps/s | $\color{#d91a1a}-1.41\\%$ | | test_values[generalized_advantage_estimate-True-True] | 23.8719ms | 22.7812ms | 43.8958 Ops/s | 43.0801 Ops/s | $\color{#35bf28}+1.89\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 80.5759ms | 3.1528ms | 317.1763 Ops/s | 314.5866 Ops/s | $\color{#35bf28}+0.82\\%$ | | test_values[td0_return_estimate-False-False] | 88.6210μs | 61.0159μs | 16.3892 KOps/s | 17.3863 KOps/s | $\textbf{\color{#d91a1a}-5.74\\%}$ | | test_values[td1_return_estimate-False-False] | 49.0603ms | 48.6414ms | 20.5586 Ops/s | 20.4633 Ops/s | $\color{#35bf28}+0.47\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.9865ms | 1.7386ms | 575.1611 Ops/s | 582.7249 Ops/s | $\color{#d91a1a}-1.30\\%$ | | test_values[td_lambda_return_estimate-True-False] | 78.6554ms | 77.9738ms | 12.8248 Ops/s | 12.5681 Ops/s | $\color{#35bf28}+2.04\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 2.1368ms | 1.7365ms | 575.8831 Ops/s | 585.5070 Ops/s | $\color{#d91a1a}-1.64\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 21.8105ms | 21.6088ms | 46.2774 Ops/s | 45.7855 Ops/s | $\color{#35bf28}+1.07\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.8747ms | 0.6686ms | 1.4957 KOps/s | 1.5221 KOps/s | $\color{#d91a1a}-1.73\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7161ms | 0.6275ms | 1.5936 KOps/s | 1.5971 KOps/s | $\color{#d91a1a}-0.21\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.4896ms | 1.4255ms | 701.4997 Ops/s | 715.5565 Ops/s | $\color{#d91a1a}-1.96\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.9228ms | 0.6441ms | 1.5526 KOps/s | 1.6196 KOps/s | $\color{#d91a1a}-4.14\\%$ | | test_dqn_speed | 8.1560ms | 1.2545ms | 797.1479 Ops/s | 824.6943 Ops/s | $\color{#d91a1a}-3.34\\%$ | | test_ddpg_speed | 3.7023ms | 2.6019ms | 384.3362 Ops/s | 395.7222 Ops/s | $\color{#d91a1a}-2.88\\%$ | | test_sac_speed | 8.0900ms | 7.4644ms | 133.9698 Ops/s | 137.7923 Ops/s | $\color{#d91a1a}-2.77\\%$ | | test_redq_speed | 10.5652ms | 9.8780ms | 101.2351 Ops/s | 103.7411 Ops/s | $\color{#d91a1a}-2.42\\%$ | | test_redq_deprec_speed | 10.9845ms | 10.5363ms | 94.9100 Ops/s | 96.6504 Ops/s | $\color{#d91a1a}-1.80\\%$ | | test_td3_speed | 17.2169ms | 7.5210ms | 132.9611 Ops/s | 137.6081 Ops/s | $\color{#d91a1a}-3.38\\%$ | | test_cql_speed | 24.6191ms | 23.8080ms | 42.0026 Ops/s | 42.3394 Ops/s | $\color{#d91a1a}-0.80\\%$ | | test_a2c_speed | 5.5057ms | 5.2671ms | 189.8567 Ops/s | 187.4076 Ops/s | $\color{#35bf28}+1.31\\%$ | | test_ppo_speed | 5.9013ms | 5.5695ms | 179.5491 Ops/s | 175.8115 Ops/s | $\color{#35bf28}+2.13\\%$ | | test_reinforce_speed | 4.5558ms | 4.3293ms | 230.9817 Ops/s | 230.7426 Ops/s | $\color{#35bf28}+0.10\\%$ | | test_iql_speed | 18.9739ms | 18.3262ms | 54.5665 Ops/s | 53.9552 Ops/s | $\color{#35bf28}+1.13\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.0688ms | 3.8952ms | 256.7276 Ops/s | 260.7549 Ops/s | $\color{#d91a1a}-1.54\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7427ms | 0.5585ms | 1.7904 KOps/s | 1.8218 KOps/s | $\color{#d91a1a}-1.72\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7756ms | 0.5397ms | 1.8529 KOps/s | 1.8681 KOps/s | $\color{#d91a1a}-0.82\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.0731ms | 3.8766ms | 257.9560 Ops/s | 260.9044 Ops/s | $\color{#d91a1a}-1.13\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8900ms | 0.5520ms | 1.8115 KOps/s | 1.8441 KOps/s | $\color{#d91a1a}-1.77\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 4.8428ms | 0.5340ms | 1.8726 KOps/s | 1.8916 KOps/s | $\color{#d91a1a}-1.01\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 1.5235ms | 1.3796ms | 724.8497 Ops/s | 738.9697 Ops/s | $\color{#d91a1a}-1.91\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 1.5171ms | 1.3159ms | 759.9445 Ops/s | 773.8639 Ops/s | $\color{#d91a1a}-1.80\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.1019ms | 3.9730ms | 251.6968 Ops/s | 252.7250 Ops/s | $\color{#d91a1a}-0.41\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.4172ms | 0.6736ms | 1.4845 KOps/s | 1.5024 KOps/s | $\color{#d91a1a}-1.19\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7956ms | 0.6568ms | 1.5225 KOps/s | 1.5468 KOps/s | $\color{#d91a1a}-1.57\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.0422ms | 3.8889ms | 257.1417 Ops/s | 262.9426 Ops/s | $\color{#d91a1a}-2.21\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6846ms | 0.5575ms | 1.7938 KOps/s | 1.8120 KOps/s | $\color{#d91a1a}-1.00\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 5.4360ms | 0.5483ms | 1.8239 KOps/s | 1.8619 KOps/s | $\color{#d91a1a}-2.04\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.1762ms | 3.8986ms | 256.5000 Ops/s | 261.8776 Ops/s | $\color{#d91a1a}-2.05\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7117ms | 0.5555ms | 1.8003 KOps/s | 1.8324 KOps/s | $\color{#d91a1a}-1.75\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6685ms | 0.5326ms | 1.8776 KOps/s | 1.8831 KOps/s | $\color{#d91a1a}-0.29\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.2529ms | 4.0302ms | 248.1255 Ops/s | 254.6457 Ops/s | $\color{#d91a1a}-2.56\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.3757ms | 0.6758ms | 1.4796 KOps/s | 1.5106 KOps/s | $\color{#d91a1a}-2.05\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8359ms | 0.6522ms | 1.5333 KOps/s | 1.5522 KOps/s | $\color{#d91a1a}-1.22\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1271s | 6.8864ms | 145.2147 Ops/s | 141.6163 Ops/s | $\color{#35bf28}+2.54\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 15.1253ms | 12.5239ms | 79.8473 Ops/s | 80.5295 Ops/s | $\color{#d91a1a}-0.85\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 7.4323ms | 1.4014ms | 713.5494 Ops/s | 714.8427 Ops/s | $\color{#d91a1a}-0.18\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1223s | 6.8298ms | 146.4175 Ops/s | 148.7574 Ops/s | $\color{#d91a1a}-1.57\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1263s | 14.8190ms | 67.4807 Ops/s | 80.8656 Ops/s | $\textbf{\color{#d91a1a}-16.55\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 7.3656ms | 1.3527ms | 739.2756 Ops/s | 699.4761 Ops/s | $\textbf{\color{#35bf28}+5.69\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1207s | 7.1104ms | 140.6391 Ops/s | 106.5843 Ops/s | $\textbf{\color{#35bf28}+31.95\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 15.4438ms | 12.8558ms | 77.7857 Ops/s | 78.6249 Ops/s | $\color{#d91a1a}-1.07\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 7.2142ms | 1.7010ms | 587.8959 Ops/s | 601.6773 Ops/s | $\color{#d91a1a}-2.29\\%$ |