search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.01k stars 269 forks source link

Revert "[BugFix] Fix non-tensor passage in _StepMDP" #2261

Closed vmoens closed 1 week ago

vmoens commented 1 week ago

Reverts pytorch/rl#2260

pytorch-bot[bot] commented 1 week ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2261

Note: Links to docs will display an error until the docs builds have been completed.

:x: 2 New Failures, 15 Unrelated Failures

As of commit fdace2cb59cd43f6960bb771ac8bc24ad8f50b4d with merge base 5fa486ccdeaf179f63f5aeed9213f4da97e985c6 (image):

NEW FAILURES - The following jobs have failed:

* [Generate documentation / build-docs (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931250711) ([gh](https://github.com/pytorch/rl/actions/runs/9757887385/job/26931250711)) `No files were found with the provided path: /home/ec2-user/actions-runner/_work/_temp/artifacts/. No artifacts will be uploaded.` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931249747) ([gh](https://github.com/pytorch/rl/actions/runs/9757887360/job/26931249747)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

* [Examples Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931251582) ([gh](https://github.com/pytorch/rl/actions/runs/9757887365/job/26931251582)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931251717) ([gh](https://github.com/pytorch/rl/actions/runs/9757887384/job/26931251717)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931261025) ([gh](https://github.com/pytorch/rl/actions/runs/9757887398/job/26931261025)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931261414) ([gh](https://github.com/pytorch/rl/actions/runs/9757887398/job/26931261414)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Lint / c-source / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931249948) ([gh](https://github.com/pytorch/rl/actions/runs/9757887383/job/26931249948)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Lint / python-source-and-configs / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931251122) ([gh](https://github.com/pytorch/rl/actions/runs/9757887383/job/26931251122)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931250654) ([gh](https://github.com/pytorch/rl/actions/runs/9757887369/job/26931250654)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.10) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931251810) ([gh](https://github.com/pytorch/rl/actions/runs/9757887375/job/26931251810)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.11) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931252469) ([gh](https://github.com/pytorch/rl/actions/runs/9757887375/job/26931252469)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.8) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931253136) ([gh](https://github.com/pytorch/rl/actions/runs/9757887375/job/26931253136)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.9) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931254116) ([gh](https://github.com/pytorch/rl/actions/runs/9757887375/job/26931254116)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-gpu (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931254950) ([gh](https://github.com/pytorch/rl/actions/runs/9757887375/job/26931254950)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931255572) ([gh](https://github.com/pytorch/rl/actions/runs/9757887375/job/26931255572)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931255883) ([gh](https://github.com/pytorch/rl/actions/runs/9757887375/job/26931255883)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2261#26931256416) ([gh](https://github.com/pytorch/rl/actions/runs/9757887375/job/26931256416)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 week ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}9$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1149s | 58.6320ms | 17.0555 Ops/s | 17.8054 Ops/s | $\color{#d91a1a}-4.21\\%$ | | test_sync | 36.6197ms | 30.2141ms | 33.0972 Ops/s | 32.6890 Ops/s | $\color{#35bf28}+1.25\\%$ | | test_async | 51.9996ms | 29.4482ms | 33.9579 Ops/s | 34.0101 Ops/s | $\color{#d91a1a}-0.15\\%$ | | test_simple | 0.3803s | 0.3741s | 2.6733 Ops/s | 2.6073 Ops/s | $\color{#35bf28}+2.53\\%$ | | test_transformed | 0.5396s | 0.5375s | 1.8606 Ops/s | 1.8196 Ops/s | $\color{#35bf28}+2.25\\%$ | | test_serial | 1.3230s | 1.2645s | 0.7908 Ops/s | 0.7773 Ops/s | $\color{#35bf28}+1.73\\%$ | | test_parallel | 1.1169s | 1.0733s | 0.9317 Ops/s | 0.9180 Ops/s | $\color{#35bf28}+1.50\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1541ms | 22.3562μs | 44.7303 KOps/s | 44.6225 KOps/s | $\color{#35bf28}+0.24\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 58.3290μs | 13.2815μs | 75.2925 KOps/s | 74.3767 KOps/s | $\color{#35bf28}+1.23\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 36.1770μs | 13.0880μs | 76.4059 KOps/s | 76.3985 KOps/s | $+0.01\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 25.8280μs | 7.7219μs | 129.5018 KOps/s | 128.3828 KOps/s | $\color{#35bf28}+0.87\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 72.5350μs | 23.7097μs | 42.1768 KOps/s | 41.5225 KOps/s | $\color{#35bf28}+1.58\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 41.8890μs | 14.5827μs | 68.5746 KOps/s | 67.4272 KOps/s | $\color{#35bf28}+1.70\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 45.0540μs | 14.2933μs | 69.9628 KOps/s | 69.1765 KOps/s | $\color{#35bf28}+1.14\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 46.4370μs | 8.8931μs | 112.4462 KOps/s | 108.8161 KOps/s | $\color{#35bf28}+3.34\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 54.4820μs | 25.1607μs | 39.7445 KOps/s | 39.7192 KOps/s | $\color{#35bf28}+0.06\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 40.0050μs | 15.9729μs | 62.6062 KOps/s | 62.2112 KOps/s | $\color{#35bf28}+0.63\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 35.3150μs | 14.2093μs | 70.3764 KOps/s | 69.0875 KOps/s | $\color{#35bf28}+1.87\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 29.6050μs | 8.9514μs | 111.7138 KOps/s | 110.0257 KOps/s | $\color{#35bf28}+1.53\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 81.6420μs | 26.2970μs | 38.0271 KOps/s | 37.7294 KOps/s | $\color{#35bf28}+0.79\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 36.6880μs | 17.0570μs | 58.6270 KOps/s | 57.3518 KOps/s | $\color{#35bf28}+2.22\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 44.0730μs | 15.5248μs | 64.4131 KOps/s | 64.1059 KOps/s | $\color{#35bf28}+0.48\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 23.4640μs | 10.1071μs | 98.9404 KOps/s | 97.0219 KOps/s | $\color{#35bf28}+1.98\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 57.2060μs | 24.9057μs | 40.1514 KOps/s | 39.7574 KOps/s | $\color{#35bf28}+0.99\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 36.4680μs | 15.7462μs | 63.5075 KOps/s | 61.9593 KOps/s | $\color{#35bf28}+2.50\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 44.5230μs | 16.4612μs | 60.7491 KOps/s | 59.7384 KOps/s | $\color{#35bf28}+1.69\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 29.2740μs | 10.1651μs | 98.3762 KOps/s | 97.0910 KOps/s | $\color{#35bf28}+1.32\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 59.9620μs | 26.1424μs | 38.2520 KOps/s | 37.6680 KOps/s | $\color{#35bf28}+1.55\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 37.5200μs | 16.8762μs | 59.2550 KOps/s | 57.7924 KOps/s | $\color{#35bf28}+2.53\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 43.6610μs | 17.5614μs | 56.9430 KOps/s | 55.9472 KOps/s | $\color{#35bf28}+1.78\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 27.5710μs | 11.2801μs | 88.6520 KOps/s | 86.9016 KOps/s | $\color{#35bf28}+2.01\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 59.6520μs | 27.7487μs | 36.0378 KOps/s | 35.5137 KOps/s | $\color{#35bf28}+1.48\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 52.0570μs | 18.4857μs | 54.0958 KOps/s | 53.1233 KOps/s | $\color{#35bf28}+1.83\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 41.8180μs | 17.6667μs | 56.6036 KOps/s | 56.3087 KOps/s | $\color{#35bf28}+0.52\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 31.4180μs | 11.4000μs | 87.7195 KOps/s | 86.3724 KOps/s | $\color{#35bf28}+1.56\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 42.7100μs | 28.9902μs | 34.4944 KOps/s | 34.0557 KOps/s | $\color{#35bf28}+1.29\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 54.1910μs | 19.6175μs | 50.9749 KOps/s | 50.7309 KOps/s | $\color{#35bf28}+0.48\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 44.3220μs | 18.7114μs | 53.4435 KOps/s | 53.1924 KOps/s | $\color{#35bf28}+0.47\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 41.4370μs | 12.4525μs | 80.3050 KOps/s | 78.7911 KOps/s | $\color{#35bf28}+1.92\\%$ | | test_values[generalized_advantage_estimate-True-True] | 9.9350ms | 9.5286ms | 104.9477 Ops/s | 98.7411 Ops/s | $\textbf{\color{#35bf28}+6.29\\%}$ | | test_values[vec_generalized_advantage_estimate-True-True] | 35.9534ms | 33.3688ms | 29.9681 Ops/s | 28.3331 Ops/s | $\textbf{\color{#35bf28}+5.77\\%}$ | | test_values[td0_return_estimate-False-False] | 0.2328ms | 0.1657ms | 6.0358 KOps/s | 6.0958 KOps/s | $\color{#d91a1a}-0.98\\%$ | | test_values[td1_return_estimate-False-False] | 27.7329ms | 23.4688ms | 42.6098 Ops/s | 39.9341 Ops/s | $\textbf{\color{#35bf28}+6.70\\%}$ | | test_values[vec_td1_return_estimate-False-False] | 34.3802ms | 33.3206ms | 30.0115 Ops/s | 28.1693 Ops/s | $\textbf{\color{#35bf28}+6.54\\%}$ | | test_values[td_lambda_return_estimate-True-False] | 36.7335ms | 33.4588ms | 29.8875 Ops/s | 27.6869 Ops/s | $\textbf{\color{#35bf28}+7.95\\%}$ | | test_values[vec_td_lambda_return_estimate-True-False] | 35.3807ms | 33.4750ms | 29.8731 Ops/s | 28.1317 Ops/s | $\textbf{\color{#35bf28}+6.19\\%}$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 17.0335ms | 8.4117ms | 118.8816 Ops/s | 113.6190 Ops/s | $\color{#35bf28}+4.63\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.3126ms | 2.0018ms | 499.5520 Ops/s | 499.1509 Ops/s | $\color{#35bf28}+0.08\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4343ms | 0.3575ms | 2.7975 KOps/s | 2.8016 KOps/s | $\color{#d91a1a}-0.15\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 40.1685ms | 39.7405ms | 25.1633 Ops/s | 21.6474 Ops/s | $\textbf{\color{#35bf28}+16.24\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.6212ms | 3.0454ms | 328.3632 Ops/s | 328.2609 Ops/s | $\color{#35bf28}+0.03\\%$ | | test_dqn_speed | 1.9662ms | 1.3514ms | 739.9821 Ops/s | 742.5264 Ops/s | $\color{#d91a1a}-0.34\\%$ | | test_ddpg_speed | 3.0316ms | 2.8494ms | 350.9492 Ops/s | 352.2232 Ops/s | $\color{#d91a1a}-0.36\\%$ | | test_sac_speed | 9.9968ms | 8.5401ms | 117.0945 Ops/s | 117.4018 Ops/s | $\color{#d91a1a}-0.26\\%$ | | test_redq_speed | 14.9270ms | 13.4671ms | 74.2550 Ops/s | 67.1649 Ops/s | $\textbf{\color{#35bf28}+10.56\\%}$ | | test_redq_deprec_speed | 14.3089ms | 13.4063ms | 74.5920 Ops/s | 74.0631 Ops/s | $\color{#35bf28}+0.71\\%$ | | test_td3_speed | 17.1724ms | 8.5674ms | 116.7217 Ops/s | 118.3254 Ops/s | $\color{#d91a1a}-1.36\\%$ | | test_cql_speed | 38.2046ms | 36.8978ms | 27.1019 Ops/s | 27.0075 Ops/s | $\color{#35bf28}+0.35\\%$ | | test_a2c_speed | 8.1138ms | 7.5136ms | 133.0916 Ops/s | 133.0267 Ops/s | $\color{#35bf28}+0.05\\%$ | | test_ppo_speed | 9.1519ms | 7.7830ms | 128.4854 Ops/s | 128.9678 Ops/s | $\color{#d91a1a}-0.37\\%$ | | test_reinforce_speed | 7.2615ms | 6.7141ms | 148.9401 Ops/s | 146.3807 Ops/s | $\color{#35bf28}+1.75\\%$ | | test_iql_speed | 34.7193ms | 32.8704ms | 30.4225 Ops/s | 30.4229 Ops/s | $-0.00\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.6377ms | 3.4655ms | 288.5594 Ops/s | 282.6413 Ops/s | $\color{#35bf28}+2.09\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 93.9115ms | 0.5567ms | 1.7962 KOps/s | 2.0295 KOps/s | $\textbf{\color{#d91a1a}-11.50\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7964ms | 0.4726ms | 2.1161 KOps/s | 2.1392 KOps/s | $\color{#d91a1a}-1.08\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.1602ms | 3.5258ms | 283.6252 Ops/s | 286.6283 Ops/s | $\color{#d91a1a}-1.05\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.0132ms | 0.5158ms | 1.9387 KOps/s | 2.0462 KOps/s | $\textbf{\color{#d91a1a}-5.25\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6555ms | 0.4632ms | 2.1589 KOps/s | 2.1490 KOps/s | $\color{#35bf28}+0.46\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.4203ms | 1.7197ms | 581.5114 Ops/s | 576.6239 Ops/s | $\color{#35bf28}+0.85\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 4.6192ms | 1.6372ms | 610.8155 Ops/s | 603.0891 Ops/s | $\color{#35bf28}+1.28\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.9352ms | 3.5959ms | 278.0916 Ops/s | 270.8480 Ops/s | $\color{#35bf28}+2.67\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.1635ms | 0.6373ms | 1.5690 KOps/s | 1.5942 KOps/s | $\color{#d91a1a}-1.58\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9652ms | 0.6079ms | 1.6449 KOps/s | 1.6553 KOps/s | $\color{#d91a1a}-0.63\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.6667ms | 3.4637ms | 288.7107 Ops/s | 284.3186 Ops/s | $\color{#35bf28}+1.54\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.0911ms | 0.4980ms | 2.0082 KOps/s | 2.0354 KOps/s | $\color{#d91a1a}-1.34\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.5800ms | 0.4709ms | 2.1235 KOps/s | 2.1450 KOps/s | $\color{#d91a1a}-1.01\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.7315ms | 3.4292ms | 291.6099 Ops/s | 283.9410 Ops/s | $\color{#35bf28}+2.70\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8247ms | 0.4944ms | 2.0228 KOps/s | 2.0481 KOps/s | $\color{#d91a1a}-1.24\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 3.6203ms | 0.4697ms | 2.1292 KOps/s | 2.1224 KOps/s | $\color{#35bf28}+0.32\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.1717ms | 3.6137ms | 276.7270 Ops/s | 273.0778 Ops/s | $\color{#35bf28}+1.34\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.7200ms | 0.6323ms | 1.5816 KOps/s | 1.5816 KOps/s | $+0.00\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9993ms | 0.6096ms | 1.6403 KOps/s | 1.6586 KOps/s | $\color{#d91a1a}-1.10\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1184s | 8.0845ms | 123.6934 Ops/s | 167.0029 Ops/s | $\textbf{\color{#d91a1a}-25.93\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 14.7342ms | 12.5592ms | 79.6229 Ops/s | 68.0180 Ops/s | $\textbf{\color{#35bf28}+17.06\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 1.0878ms | 1.0354ms | 965.7875 Ops/s | 971.7071 Ops/s | $\color{#d91a1a}-0.61\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1121s | 5.8273ms | 171.6071 Ops/s | 174.9488 Ops/s | $\color{#d91a1a}-1.91\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 14.6765ms | 12.4332ms | 80.4300 Ops/s | 79.3372 Ops/s | $\color{#35bf28}+1.38\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.1429ms | 1.0377ms | 963.7091 Ops/s | 969.8791 Ops/s | $\color{#d91a1a}-0.64\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1101s | 5.9548ms | 167.9307 Ops/s | 166.7627 Ops/s | $\color{#35bf28}+0.70\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 0.1183s | 14.9869ms | 66.7251 Ops/s | 78.6301 Ops/s | $\textbf{\color{#d91a1a}-15.14\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.8755ms | 1.2113ms | 825.5506 Ops/s | 844.4968 Ops/s | $\color{#d91a1a}-2.24\\%$ |
github-actions[bot] commented 1 week ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1615s | 0.1089s | 9.1824 Ops/s | 9.3615 Ops/s | $\color{#d91a1a}-1.91\\%$ | | test_sync | 91.9906ms | 90.1155ms | 11.0969 Ops/s | 11.2863 Ops/s | $\color{#d91a1a}-1.68\\%$ | | test_async | 0.1627s | 70.8893ms | 14.1065 Ops/s | 13.5014 Ops/s | $\color{#35bf28}+4.48\\%$ | | test_single_pixels | 0.1135s | 0.1131s | 8.8436 Ops/s | 8.6231 Ops/s | $\color{#35bf28}+2.56\\%$ | | test_sync_pixels | 79.4976ms | 72.7997ms | 13.7363 Ops/s | 14.6265 Ops/s | $\textbf{\color{#d91a1a}-6.09\\%}$ | | test_async_pixels | 0.1218s | 57.0609ms | 17.5251 Ops/s | 17.5530 Ops/s | $\color{#d91a1a}-0.16\\%$ | | test_simple | 0.7235s | 0.7213s | 1.3864 Ops/s | 1.3200 Ops/s | $\textbf{\color{#35bf28}+5.03\\%}$ | | test_transformed | 1.0284s | 0.9663s | 1.0349 Ops/s | 1.0040 Ops/s | $\color{#35bf28}+3.07\\%$ | | test_serial | 2.1909s | 2.1300s | 0.4695 Ops/s | 0.4491 Ops/s | $\color{#35bf28}+4.55\\%$ | | test_parallel | 1.9131s | 1.8548s | 0.5392 Ops/s | 0.5318 Ops/s | $\color{#35bf28}+1.38\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1061ms | 32.4645μs | 30.8029 KOps/s | 30.0047 KOps/s | $\color{#35bf28}+2.66\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 49.0720μs | 18.7387μs | 53.3656 KOps/s | 50.9908 KOps/s | $\color{#35bf28}+4.66\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 0.1290ms | 18.5165μs | 54.0059 KOps/s | 53.1421 KOps/s | $\color{#35bf28}+1.63\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 27.4120μs | 10.7981μs | 92.6092 KOps/s | 89.5863 KOps/s | $\color{#35bf28}+3.37\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 0.1396ms | 34.1048μs | 29.3214 KOps/s | 27.9866 KOps/s | $\color{#35bf28}+4.77\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 0.1308ms | 20.6075μs | 48.5260 KOps/s | 46.8189 KOps/s | $\color{#35bf28}+3.65\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 37.7020μs | 20.1738μs | 49.5692 KOps/s | 47.6297 KOps/s | $\color{#35bf28}+4.07\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 0.1153ms | 12.4434μs | 80.3641 KOps/s | 77.5884 KOps/s | $\color{#35bf28}+3.58\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 0.1407ms | 36.2017μs | 27.6230 KOps/s | 26.8028 KOps/s | $\color{#35bf28}+3.06\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 45.6230μs | 22.8378μs | 43.7870 KOps/s | 42.6873 KOps/s | $\color{#35bf28}+2.58\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 37.1020μs | 20.1544μs | 49.6171 KOps/s | 47.5672 KOps/s | $\color{#35bf28}+4.31\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 0.1206ms | 12.4716μs | 80.1821 KOps/s | 77.2970 KOps/s | $\color{#35bf28}+3.73\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 0.1419ms | 37.9494μs | 26.3509 KOps/s | 25.6361 KOps/s | $\color{#35bf28}+2.79\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 58.1840μs | 24.4370μs | 40.9216 KOps/s | 40.0240 KOps/s | $\color{#35bf28}+2.24\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 0.1320ms | 21.7333μs | 46.0122 KOps/s | 44.1817 KOps/s | $\color{#35bf28}+4.14\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 0.1222ms | 14.2640μs | 70.1067 KOps/s | 68.2428 KOps/s | $\color{#35bf28}+2.73\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 0.1509ms | 36.1287μs | 27.6788 KOps/s | 26.6233 KOps/s | $\color{#35bf28}+3.96\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 44.5920μs | 22.5659μs | 44.3146 KOps/s | 42.5446 KOps/s | $\color{#35bf28}+4.16\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 0.1362ms | 24.3500μs | 41.0678 KOps/s | 40.6510 KOps/s | $\color{#35bf28}+1.03\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 0.1110ms | 14.1775μs | 70.5341 KOps/s | 67.8261 KOps/s | $\color{#35bf28}+3.99\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 68.3240μs | 37.9022μs | 26.3837 KOps/s | 25.5636 KOps/s | $\color{#35bf28}+3.21\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 0.1279ms | 24.1382μs | 41.4281 KOps/s | 39.9535 KOps/s | $\color{#35bf28}+3.69\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 0.1319ms | 25.9155μs | 38.5870 KOps/s | 37.3450 KOps/s | $\color{#35bf28}+3.33\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 0.1213ms | 15.9883μs | 62.5458 KOps/s | 60.2199 KOps/s | $\color{#35bf28}+3.86\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 81.0250μs | 39.4594μs | 25.3425 KOps/s | 24.5121 KOps/s | $\color{#35bf28}+3.39\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 0.1288ms | 26.0873μs | 38.3328 KOps/s | 37.2774 KOps/s | $\color{#35bf28}+2.83\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 0.1284ms | 26.1215μs | 38.2826 KOps/s | 37.6329 KOps/s | $\color{#35bf28}+1.73\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 0.1321ms | 16.0477μs | 62.3142 KOps/s | 60.4159 KOps/s | $\color{#35bf28}+3.14\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 69.2140μs | 42.5235μs | 23.5164 KOps/s | 23.1807 KOps/s | $\color{#35bf28}+1.45\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 51.5230μs | 28.0798μs | 35.6128 KOps/s | 35.0215 KOps/s | $\color{#35bf28}+1.69\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 0.1458ms | 26.9484μs | 37.1080 KOps/s | 35.6924 KOps/s | $\color{#35bf28}+3.97\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 0.1211ms | 17.6657μs | 56.6070 KOps/s | 54.9674 KOps/s | $\color{#35bf28}+2.98\\%$ | | test_values[generalized_advantage_estimate-True-True] | 27.1892ms | 26.3186ms | 37.9960 Ops/s | 38.8484 Ops/s | $\color{#d91a1a}-2.19\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 97.6702ms | 2.8674ms | 348.7497 Ops/s | 350.0489 Ops/s | $\color{#d91a1a}-0.37\\%$ | | test_values[td0_return_estimate-False-False] | 93.9850μs | 66.3150μs | 15.0795 KOps/s | 14.9159 KOps/s | $\color{#35bf28}+1.10\\%$ | | test_values[td1_return_estimate-False-False] | 59.2139ms | 57.8956ms | 17.2725 Ops/s | 17.0783 Ops/s | $\color{#35bf28}+1.14\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.4950ms | 1.1137ms | 897.8822 Ops/s | 903.0150 Ops/s | $\color{#d91a1a}-0.57\\%$ | | test_values[td_lambda_return_estimate-True-False] | 92.8340ms | 91.4625ms | 10.9334 Ops/s | 10.6492 Ops/s | $\color{#35bf28}+2.67\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.3103ms | 1.1023ms | 907.2195 Ops/s | 911.9926 Ops/s | $\color{#d91a1a}-0.52\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 26.5570ms | 26.3037ms | 38.0174 Ops/s | 37.8583 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 1.0133ms | 0.7489ms | 1.3353 KOps/s | 1.3494 KOps/s | $\color{#d91a1a}-1.05\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7718ms | 0.6900ms | 1.4492 KOps/s | 1.4570 KOps/s | $\color{#d91a1a}-0.53\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.6092ms | 1.4893ms | 671.4755 Ops/s | 674.1877 Ops/s | $\color{#d91a1a}-0.40\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7515ms | 0.7052ms | 1.4180 KOps/s | 1.4322 KOps/s | $\color{#d91a1a}-0.99\\%$ | | test_dqn_speed | 7.9608ms | 1.4777ms | 676.7161 Ops/s | 653.3616 Ops/s | $\color{#35bf28}+3.57\\%$ | | test_ddpg_speed | 3.2402ms | 2.9940ms | 334.0027 Ops/s | 319.5708 Ops/s | $\color{#35bf28}+4.52\\%$ | | test_sac_speed | 9.1161ms | 8.5823ms | 116.5188 Ops/s | 111.9805 Ops/s | $\color{#35bf28}+4.05\\%$ | | test_redq_speed | 0.1068s | 12.3543ms | 80.9437 Ops/s | 88.4832 Ops/s | $\textbf{\color{#d91a1a}-8.52\\%}$ | | test_redq_deprec_speed | 12.5631ms | 11.9300ms | 83.8222 Ops/s | 81.0488 Ops/s | $\color{#35bf28}+3.42\\%$ | | test_td3_speed | 8.9389ms | 8.5030ms | 117.6049 Ops/s | 112.6447 Ops/s | $\color{#35bf28}+4.40\\%$ | | test_cql_speed | 27.4571ms | 26.4892ms | 37.7513 Ops/s | 36.4287 Ops/s | $\color{#35bf28}+3.63\\%$ | | test_a2c_speed | 6.8413ms | 5.9558ms | 167.9027 Ops/s | 166.9411 Ops/s | $\color{#35bf28}+0.58\\%$ | | test_ppo_speed | 6.9503ms | 6.2761ms | 159.3347 Ops/s | 154.8805 Ops/s | $\color{#35bf28}+2.88\\%$ | | test_reinforce_speed | 5.1948ms | 4.8557ms | 205.9437 Ops/s | 202.2142 Ops/s | $\color{#35bf28}+1.84\\%$ | | test_iql_speed | 20.9418ms | 20.3796ms | 49.0688 Ops/s | 48.4120 Ops/s | $\color{#35bf28}+1.36\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.8034ms | 4.5776ms | 218.4571 Ops/s | 212.7743 Ops/s | $\color{#35bf28}+2.67\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.1111s | 0.6398ms | 1.5629 KOps/s | 1.7700 KOps/s | $\textbf{\color{#d91a1a}-11.70\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7735ms | 0.5305ms | 1.8851 KOps/s | 1.8473 KOps/s | $\color{#35bf28}+2.05\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.7459ms | 4.5075ms | 221.8510 Ops/s | 213.4592 Ops/s | $\color{#35bf28}+3.93\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.6425ms | 0.5504ms | 1.8170 KOps/s | 1.7805 KOps/s | $\color{#35bf28}+2.05\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6756ms | 0.5262ms | 1.9004 KOps/s | 1.8465 KOps/s | $\color{#35bf28}+2.92\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.1887ms | 2.0197ms | 495.1315 Ops/s | 471.4755 Ops/s | $\textbf{\color{#35bf28}+5.02\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.0982ms | 1.9164ms | 521.8227 Ops/s | 497.8112 Ops/s | $\color{#35bf28}+4.82\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.7895ms | 4.6911ms | 213.1709 Ops/s | 206.7553 Ops/s | $\color{#35bf28}+3.10\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.4184ms | 0.7062ms | 1.4160 KOps/s | 1.1866 KOps/s | $\textbf{\color{#35bf28}+19.33\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8683ms | 0.6825ms | 1.4652 KOps/s | 1.4426 KOps/s | $\color{#35bf28}+1.57\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.9539ms | 4.5702ms | 218.8089 Ops/s | 213.0465 Ops/s | $\color{#35bf28}+2.70\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.2832ms | 0.5542ms | 1.8044 KOps/s | 1.7782 KOps/s | $\color{#35bf28}+1.47\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7107ms | 0.5296ms | 1.8882 KOps/s | 1.8442 KOps/s | $\color{#35bf28}+2.39\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.8781ms | 4.5663ms | 218.9940 Ops/s | 214.3688 Ops/s | $\color{#35bf28}+2.16\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7382ms | 0.5515ms | 1.8132 KOps/s | 1.7874 KOps/s | $\color{#35bf28}+1.44\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7220ms | 0.5246ms | 1.9062 KOps/s | 1.8598 KOps/s | $\color{#35bf28}+2.49\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.8436ms | 4.7274ms | 211.5332 Ops/s | 206.2625 Ops/s | $\color{#35bf28}+2.56\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.6147ms | 0.7096ms | 1.4092 KOps/s | 1.3912 KOps/s | $\color{#35bf28}+1.29\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8726ms | 0.6877ms | 1.4542 KOps/s | 1.4414 KOps/s | $\color{#35bf28}+0.88\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1518s | 7.8001ms | 128.2034 Ops/s | 125.8300 Ops/s | $\color{#35bf28}+1.89\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 17.6685ms | 15.4248ms | 64.8307 Ops/s | 59.9304 Ops/s | $\textbf{\color{#35bf28}+8.18\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 7.5282ms | 1.3586ms | 736.0560 Ops/s | 735.2182 Ops/s | $\color{#35bf28}+0.11\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1279s | 7.3399ms | 136.2422 Ops/s | 133.8484 Ops/s | $\color{#35bf28}+1.79\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 17.8290ms | 15.5084ms | 64.4811 Ops/s | 59.6984 Ops/s | $\textbf{\color{#35bf28}+8.01\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 7.4027ms | 1.3086ms | 764.1618 Ops/s | 765.7356 Ops/s | $\color{#d91a1a}-0.21\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1288s | 7.5515ms | 132.4232 Ops/s | 98.5209 Ops/s | $\textbf{\color{#35bf28}+34.41\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.4703ms | 15.6137ms | 64.0463 Ops/s | 59.2586 Ops/s | $\textbf{\color{#35bf28}+8.08\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.6040ms | 1.3879ms | 720.5259 Ops/s | 726.7679 Ops/s | $\color{#d91a1a}-0.86\\%$ |