search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.01k stars 269 forks source link

[BugFix] Fix non-tensor passage in _StepMDP #2262

Closed vmoens closed 1 week ago

vmoens commented 1 week ago

follow up on #2260

pytorch-bot[bot] commented 1 week ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2262

Note: Links to docs will display an error until the docs builds have been completed.

:x: 1 New Failure, 16 Unrelated Failures

As of commit 3b5e059853e01e317db64a6abe534881f3941638 with merge base 39462f011d3fe2b7bba1e4c9acc9648b68bc6ff5 (image):

NEW FAILURE - The following job has failed:

* [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931283668) ([gh](https://github.com/pytorch/rl/actions/runs/9757897729/job/26931283668)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

* [Generate documentation / build-docs (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931284089) ([gh](https://github.com/pytorch/rl/actions/runs/9757897683/job/26931284089)) ([similar failure](https://hud.pytorch.org/pytorch/rl/commit/3b5e059853e01e317db64a6abe534881f3941638#26931285300)) * [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931284661) ([gh](https://github.com/pytorch/rl/actions/runs/9757897700/job/26931284661)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931290657) ([gh](https://github.com/pytorch/rl/actions/runs/9757897718/job/26931290657)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931290978) ([gh](https://github.com/pytorch/rl/actions/runs/9757897718/job/26931290978)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Lint / c-source / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931283550) ([gh](https://github.com/pytorch/rl/actions/runs/9757897687/job/26931283550)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Lint / python-source-and-configs / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931284494) ([gh](https://github.com/pytorch/rl/actions/runs/9757897687/job/26931284494)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931284130) ([gh](https://github.com/pytorch/rl/actions/runs/9757897704/job/26931284130)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.10) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931285007) ([gh](https://github.com/pytorch/rl/actions/runs/9757897710/job/26931285007)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.11) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931285460) ([gh](https://github.com/pytorch/rl/actions/runs/9757897710/job/26931285460)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.8) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931285982) ([gh](https://github.com/pytorch/rl/actions/runs/9757897710/job/26931285982)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.9) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931286688) ([gh](https://github.com/pytorch/rl/actions/runs/9757897710/job/26931286688)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-gpu (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931287183) ([gh](https://github.com/pytorch/rl/actions/runs/9757897710/job/26931287183)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931287432) ([gh](https://github.com/pytorch/rl/actions/runs/9757897710/job/26931287432)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931287712) ([gh](https://github.com/pytorch/rl/actions/runs/9757897710/job/26931287712)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931287975) ([gh](https://github.com/pytorch/rl/actions/runs/9757897710/job/26931287975)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128`

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Examples Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2262#26931285300) ([gh](https://github.com/pytorch/rl/actions/runs/9757897681/job/26931285300)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/39462f011d3fe2b7bba1e4c9acc9648b68bc6ff5#26862911662))

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 week ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1196s | 59.6861ms | 16.7543 Ops/s | 18.0719 Ops/s | $\textbf{\color{#d91a1a}-7.29\\%}$ | | test_sync | 41.8159ms | 31.7178ms | 31.5280 Ops/s | 32.5041 Ops/s | $\color{#d91a1a}-3.00\\%$ | | test_async | 53.9892ms | 28.5040ms | 35.0827 Ops/s | 34.3206 Ops/s | $\color{#35bf28}+2.22\\%$ | | test_simple | 0.3797s | 0.3781s | 2.6449 Ops/s | 2.6780 Ops/s | $\color{#d91a1a}-1.23\\%$ | | test_transformed | 0.5356s | 0.5320s | 1.8797 Ops/s | 1.8321 Ops/s | $\color{#35bf28}+2.60\\%$ | | test_serial | 1.3040s | 1.2470s | 0.8019 Ops/s | 0.7886 Ops/s | $\color{#35bf28}+1.70\\%$ | | test_parallel | 1.1364s | 1.0851s | 0.9216 Ops/s | 0.9232 Ops/s | $\color{#d91a1a}-0.17\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1578ms | 22.4761μs | 44.4918 KOps/s | 44.2217 KOps/s | $\color{#35bf28}+0.61\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 37.9110μs | 13.2989μs | 75.1940 KOps/s | 74.8077 KOps/s | $\color{#35bf28}+0.52\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 66.4640μs | 13.1779μs | 75.8845 KOps/s | 76.7808 KOps/s | $\color{#d91a1a}-1.17\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 34.2640μs | 7.6969μs | 129.9220 KOps/s | 128.7935 KOps/s | $\color{#35bf28}+0.88\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 57.0170μs | 23.7509μs | 42.1036 KOps/s | 41.2916 KOps/s | $\color{#35bf28}+1.97\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 36.1670μs | 14.5561μs | 68.6998 KOps/s | 68.6101 KOps/s | $\color{#35bf28}+0.13\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 49.3920μs | 14.3485μs | 69.6935 KOps/s | 68.9411 KOps/s | $\color{#35bf28}+1.09\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 30.0060μs | 9.0465μs | 110.5398 KOps/s | 111.3319 KOps/s | $\color{#d91a1a}-0.71\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 60.9030μs | 25.2439μs | 39.6135 KOps/s | 39.4146 KOps/s | $\color{#35bf28}+0.50\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 47.7690μs | 15.9044μs | 62.8756 KOps/s | 62.4605 KOps/s | $\color{#35bf28}+0.66\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 48.8910μs | 14.4150μs | 69.3721 KOps/s | 68.9517 KOps/s | $\color{#35bf28}+0.61\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 36.9290μs | 9.0061μs | 111.0353 KOps/s | 111.0626 KOps/s | $\color{#d91a1a}-0.02\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 52.1470μs | 26.4837μs | 37.7591 KOps/s | 37.4675 KOps/s | $\color{#35bf28}+0.78\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 67.0750μs | 17.1350μs | 58.3601 KOps/s | 57.9228 KOps/s | $\color{#35bf28}+0.75\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 51.3860μs | 15.4165μs | 64.8654 KOps/s | 63.7719 KOps/s | $\color{#35bf28}+1.71\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 30.5470μs | 10.1499μs | 98.5236 KOps/s | 97.7268 KOps/s | $\color{#35bf28}+0.82\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 63.3980μs | 25.3488μs | 39.4496 KOps/s | 39.5598 KOps/s | $\color{#d91a1a}-0.28\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 51.2960μs | 16.0193μs | 62.4249 KOps/s | 61.8310 KOps/s | $\color{#35bf28}+0.96\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 42.2680μs | 16.8020μs | 59.5167 KOps/s | 59.2579 KOps/s | $\color{#35bf28}+0.44\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 59.9500μs | 10.2339μs | 97.7148 KOps/s | 98.1033 KOps/s | $\color{#d91a1a}-0.40\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 56.2640μs | 26.3190μs | 37.9954 KOps/s | 37.4671 KOps/s | $\color{#35bf28}+1.41\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 52.1170μs | 17.0452μs | 58.6676 KOps/s | 58.2234 KOps/s | $\color{#35bf28}+0.76\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 39.4230μs | 17.9423μs | 55.7342 KOps/s | 55.9113 KOps/s | $\color{#d91a1a}-0.32\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 38.8630μs | 11.3983μs | 87.7325 KOps/s | 87.8085 KOps/s | $\color{#d91a1a}-0.09\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 52.7590μs | 27.6795μs | 36.1279 KOps/s | 35.8166 KOps/s | $\color{#35bf28}+0.87\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 44.1920μs | 18.4593μs | 54.1733 KOps/s | 53.8143 KOps/s | $\color{#35bf28}+0.67\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 46.6070μs | 17.9348μs | 55.7577 KOps/s | 55.7056 KOps/s | $\color{#35bf28}+0.09\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 38.3320μs | 11.3238μs | 88.3099 KOps/s | 86.5730 KOps/s | $\color{#35bf28}+2.01\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 40.1050μs | 29.0495μs | 34.4240 KOps/s | 33.9157 KOps/s | $\color{#35bf28}+1.50\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 44.6130μs | 19.5687μs | 51.1021 KOps/s | 50.5259 KOps/s | $\color{#35bf28}+1.14\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 41.3170μs | 18.7723μs | 53.2701 KOps/s | 53.0796 KOps/s | $\color{#35bf28}+0.36\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 40.5360μs | 12.4646μs | 80.2273 KOps/s | 79.7348 KOps/s | $\color{#35bf28}+0.62\\%$ | | test_values[generalized_advantage_estimate-True-True] | 11.1917ms | 9.7697ms | 102.3576 Ops/s | 103.3322 Ops/s | $\color{#d91a1a}-0.94\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 37.9192ms | 35.2319ms | 28.3834 Ops/s | 27.9297 Ops/s | $\color{#35bf28}+1.62\\%$ | | test_values[td0_return_estimate-False-False] | 0.2396ms | 0.1664ms | 6.0108 KOps/s | 6.0570 KOps/s | $\color{#d91a1a}-0.76\\%$ | | test_values[td1_return_estimate-False-False] | 25.8330ms | 24.1286ms | 41.4446 Ops/s | 42.6380 Ops/s | $\color{#d91a1a}-2.80\\%$ | | test_values[vec_td1_return_estimate-False-False] | 38.3449ms | 35.4434ms | 28.2140 Ops/s | 28.2804 Ops/s | $\color{#d91a1a}-0.23\\%$ | | test_values[td_lambda_return_estimate-True-False] | 35.8490ms | 34.6608ms | 28.8510 Ops/s | 29.4852 Ops/s | $\color{#d91a1a}-2.15\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 52.3876ms | 36.0314ms | 27.7536 Ops/s | 28.2273 Ops/s | $\color{#d91a1a}-1.68\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 10.8839ms | 8.5162ms | 117.4237 Ops/s | 117.8969 Ops/s | $\color{#d91a1a}-0.40\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 3.1340ms | 2.2666ms | 441.1832 Ops/s | 491.8495 Ops/s | $\textbf{\color{#d91a1a}-10.30\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.5704ms | 0.3560ms | 2.8089 KOps/s | 2.7787 KOps/s | $\color{#35bf28}+1.09\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 47.1351ms | 46.4985ms | 21.5061 Ops/s | 21.4184 Ops/s | $\color{#35bf28}+0.41\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.4149ms | 3.0305ms | 329.9732 Ops/s | 328.5353 Ops/s | $\color{#35bf28}+0.44\\%$ | | test_dqn_speed | 1.7319ms | 1.3347ms | 749.2226 Ops/s | 743.3229 Ops/s | $\color{#35bf28}+0.79\\%$ | | test_ddpg_speed | 3.6098ms | 2.8286ms | 353.5268 Ops/s | 352.6461 Ops/s | $\color{#35bf28}+0.25\\%$ | | test_sac_speed | 10.0331ms | 8.4486ms | 118.3623 Ops/s | 117.6941 Ops/s | $\color{#35bf28}+0.57\\%$ | | test_redq_speed | 15.3618ms | 13.6091ms | 73.4801 Ops/s | 72.8697 Ops/s | $\color{#35bf28}+0.84\\%$ | | test_redq_deprec_speed | 15.1748ms | 13.3338ms | 74.9974 Ops/s | 67.8762 Ops/s | $\textbf{\color{#35bf28}+10.49\\%}$ | | test_td3_speed | 17.3419ms | 8.4618ms | 118.1783 Ops/s | 118.2437 Ops/s | $\color{#d91a1a}-0.06\\%$ | | test_cql_speed | 38.6158ms | 37.1390ms | 26.9259 Ops/s | 26.8793 Ops/s | $\color{#35bf28}+0.17\\%$ | | test_a2c_speed | 8.2000ms | 7.5119ms | 133.1227 Ops/s | 133.6776 Ops/s | $\color{#d91a1a}-0.42\\%$ | | test_ppo_speed | 8.8276ms | 7.7834ms | 128.4782 Ops/s | 128.4907 Ops/s | $-0.01\\%$ | | test_reinforce_speed | 7.3323ms | 6.8118ms | 146.8031 Ops/s | 148.8907 Ops/s | $\color{#d91a1a}-1.40\\%$ | | test_iql_speed | 35.0114ms | 33.3984ms | 29.9416 Ops/s | 30.0722 Ops/s | $\color{#d91a1a}-0.43\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.5055ms | 3.5390ms | 282.5664 Ops/s | 278.1836 Ops/s | $\color{#35bf28}+1.58\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.0102ms | 0.5143ms | 1.9443 KOps/s | 1.7786 KOps/s | $\textbf{\color{#35bf28}+9.31\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.8089ms | 0.4703ms | 2.1264 KOps/s | 2.1131 KOps/s | $\color{#35bf28}+0.63\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.8413ms | 3.4780ms | 287.5202 Ops/s | 284.0612 Ops/s | $\color{#35bf28}+1.22\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.0987ms | 0.4905ms | 2.0389 KOps/s | 2.0513 KOps/s | $\color{#d91a1a}-0.60\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7440ms | 0.4706ms | 2.1251 KOps/s | 2.1207 KOps/s | $\color{#35bf28}+0.21\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.0312ms | 1.7209ms | 581.0915 Ops/s | 579.1739 Ops/s | $\color{#35bf28}+0.33\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 4.7091ms | 1.6436ms | 608.4303 Ops/s | 612.8945 Ops/s | $\color{#d91a1a}-0.73\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.5814ms | 3.6851ms | 271.3625 Ops/s | 273.8588 Ops/s | $\color{#d91a1a}-0.91\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9806ms | 0.6332ms | 1.5792 KOps/s | 1.5869 KOps/s | $\color{#d91a1a}-0.49\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9810ms | 0.6071ms | 1.6472 KOps/s | 1.6568 KOps/s | $\color{#d91a1a}-0.58\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 7.8361ms | 3.6495ms | 274.0073 Ops/s | 281.6982 Ops/s | $\color{#d91a1a}-2.73\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.5979ms | 0.4959ms | 2.0165 KOps/s | 1.9958 KOps/s | $\color{#35bf28}+1.04\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7567ms | 0.4764ms | 2.0992 KOps/s | 2.1142 KOps/s | $\color{#d91a1a}-0.71\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.7923ms | 3.5134ms | 284.6273 Ops/s | 281.5313 Ops/s | $\color{#35bf28}+1.10\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.1393ms | 0.4918ms | 2.0335 KOps/s | 2.0439 KOps/s | $\color{#d91a1a}-0.50\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.8376ms | 0.4746ms | 2.1072 KOps/s | 2.1035 KOps/s | $\color{#35bf28}+0.18\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.0686ms | 3.7145ms | 269.2187 Ops/s | 274.5295 Ops/s | $\color{#d91a1a}-1.93\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.7489ms | 0.6332ms | 1.5793 KOps/s | 1.5772 KOps/s | $\color{#35bf28}+0.13\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 3.8330ms | 0.6129ms | 1.6315 KOps/s | 1.6418 KOps/s | $\color{#d91a1a}-0.63\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1283s | 6.3449ms | 157.6075 Ops/s | 123.2663 Ops/s | $\textbf{\color{#35bf28}+27.86\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 14.9133ms | 12.6013ms | 79.3568 Ops/s | 80.5324 Ops/s | $\color{#d91a1a}-1.46\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 1.6758ms | 1.1015ms | 907.8508 Ops/s | 871.2174 Ops/s | $\color{#35bf28}+4.20\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1171s | 8.2730ms | 120.8748 Ops/s | 170.9686 Ops/s | $\textbf{\color{#d91a1a}-29.30\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 15.2054ms | 12.6837ms | 78.8416 Ops/s | 80.1092 Ops/s | $\color{#d91a1a}-1.58\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.7723ms | 1.0651ms | 938.8720 Ops/s | 902.6991 Ops/s | $\color{#35bf28}+4.01\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1241s | 6.3186ms | 158.2623 Ops/s | 164.8073 Ops/s | $\color{#d91a1a}-3.97\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 20.3105ms | 12.9560ms | 77.1844 Ops/s | 78.8988 Ops/s | $\color{#d91a1a}-2.17\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.8741ms | 1.2213ms | 818.7921 Ops/s | 814.9078 Ops/s | $\color{#35bf28}+0.48\\%$ |
github-actions[bot] commented 1 week ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1624s | 0.1096s | 9.1230 Ops/s | 9.6847 Ops/s | $\textbf{\color{#d91a1a}-5.80\\%}$ | | test_sync | 91.7233ms | 88.3603ms | 11.3173 Ops/s | 11.2766 Ops/s | $\color{#35bf28}+0.36\\%$ | | test_async | 0.1711s | 85.5422ms | 11.6901 Ops/s | 11.5519 Ops/s | $\color{#35bf28}+1.20\\%$ | | test_single_pixels | 0.1139s | 0.1137s | 8.7958 Ops/s | 8.7441 Ops/s | $\color{#35bf28}+0.59\\%$ | | test_sync_pixels | 69.9243ms | 68.3983ms | 14.6202 Ops/s | 14.6760 Ops/s | $\color{#d91a1a}-0.38\\%$ | | test_async_pixels | 0.1197s | 56.3438ms | 17.7482 Ops/s | 17.4925 Ops/s | $\color{#35bf28}+1.46\\%$ | | test_simple | 0.7273s | 0.7267s | 1.3761 Ops/s | 1.3457 Ops/s | $\color{#35bf28}+2.26\\%$ | | test_transformed | 1.0576s | 0.9919s | 1.0081 Ops/s | 1.0220 Ops/s | $\color{#d91a1a}-1.36\\%$ | | test_serial | 2.2706s | 2.2022s | 0.4541 Ops/s | 0.4607 Ops/s | $\color{#d91a1a}-1.43\\%$ | | test_parallel | 1.9455s | 1.8646s | 0.5363 Ops/s | 0.5390 Ops/s | $\color{#d91a1a}-0.50\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 69.1110μs | 34.1463μs | 29.2858 KOps/s | 29.5996 KOps/s | $\color{#d91a1a}-1.06\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 49.8410μs | 19.8272μs | 50.4358 KOps/s | 51.7770 KOps/s | $\color{#d91a1a}-2.59\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 51.5210μs | 19.6863μs | 50.7967 KOps/s | 53.0211 KOps/s | $\color{#d91a1a}-4.20\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 34.7310μs | 11.3498μs | 88.1072 KOps/s | 90.8226 KOps/s | $\color{#d91a1a}-2.99\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 62.0510μs | 36.8445μs | 27.1411 KOps/s | 28.8085 KOps/s | $\textbf{\color{#d91a1a}-5.79\\%}$ | | test_step_mdp_speed[True-True-False-True-False] | 50.0210μs | 21.6528μs | 46.1834 KOps/s | 47.7539 KOps/s | $\color{#d91a1a}-3.29\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 39.3510μs | 21.5402μs | 46.4249 KOps/s | 48.8554 KOps/s | $\color{#d91a1a}-4.97\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 38.3710μs | 13.0772μs | 76.4687 KOps/s | 78.2276 KOps/s | $\color{#d91a1a}-2.25\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 56.4610μs | 38.4514μs | 26.0069 KOps/s | 27.0339 KOps/s | $\color{#d91a1a}-3.80\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 57.9710μs | 23.5307μs | 42.4978 KOps/s | 43.7952 KOps/s | $\color{#d91a1a}-2.96\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 47.6210μs | 21.6185μs | 46.2567 KOps/s | 48.8715 KOps/s | $\textbf{\color{#d91a1a}-5.35\\%}$ | | test_step_mdp_speed[True-False-True-False-False] | 29.5100μs | 13.1373μs | 76.1189 KOps/s | 78.6494 KOps/s | $\color{#d91a1a}-3.22\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 57.7610μs | 40.2770μs | 24.8280 KOps/s | 26.2869 KOps/s | $\textbf{\color{#d91a1a}-5.55\\%}$ | | test_step_mdp_speed[True-False-False-True-False] | 41.4100μs | 25.3574μs | 39.4363 KOps/s | 40.4949 KOps/s | $\color{#d91a1a}-2.61\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 73.4620μs | 23.4015μs | 42.7323 KOps/s | 45.6288 KOps/s | $\textbf{\color{#d91a1a}-6.35\\%}$ | | test_step_mdp_speed[True-False-False-False-False] | 68.9710μs | 14.9376μs | 66.9454 KOps/s | 69.7740 KOps/s | $\color{#d91a1a}-4.05\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 66.0910μs | 38.5473μs | 25.9421 KOps/s | 27.4342 KOps/s | $\textbf{\color{#d91a1a}-5.44\\%}$ | | test_step_mdp_speed[False-True-True-True-False] | 40.6700μs | 23.7341μs | 42.1335 KOps/s | 43.7114 KOps/s | $\color{#d91a1a}-3.61\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 43.3910μs | 25.2312μs | 39.6335 KOps/s | 41.2033 KOps/s | $\color{#d91a1a}-3.81\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 31.9100μs | 14.8651μs | 67.2718 KOps/s | 69.4411 KOps/s | $\color{#d91a1a}-3.12\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 67.7210μs | 40.2466μs | 24.8468 KOps/s | 26.2818 KOps/s | $\textbf{\color{#d91a1a}-5.46\\%}$ | | test_step_mdp_speed[False-True-False-True-False] | 49.1210μs | 25.2697μs | 39.5731 KOps/s | 40.8416 KOps/s | $\color{#d91a1a}-3.11\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 45.7110μs | 27.2412μs | 36.7090 KOps/s | 38.3898 KOps/s | $\color{#d91a1a}-4.38\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 33.2500μs | 16.7161μs | 59.8227 KOps/s | 61.7816 KOps/s | $\color{#d91a1a}-3.17\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 0.2006ms | 42.1587μs | 23.7199 KOps/s | 25.2139 KOps/s | $\textbf{\color{#d91a1a}-5.93\\%}$ | | test_step_mdp_speed[False-False-True-True-False] | 51.3200μs | 27.7756μs | 36.0028 KOps/s | 37.8600 KOps/s | $\color{#d91a1a}-4.91\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 51.9610μs | 27.4632μs | 36.4124 KOps/s | 38.7346 KOps/s | $\textbf{\color{#d91a1a}-6.00\\%}$ | | test_step_mdp_speed[False-False-True-False-False] | 33.2710μs | 16.7247μs | 59.7918 KOps/s | 62.0086 KOps/s | $\color{#d91a1a}-3.58\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 77.4410μs | 44.3146μs | 22.5660 KOps/s | 23.4030 KOps/s | $\color{#d91a1a}-3.58\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 52.4710μs | 29.5527μs | 33.8379 KOps/s | 35.2299 KOps/s | $\color{#d91a1a}-3.95\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 60.2610μs | 28.5639μs | 35.0093 KOps/s | 36.2355 KOps/s | $\color{#d91a1a}-3.38\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 42.4910μs | 18.5817μs | 53.8164 KOps/s | 55.4904 KOps/s | $\color{#d91a1a}-3.02\\%$ | | test_values[generalized_advantage_estimate-True-True] | 24.3059ms | 23.8802ms | 41.8756 Ops/s | 41.4200 Ops/s | $\color{#35bf28}+1.10\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 90.1319ms | 2.7012ms | 370.2061 Ops/s | 375.6475 Ops/s | $\color{#d91a1a}-1.45\\%$ | | test_values[td0_return_estimate-False-False] | 91.3420μs | 65.9864μs | 15.1546 KOps/s | 15.0087 KOps/s | $\color{#35bf28}+0.97\\%$ | | test_values[td1_return_estimate-False-False] | 54.0447ms | 53.7496ms | 18.6048 Ops/s | 18.4992 Ops/s | $\color{#35bf28}+0.57\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.3559ms | 1.0822ms | 924.0116 Ops/s | 925.4512 Ops/s | $\color{#d91a1a}-0.16\\%$ | | test_values[td_lambda_return_estimate-True-False] | 91.8395ms | 86.2820ms | 11.5899 Ops/s | 11.6204 Ops/s | $\color{#d91a1a}-0.26\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.4188ms | 1.0827ms | 923.6467 Ops/s | 926.7681 Ops/s | $\color{#d91a1a}-0.34\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 26.1639ms | 25.3757ms | 39.4077 Ops/s | 40.9831 Ops/s | $\color{#d91a1a}-3.84\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9545ms | 0.7116ms | 1.4052 KOps/s | 1.4074 KOps/s | $\color{#d91a1a}-0.16\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7445ms | 0.6646ms | 1.5047 KOps/s | 1.5147 KOps/s | $\color{#d91a1a}-0.66\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.4954ms | 1.4626ms | 683.6936 Ops/s | 684.6861 Ops/s | $\color{#d91a1a}-0.14\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7256ms | 0.6793ms | 1.4722 KOps/s | 1.4823 KOps/s | $\color{#d91a1a}-0.68\\%$ | | test_dqn_speed | 7.7477ms | 1.4629ms | 683.5629 Ops/s | 691.7536 Ops/s | $\color{#d91a1a}-1.18\\%$ | | test_ddpg_speed | 3.1787ms | 2.9413ms | 339.9857 Ops/s | 343.0522 Ops/s | $\color{#d91a1a}-0.89\\%$ | | test_sac_speed | 9.2849ms | 8.4116ms | 118.8839 Ops/s | 119.4424 Ops/s | $\color{#d91a1a}-0.47\\%$ | | test_redq_speed | 0.1036s | 11.6076ms | 86.1504 Ops/s | 94.4188 Ops/s | $\textbf{\color{#d91a1a}-8.76\\%}$ | | test_redq_deprec_speed | 12.0607ms | 11.3994ms | 87.7241 Ops/s | 79.0638 Ops/s | $\textbf{\color{#35bf28}+10.95\\%}$ | | test_td3_speed | 8.4700ms | 8.3456ms | 119.8241 Ops/s | 119.8804 Ops/s | $\color{#d91a1a}-0.05\\%$ | | test_cql_speed | 26.6621ms | 25.6975ms | 38.9143 Ops/s | 39.0957 Ops/s | $\color{#d91a1a}-0.46\\%$ | | test_a2c_speed | 6.4382ms | 5.6280ms | 177.6840 Ops/s | 177.6682 Ops/s | $+0.01\\%$ | | test_ppo_speed | 6.5367ms | 5.9335ms | 168.5349 Ops/s | 166.7690 Ops/s | $\color{#35bf28}+1.06\\%$ | | test_reinforce_speed | 4.8022ms | 4.6298ms | 215.9936 Ops/s | 215.1430 Ops/s | $\color{#35bf28}+0.40\\%$ | | test_iql_speed | 20.2516ms | 19.4168ms | 51.5019 Ops/s | 51.1620 Ops/s | $\color{#35bf28}+0.66\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.8277ms | 4.6691ms | 214.1747 Ops/s | 216.4766 Ops/s | $\color{#d91a1a}-1.06\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.1082s | 0.6175ms | 1.6193 KOps/s | 1.8622 KOps/s | $\textbf{\color{#d91a1a}-13.04\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7853ms | 0.5153ms | 1.9406 KOps/s | 1.9437 KOps/s | $\color{#d91a1a}-0.16\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.7920ms | 4.6202ms | 216.4392 Ops/s | 215.9365 Ops/s | $\color{#35bf28}+0.23\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6646ms | 0.5333ms | 1.8753 KOps/s | 1.8689 KOps/s | $\color{#35bf28}+0.34\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 4.6924ms | 0.5156ms | 1.9395 KOps/s | 1.9521 KOps/s | $\color{#d91a1a}-0.65\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.1390ms | 1.9276ms | 518.7723 Ops/s | 513.2364 Ops/s | $\color{#35bf28}+1.08\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 5.9279ms | 1.8450ms | 542.0189 Ops/s | 544.0627 Ops/s | $\color{#d91a1a}-0.38\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.9605ms | 4.8065ms | 208.0533 Ops/s | 209.7619 Ops/s | $\color{#d91a1a}-0.81\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.7237ms | 0.6862ms | 1.4573 KOps/s | 1.4733 KOps/s | $\color{#d91a1a}-1.09\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8397ms | 0.6650ms | 1.5037 KOps/s | 1.5191 KOps/s | $\color{#d91a1a}-1.02\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.8611ms | 4.6707ms | 214.0984 Ops/s | 215.8393 Ops/s | $\color{#d91a1a}-0.81\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.6222ms | 0.5381ms | 1.8584 KOps/s | 1.8544 KOps/s | $\color{#35bf28}+0.21\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6772ms | 0.5197ms | 1.9243 KOps/s | 1.9130 KOps/s | $\color{#35bf28}+0.59\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.8192ms | 4.6176ms | 216.5628 Ops/s | 217.0998 Ops/s | $\color{#d91a1a}-0.25\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6613ms | 0.5319ms | 1.8801 KOps/s | 1.8569 KOps/s | $\color{#35bf28}+1.25\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 5.0933ms | 0.5198ms | 1.9238 KOps/s | 1.9435 KOps/s | $\color{#d91a1a}-1.02\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.9655ms | 4.8449ms | 206.4014 Ops/s | 207.8397 Ops/s | $\color{#d91a1a}-0.69\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8009ms | 0.6876ms | 1.4543 KOps/s | 1.4652 KOps/s | $\color{#d91a1a}-0.75\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8028ms | 0.6636ms | 1.5070 KOps/s | 1.4922 KOps/s | $\color{#35bf28}+0.99\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1422s | 7.5461ms | 132.5189 Ops/s | 138.9337 Ops/s | $\color{#d91a1a}-4.62\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.0998ms | 15.2750ms | 65.4664 Ops/s | 64.9335 Ops/s | $\color{#35bf28}+0.82\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.4254ms | 1.1336ms | 882.1151 Ops/s | 880.0233 Ops/s | $\color{#35bf28}+0.24\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1257s | 9.6057ms | 104.1053 Ops/s | 137.5298 Ops/s | $\textbf{\color{#d91a1a}-24.30\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 17.8301ms | 15.4354ms | 64.7860 Ops/s | 56.5203 Ops/s | $\textbf{\color{#35bf28}+14.62\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.2278ms | 1.1179ms | 894.4955 Ops/s | 862.6461 Ops/s | $\color{#35bf28}+3.69\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1250s | 7.3087ms | 136.8228 Ops/s | 134.0514 Ops/s | $\color{#35bf28}+2.07\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 17.9781ms | 15.5451ms | 64.3289 Ops/s | 64.9573 Ops/s | $\color{#d91a1a}-0.97\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 7.4294ms | 1.3852ms | 721.9116 Ops/s | 747.4698 Ops/s | $\color{#d91a1a}-3.42\\%$ |