[Doc] Fix algorithms references in tutos

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2320

• :page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

:hourglass_flowing_sand: 8 Pending, 17 Unrelated Failures

As of commit 65ac1398aee4579f4151bf17cdbab8cb2927e882 with merge base 1ca33a2296a9e4f53db6b8ce855621a2eda5b246 ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

* [Examples Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879539681) ([gh](https://github.com/pytorch/rl/actions/runs/10083286216/job/27879539681)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Generate documentation / build-docs (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879540774) ([gh](https://github.com/pytorch/rl/actions/runs/10083286213/job/27879540774)) ([similar failure](https://hud.pytorch.org/pytorch/rl/commit/65ac1398aee4579f4151bf17cdbab8cb2927e882#27879517959)) * [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879539687) ([gh](https://github.com/pytorch/rl/actions/runs/10083286214/job/27879539687)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879555006) ([gh](https://github.com/pytorch/rl/actions/runs/10083286217/job/27879555006)) (detected as infra flaky with no log or failing log classifier) * [Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879555401) ([gh](https://github.com/pytorch/rl/actions/runs/10083286217/job/27879555401)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Lint / c-source / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879539677) ([gh](https://github.com/pytorch/rl/actions/runs/10083286225/job/27879539677)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Lint / python-source-and-configs / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879540794) ([gh](https://github.com/pytorch/rl/actions/runs/10083286225/job/27879540794)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879539723) ([gh](https://github.com/pytorch/rl/actions/runs/10083286220/job/27879539723)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.10) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879540788) ([gh](https://github.com/pytorch/rl/actions/runs/10083286231/job/27879540788)) (detected as infra flaky with no log or failing log classifier) * [Unit-tests on Linux / tests-cpu (3.11) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879541602) ([gh](https://github.com/pytorch/rl/actions/runs/10083286231/job/27879541602)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.8) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879542140) ([gh](https://github.com/pytorch/rl/actions/runs/10083286231/job/27879542140)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.9) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879542834) ([gh](https://github.com/pytorch/rl/actions/runs/10083286231/job/27879542834)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-gpu (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879544383) ([gh](https://github.com/pytorch/rl/actions/runs/10083286231/job/27879544383)) (detected as infra flaky with no log or failing log classifier) * [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879545265) ([gh](https://github.com/pytorch/rl/actions/runs/10083286231/job/27879545265)) (detected as infra flaky with no log or failing log classifier) * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879545985) ([gh](https://github.com/pytorch/rl/actions/runs/10083286231/job/27879545985)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879547125) ([gh](https://github.com/pytorch/rl/actions/runs/10083286231/job/27879547125)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2320#27879517959) ([gh](https://github.com/pytorch/rl/actions/runs/10083286227/job/27879517959)) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions[bot]]

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results

| Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1094s | 0.1071s | 9.3385 Ops/s | 9.2130 Ops/s | $\color{#35bf28}+1.36\\%$ | | test_sync | 95.6874ms | 95.0289ms | 10.5231 Ops/s | 10.5319 Ops/s | $\color{#d91a1a}-0.08\\%$ | | test_async | 0.1874s | 88.8068ms | 11.2604 Ops/s | 11.1975 Ops/s | $\color{#35bf28}+0.56\\%$ | | test_single_pixels | 0.1206s | 0.1177s | 8.4963 Ops/s | 8.4006 Ops/s | $\color{#35bf28}+1.14\\%$ | | test_sync_pixels | 76.1591ms | 73.1108ms | 13.6779 Ops/s | 13.4582 Ops/s | $\color{#35bf28}+1.63\\%$ | | test_async_pixels | 0.1383s | 68.5487ms | 14.5882 Ops/s | 13.9836 Ops/s | $\color{#35bf28}+4.32\\%$ | | test_simple | 0.8568s | 0.7792s | 1.2834 Ops/s | 1.2496 Ops/s | $\color{#35bf28}+2.70\\%$ | | test_transformed | 1.0925s | 1.0172s | 0.9831 Ops/s | 0.9773 Ops/s | $\color{#35bf28}+0.60\\%$ | | test_serial | 2.2929s | 2.2386s | 0.4467 Ops/s | 0.4357 Ops/s | $\color{#35bf28}+2.53\\%$ | | test_parallel | 1.9410s | 1.9267s | 0.5190 Ops/s | 0.5070 Ops/s | $\color{#35bf28}+2.38\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.2196ms | 37.0772μs | 26.9708 KOps/s | 26.3486 KOps/s | $\color{#35bf28}+2.36\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 0.1463ms | 20.9833μs | 47.6570 KOps/s | 46.7228 KOps/s | $\color{#35bf28}+2.00\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 0.1392ms | 20.9158μs | 47.8107 KOps/s | 46.3624 KOps/s | $\color{#35bf28}+3.12\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 0.1332ms | 12.0152μs | 83.2282 KOps/s | 81.5592 KOps/s | $\color{#35bf28}+2.05\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 0.1312ms | 39.6358μs | 25.2297 KOps/s | 24.6872 KOps/s | $\color{#35bf28}+2.20\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 42.7910μs | 23.2281μs | 43.0514 KOps/s | 42.0390 KOps/s | $\color{#35bf28}+2.41\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 0.1118ms | 23.2421μs | 43.0254 KOps/s | 41.1160 KOps/s | $\color{#35bf28}+4.64\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 0.1006ms | 14.3681μs | 69.5984 KOps/s | 69.1657 KOps/s | $\color{#35bf28}+0.63\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 71.5010μs | 41.7214μs | 23.9685 KOps/s | 23.4676 KOps/s | $\color{#35bf28}+2.13\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 0.1721ms | 25.8657μs | 38.6612 KOps/s | 38.2273 KOps/s | $\color{#35bf28}+1.14\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 59.0510μs | 23.1987μs | 43.1058 KOps/s | 41.4913 KOps/s | $\color{#35bf28}+3.89\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 44.4010μs | 14.3289μs | 69.7891 KOps/s | 68.9995 KOps/s | $\color{#35bf28}+1.14\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 91.4510μs | 43.9683μs | 22.7437 KOps/s | 22.1959 KOps/s | $\color{#35bf28}+2.47\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 46.0300μs | 27.5342μs | 36.3184 KOps/s | 35.1438 KOps/s | $\color{#35bf28}+3.34\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 0.1934ms | 25.0009μs | 39.9985 KOps/s | 37.9881 KOps/s | $\textbf{\color{#35bf28}+5.29\\%}$ | | test_step_mdp_speed[True-False-False-False-False] | 0.1848ms | 16.4276μs | 60.8732 KOps/s | 59.7257 KOps/s | $\color{#35bf28}+1.92\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 0.2369ms | 41.4839μs | 24.1058 KOps/s | 23.4723 KOps/s | $\color{#35bf28}+2.70\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 0.1243ms | 25.6553μs | 38.9783 KOps/s | 38.2975 KOps/s | $\color{#35bf28}+1.78\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 0.2017ms | 27.1432μs | 36.8417 KOps/s | 35.7457 KOps/s | $\color{#35bf28}+3.07\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 0.2123ms | 16.3551μs | 61.1429 KOps/s | 60.6595 KOps/s | $\color{#35bf28}+0.80\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 0.1383ms | 43.4215μs | 23.0300 KOps/s | 22.0968 KOps/s | $\color{#35bf28}+4.22\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 48.7000μs | 27.7120μs | 36.0855 KOps/s | 34.8463 KOps/s | $\color{#35bf28}+3.56\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 52.1810μs | 28.9064μs | 34.5944 KOps/s | 32.4267 KOps/s | $\textbf{\color{#35bf28}+6.68\\%}$ | | test_step_mdp_speed[False-True-False-False-False] | 50.0420μs | 18.3098μs | 54.6156 KOps/s | 52.1658 KOps/s | $\color{#35bf28}+4.70\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 4.4854ms | 46.3755μs | 21.5631 KOps/s | 20.7014 KOps/s | $\color{#35bf28}+4.16\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 0.1181ms | 30.3762μs | 32.9205 KOps/s | 32.2925 KOps/s | $\color{#35bf28}+1.94\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 61.3410μs | 29.7733μs | 33.5871 KOps/s | 32.6959 KOps/s | $\color{#35bf28}+2.73\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 36.1700μs | 18.2766μs | 54.7147 KOps/s | 52.3511 KOps/s | $\color{#35bf28}+4.51\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 67.4310μs | 47.4523μs | 21.0738 KOps/s | 19.9428 KOps/s | $\textbf{\color{#35bf28}+5.67\\%}$ | | test_step_mdp_speed[False-False-False-True-False] | 0.1225ms | 31.7341μs | 31.5119 KOps/s | 30.3912 KOps/s | $\color{#35bf28}+3.69\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 51.0010μs | 30.8242μs | 32.4421 KOps/s | 30.5506 KOps/s | $\textbf{\color{#35bf28}+6.19\\%}$ | | test_step_mdp_speed[False-False-False-False-False] | 39.5410μs | 20.1092μs | 49.7286 KOps/s | 48.0513 KOps/s | $\color{#35bf28}+3.49\\%$ | | test_values[generalized_advantage_estimate-True-True] | 25.1488ms | 24.1698ms | 41.3740 Ops/s | 41.0136 Ops/s | $\color{#35bf28}+0.88\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 89.5390ms | 2.6744ms | 373.9159 Ops/s | 369.8658 Ops/s | $\color{#35bf28}+1.10\\%$ | | test_values[td0_return_estimate-False-False] | 90.6510μs | 64.7128μs | 15.4529 KOps/s | 15.4643 KOps/s | $\color{#d91a1a}-0.07\\%$ | | test_values[td1_return_estimate-False-False] | 56.4953ms | 55.5581ms | 17.9992 Ops/s | 18.4682 Ops/s | $\color{#d91a1a}-2.54\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.3831ms | 1.0873ms | 919.6737 Ops/s | 928.9589 Ops/s | $\color{#d91a1a}-1.00\\%$ | | test_values[td_lambda_return_estimate-True-False] | 88.8665ms | 87.9837ms | 11.3657 Ops/s | 11.3554 Ops/s | $\color{#35bf28}+0.09\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.4003ms | 1.0876ms | 919.4964 Ops/s | 932.7095 Ops/s | $\color{#d91a1a}-1.42\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 24.9147ms | 23.9605ms | 41.7354 Ops/s | 40.7883 Ops/s | $\color{#35bf28}+2.32\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9579ms | 0.7087ms | 1.4111 KOps/s | 1.4081 KOps/s | $\color{#35bf28}+0.21\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.8275ms | 0.6651ms | 1.5035 KOps/s | 1.5164 KOps/s | $\color{#d91a1a}-0.85\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.6303ms | 1.4580ms | 685.8489 Ops/s | 682.9355 Ops/s | $\color{#35bf28}+0.43\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.8567ms | 0.6817ms | 1.4669 KOps/s | 1.4863 KOps/s | $\color{#d91a1a}-1.30\\%$ | | test_dqn_speed | 7.5800ms | 1.4919ms | 670.2976 Ops/s | 649.9570 Ops/s | $\color{#35bf28}+3.13\\%$ | | test_ddpg_speed | 3.3484ms | 3.0358ms | 329.4048 Ops/s | 329.8935 Ops/s | $\color{#d91a1a}-0.15\\%$ | | test_sac_speed | 9.1939ms | 8.5812ms | 116.5338 Ops/s | 117.5373 Ops/s | $\color{#d91a1a}-0.85\\%$ | | test_redq_speed | 12.0505ms | 11.1897ms | 89.3682 Ops/s | 89.4774 Ops/s | $\color{#d91a1a}-0.12\\%$ | | test_redq_deprec_speed | 12.9115ms | 11.9219ms | 83.8793 Ops/s | 87.1367 Ops/s | $\color{#d91a1a}-3.74\\%$ | | test_td3_speed | 8.6391ms | 8.3940ms | 119.1322 Ops/s | 117.3601 Ops/s | $\color{#35bf28}+1.51\\%$ | | test_cql_speed | 27.3549ms | 26.4655ms | 37.7850 Ops/s | 33.4605 Ops/s | $\textbf{\color{#35bf28}+12.92\\%}$ | | test_a2c_speed | 6.5043ms | 5.8779ms | 170.1293 Ops/s | 170.9295 Ops/s | $\color{#d91a1a}-0.47\\%$ | | test_ppo_speed | 6.9147ms | 6.2111ms | 161.0025 Ops/s | 161.9000 Ops/s | $\color{#d91a1a}-0.55\\%$ | | test_reinforce_speed | 5.1050ms | 4.8202ms | 207.4584 Ops/s | 212.1479 Ops/s | $\color{#d91a1a}-2.21\\%$ | | test_iql_speed | 21.0641ms | 20.1997ms | 49.5056 Ops/s | 48.6852 Ops/s | $\color{#35bf28}+1.69\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.8449ms | 6.6329ms | 150.7647 Ops/s | 147.7673 Ops/s | $\color{#35bf28}+2.03\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.6466ms | 0.5198ms | 1.9239 KOps/s | 1.8942 KOps/s | $\color{#35bf28}+1.57\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6869ms | 0.4970ms | 2.0121 KOps/s | 1.9728 KOps/s | $\color{#35bf28}+1.99\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.9906ms | 6.6457ms | 150.4728 Ops/s | 148.6777 Ops/s | $\color{#35bf28}+1.21\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7972ms | 0.5123ms | 1.9518 KOps/s | 1.9021 KOps/s | $\color{#35bf28}+2.61\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7037ms | 0.4918ms | 2.0336 KOps/s | 1.9942 KOps/s | $\color{#35bf28}+1.97\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 3.1230ms | 2.0006ms | 499.8571 Ops/s | 490.6818 Ops/s | $\color{#35bf28}+1.87\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2147ms | 1.9265ms | 519.0749 Ops/s | 515.2807 Ops/s | $\color{#35bf28}+0.74\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 7.0257ms | 6.8335ms | 146.3377 Ops/s | 144.4291 Ops/s | $\color{#35bf28}+1.32\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.1534s | 0.7964ms | 1.2557 KOps/s | 1.4755 KOps/s | $\textbf{\color{#d91a1a}-14.90\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8393ms | 0.6504ms | 1.5375 KOps/s | 1.5204 KOps/s | $\color{#35bf28}+1.13\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.9968ms | 6.6938ms | 149.3920 Ops/s | 147.1533 Ops/s | $\color{#35bf28}+1.52\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.6066ms | 0.5199ms | 1.9234 KOps/s | 1.8990 KOps/s | $\color{#35bf28}+1.28\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6675ms | 0.5003ms | 1.9988 KOps/s | 1.9656 KOps/s | $\color{#35bf28}+1.69\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 7.0623ms | 6.6590ms | 150.1730 Ops/s | 148.5450 Ops/s | $\color{#35bf28}+1.10\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7146ms | 0.5175ms | 1.9325 KOps/s | 1.9158 KOps/s | $\color{#35bf28}+0.87\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 4.5248ms | 0.4986ms | 2.0056 KOps/s | 1.9991 KOps/s | $\color{#35bf28}+0.32\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 7.0067ms | 6.8454ms | 146.0830 Ops/s | 144.0346 Ops/s | $\color{#35bf28}+1.42\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.1659ms | 0.6724ms | 1.4873 KOps/s | 1.4645 KOps/s | $\color{#35bf28}+1.56\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8204ms | 0.6507ms | 1.5369 KOps/s | 1.1839 KOps/s | $\textbf{\color{#35bf28}+29.82\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1546s | 8.1634ms | 122.4976 Ops/s | 127.4991 Ops/s | $\color{#d91a1a}-3.92\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.5095ms | 16.1607ms | 61.8783 Ops/s | 59.0297 Ops/s | $\color{#35bf28}+4.83\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.1937ms | 1.2559ms | 796.2631 Ops/s | 825.8142 Ops/s | $\color{#d91a1a}-3.58\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1316s | 7.7495ms | 129.0398 Ops/s | 129.3887 Ops/s | $\color{#d91a1a}-0.27\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1399s | 18.6552ms | 53.6044 Ops/s | 59.2156 Ops/s | $\textbf{\color{#d91a1a}-9.48\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.1966ms | 1.2450ms | 803.2010 Ops/s | 724.8087 Ops/s | $\textbf{\color{#35bf28}+10.82\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1318s | 7.8944ms | 126.6714 Ops/s | 126.7442 Ops/s | $\color{#d91a1a}-0.06\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.6749ms | 16.3779ms | 61.0578 Ops/s | 58.2538 Ops/s | $\color{#35bf28}+4.81\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.5369ms | 1.4107ms | 708.8908 Ops/s | 756.6148 Ops/s | $\textbf{\color{#d91a1a}-6.31\\%}$ |

[github-actions[bot]]

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results

| Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 59.4068ms | 58.5021ms | 17.0934 Ops/s | 17.3587 Ops/s | $\color{#d91a1a}-1.53\\%$ | | test_sync | 44.4433ms | 34.3640ms | 29.1002 Ops/s | 28.3032 Ops/s | $\color{#35bf28}+2.82\\%$ | | test_async | 54.0738ms | 30.5716ms | 32.7101 Ops/s | 33.1987 Ops/s | $\color{#d91a1a}-1.47\\%$ | | test_simple | 0.4818s | 0.4131s | 2.4208 Ops/s | 2.4642 Ops/s | $\color{#d91a1a}-1.76\\%$ | | test_transformed | 0.6449s | 0.5824s | 1.7171 Ops/s | 1.7520 Ops/s | $\color{#d91a1a}-1.99\\%$ | | test_serial | 1.3461s | 1.2788s | 0.7820 Ops/s | 0.7934 Ops/s | $\color{#d91a1a}-1.44\\%$ | | test_parallel | 1.1819s | 1.1116s | 0.8996 Ops/s | 0.9083 Ops/s | $\color{#d91a1a}-0.96\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.2535ms | 25.4528μs | 39.2884 KOps/s | 39.1258 KOps/s | $\color{#35bf28}+0.42\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 42.4790μs | 14.7405μs | 67.8401 KOps/s | 68.8984 KOps/s | $\color{#d91a1a}-1.54\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 69.2500μs | 14.4417μs | 69.2437 KOps/s | 67.8243 KOps/s | $\color{#35bf28}+2.09\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 40.2450μs | 8.6185μs | 116.0290 KOps/s | 119.1736 KOps/s | $\color{#d91a1a}-2.64\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 76.2930μs | 26.7685μs | 37.3574 KOps/s | 37.3439 KOps/s | $\color{#35bf28}+0.04\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 45.6360μs | 16.3593μs | 61.1273 KOps/s | 61.8543 KOps/s | $\color{#d91a1a}-1.18\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 57.8580μs | 15.9983μs | 62.5068 KOps/s | 62.0413 KOps/s | $\color{#35bf28}+0.75\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 42.2480μs | 10.2445μs | 97.6132 KOps/s | 99.7971 KOps/s | $\color{#d91a1a}-2.19\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 54.0910μs | 28.4413μs | 35.1601 KOps/s | 35.1161 KOps/s | $\color{#35bf28}+0.13\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 60.5630μs | 18.0237μs | 55.4826 KOps/s | 56.2201 KOps/s | $\color{#d91a1a}-1.31\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 46.2470μs | 16.0989μs | 62.1161 KOps/s | 61.8191 KOps/s | $\color{#35bf28}+0.48\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 55.7040μs | 10.2707μs | 97.3647 KOps/s | 99.9580 KOps/s | $\color{#d91a1a}-2.59\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 66.8350μs | 30.1699μs | 33.1456 KOps/s | 33.4531 KOps/s | $\color{#d91a1a}-0.92\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 68.9990μs | 19.3882μs | 51.5777 KOps/s | 51.5642 KOps/s | $\color{#35bf28}+0.03\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 59.9520μs | 17.5373μs | 57.0214 KOps/s | 57.1224 KOps/s | $\color{#d91a1a}-0.18\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 38.0320μs | 11.6675μs | 85.7078 KOps/s | 85.8773 KOps/s | $\color{#d91a1a}-0.20\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 77.2840μs | 28.5608μs | 35.0130 KOps/s | 34.9285 KOps/s | $\color{#35bf28}+0.24\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 47.7390μs | 18.1984μs | 54.9500 KOps/s | 56.2336 KOps/s | $\color{#d91a1a}-2.28\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 74.7400μs | 18.5708μs | 53.8481 KOps/s | 53.8994 KOps/s | $\color{#d91a1a}-0.10\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 30.5480μs | 11.5571μs | 86.5271 KOps/s | 89.8300 KOps/s | $\color{#d91a1a}-3.68\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 79.6290μs | 29.8745μs | 33.4734 KOps/s | 33.3287 KOps/s | $\color{#35bf28}+0.43\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 58.4490μs | 19.6180μs | 50.9735 KOps/s | 51.6402 KOps/s | $\color{#d91a1a}-1.29\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 61.7160μs | 20.0917μs | 49.7719 KOps/s | 49.8237 KOps/s | $\color{#d91a1a}-0.10\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 41.0970μs | 13.0846μs | 76.4257 KOps/s | 78.7928 KOps/s | $\color{#d91a1a}-3.00\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.2278ms | 31.7062μs | 31.5396 KOps/s | 31.5444 KOps/s | $\color{#d91a1a}-0.02\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 70.1110μs | 21.1541μs | 47.2721 KOps/s | 47.6844 KOps/s | $\color{#d91a1a}-0.86\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 54.0010μs | 20.4091μs | 48.9976 KOps/s | 50.0820 KOps/s | $\color{#d91a1a}-2.17\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 67.2900μs | 12.9786μs | 77.0497 KOps/s | 78.9401 KOps/s | $\color{#d91a1a}-2.39\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 0.1142ms | 32.7127μs | 30.5691 KOps/s | 30.5159 KOps/s | $\color{#35bf28}+0.17\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 51.6760μs | 22.5570μs | 44.3322 KOps/s | 44.8901 KOps/s | $\color{#d91a1a}-1.24\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 84.7310μs | 21.1676μs | 47.2420 KOps/s | 46.9985 KOps/s | $\color{#35bf28}+0.52\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 53.5100μs | 14.2819μs | 70.0187 KOps/s | 70.8055 KOps/s | $\color{#d91a1a}-1.11\\%$ | | test_values[generalized_advantage_estimate-True-True] | 10.3898ms | 9.2702ms | 107.8728 Ops/s | 102.2400 Ops/s | $\textbf{\color{#35bf28}+5.51\\%}$ | | test_values[vec_generalized_advantage_estimate-True-True] | 38.3692ms | 36.0294ms | 27.7551 Ops/s | 28.1818 Ops/s | $\color{#d91a1a}-1.51\\%$ | | test_values[td0_return_estimate-False-False] | 0.2313ms | 0.1661ms | 6.0210 KOps/s | 5.9459 KOps/s | $\color{#35bf28}+1.26\\%$ | | test_values[td1_return_estimate-False-False] | 27.2272ms | 23.5628ms | 42.4398 Ops/s | 42.5090 Ops/s | $\color{#d91a1a}-0.16\\%$ | | test_values[vec_td1_return_estimate-False-False] | 46.7451ms | 36.4067ms | 27.4675 Ops/s | 28.0370 Ops/s | $\color{#d91a1a}-2.03\\%$ | | test_values[td_lambda_return_estimate-True-False] | 37.6896ms | 34.1238ms | 29.3051 Ops/s | 29.2210 Ops/s | $\color{#35bf28}+0.29\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 38.0558ms | 36.0762ms | 27.7191 Ops/s | 28.0028 Ops/s | $\color{#d91a1a}-1.01\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.3565ms | 8.1302ms | 122.9981 Ops/s | 118.7957 Ops/s | $\color{#35bf28}+3.54\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.6317ms | 1.9856ms | 503.6333 Ops/s | 503.5336 Ops/s | $\color{#35bf28}+0.02\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.5170ms | 0.3631ms | 2.7537 KOps/s | 2.8044 KOps/s | $\color{#d91a1a}-1.81\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 48.5358ms | 47.0119ms | 21.2712 Ops/s | 22.2115 Ops/s | $\color{#d91a1a}-4.23\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.9509ms | 3.0375ms | 329.2165 Ops/s | 330.3533 Ops/s | $\color{#d91a1a}-0.34\\%$ | | test_dqn_speed | 2.2844ms | 1.4052ms | 711.6638 Ops/s | 716.7310 Ops/s | $\color{#d91a1a}-0.71\\%$ | | test_ddpg_speed | 3.9579ms | 2.9384ms | 340.3189 Ops/s | 344.1063 Ops/s | $\color{#d91a1a}-1.10\\%$ | | test_sac_speed | 9.0977ms | 8.5402ms | 117.0930 Ops/s | 118.0977 Ops/s | $\color{#d91a1a}-0.85\\%$ | | test_redq_speed | 18.5147ms | 13.5736ms | 73.6725 Ops/s | 72.8935 Ops/s | $\color{#35bf28}+1.07\\%$ | | test_redq_deprec_speed | 14.7618ms | 13.0937ms | 76.3728 Ops/s | 76.0396 Ops/s | $\color{#35bf28}+0.44\\%$ | | test_td3_speed | 8.7370ms | 8.4685ms | 118.0841 Ops/s | 117.3502 Ops/s | $\color{#35bf28}+0.63\\%$ | | test_cql_speed | 37.1620ms | 36.1947ms | 27.6284 Ops/s | 27.5631 Ops/s | $\color{#35bf28}+0.24\\%$ | | test_a2c_speed | 9.1694ms | 7.4197ms | 134.7770 Ops/s | 134.8802 Ops/s | $\color{#d91a1a}-0.08\\%$ | | test_ppo_speed | 8.5608ms | 7.7006ms | 129.8592 Ops/s | 130.0295 Ops/s | $\color{#d91a1a}-0.13\\%$ | | test_reinforce_speed | 7.9003ms | 6.5650ms | 152.3220 Ops/s | 151.7029 Ops/s | $\color{#35bf28}+0.41\\%$ | | test_iql_speed | 34.5755ms | 32.3350ms | 30.9263 Ops/s | 30.7197 Ops/s | $\color{#35bf28}+0.67\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.6657ms | 4.8915ms | 204.4372 Ops/s | 204.6633 Ops/s | $\color{#d91a1a}-0.11\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.6305ms | 0.4795ms | 2.0854 KOps/s | 2.0941 KOps/s | $\color{#d91a1a}-0.42\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6113ms | 0.4509ms | 2.2177 KOps/s | 2.2097 KOps/s | $\color{#35bf28}+0.36\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 7.5747ms | 4.8504ms | 206.1695 Ops/s | 206.0798 Ops/s | $\color{#35bf28}+0.04\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.1985ms | 0.4734ms | 2.1123 KOps/s | 2.1166 KOps/s | $\color{#d91a1a}-0.20\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7111ms | 0.4509ms | 2.2178 KOps/s | 2.2176 KOps/s | $+0.01\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.2123ms | 1.7011ms | 587.8416 Ops/s | 593.0031 Ops/s | $\color{#d91a1a}-0.87\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 1.7941ms | 1.6066ms | 622.4269 Ops/s | 625.2559 Ops/s | $\color{#d91a1a}-0.45\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.4250ms | 4.9782ms | 200.8763 Ops/s | 200.3645 Ops/s | $\color{#35bf28}+0.26\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9453ms | 0.6151ms | 1.6258 KOps/s | 1.4017 KOps/s | $\textbf{\color{#35bf28}+15.99\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8451ms | 0.5915ms | 1.6907 KOps/s | 1.6976 KOps/s | $\color{#d91a1a}-0.40\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.1229ms | 4.8890ms | 204.5428 Ops/s | 204.4627 Ops/s | $\color{#35bf28}+0.04\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.1349ms | 0.4849ms | 2.0622 KOps/s | 2.0846 KOps/s | $\color{#d91a1a}-1.08\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6360ms | 0.4561ms | 2.1923 KOps/s | 2.1803 KOps/s | $\color{#35bf28}+0.55\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 7.6515ms | 4.7945ms | 208.5730 Ops/s | 208.8409 Ops/s | $\color{#d91a1a}-0.13\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8583ms | 0.4707ms | 2.1243 KOps/s | 2.1120 KOps/s | $\color{#35bf28}+0.58\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6476ms | 0.4505ms | 2.2198 KOps/s | 2.2212 KOps/s | $\color{#d91a1a}-0.06\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 7.6146ms | 5.0045ms | 199.8206 Ops/s | 200.5960 Ops/s | $\color{#d91a1a}-0.39\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.6982ms | 0.6137ms | 1.6295 KOps/s | 1.6085 KOps/s | $\color{#35bf28}+1.30\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 7.3712ms | 0.5997ms | 1.6675 KOps/s | 1.6820 KOps/s | $\color{#d91a1a}-0.86\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1331s | 8.4725ms | 118.0292 Ops/s | 157.1661 Ops/s | $\textbf{\color{#d91a1a}-24.90\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 16.7779ms | 12.7630ms | 78.3517 Ops/s | 73.3022 Ops/s | $\textbf{\color{#35bf28}+6.89\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 5.9619ms | 1.2112ms | 825.5989 Ops/s | 927.2830 Ops/s | $\textbf{\color{#d91a1a}-10.97\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1068s | 5.8398ms | 171.2389 Ops/s | 126.5668 Ops/s | $\textbf{\color{#35bf28}+35.30\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 18.1243ms | 12.7882ms | 78.1970 Ops/s | 76.8840 Ops/s | $\color{#35bf28}+1.71\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 5.6566ms | 1.1754ms | 850.8031 Ops/s | 795.7197 Ops/s | $\textbf{\color{#35bf28}+6.92\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1083s | 6.0032ms | 166.5777 Ops/s | 167.2402 Ops/s | $\color{#d91a1a}-0.40\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 16.9570ms | 12.9717ms | 77.0911 Ops/s | 75.8137 Ops/s | $\color{#35bf28}+1.68\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 5.7556ms | 1.3291ms | 752.3891 Ops/s | 840.2756 Ops/s | $\textbf{\color{#d91a1a}-10.46\\%}$ |