search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.05k stars 273 forks source link

[BugFix] Fix done/terminated computation in slice samplers #2213

Closed vmoens closed 1 month ago

pytorch-bot[bot] commented 1 month ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2213

Note: Links to docs will display an error until the docs builds have been completed.

:x: 10 New Failures, 5 Unrelated Failures

As of commit 378fa6f554768c08d9db22301c1f3eba52130f94 with merge base d9341531ec18848366fbb95b71ae893656a65a8b (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2213#25928760563) ([gh](https://github.com/pytorch/rl/actions/runs/9412952351/job/25928760563)) `RuntimeError: Command docker exec -t 3661f209f349ff90fb0bf0de25f84d552fbaa6d29ff84d3cfd47762e85a172d2 /exec failed with exit code 1` * [Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2213#25928765144) ([gh](https://github.com/pytorch/rl/actions/runs/9412952355/job/25928765144)) `RuntimeError: Command docker exec -t cc1ac4020c93313235f6e8d89d7bfe32626e9f0d02f7a33e7f597ccd16853d94 /exec failed with exit code 1` * [RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2213#25928760693) ([gh](https://github.com/pytorch/rl/actions/runs/9412952341/job/25928760693)) `RuntimeError: Command docker exec -t 7e93d43f82eae4eb134080c421f1db4a047ae108f6e8e41bb3dac5b38de41fa9 /exec failed with exit code 1` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2213#25928763209) ([gh](https://github.com/pytorch/rl/actions/runs/9412952349/job/25928763209)) `RuntimeError: Command docker exec -t 1baa657a4f5080620733a7028e4a0e96015355247aa5bfccd4f3b7411c13fe76 /exec failed with exit code 1` * [Wheels / test-wheel (linux, ubuntu-20.04, 3.10)](https://hud.pytorch.org/pr/pytorch/rl/2213#25928808974) ([gh](https://github.com/pytorch/rl/actions/runs/9412952318/job/25928808974)) `ModuleNotFoundError: No module named 'dm_env'` * [Wheels / test-wheel (linux, ubuntu-20.04, 3.11)](https://hud.pytorch.org/pr/pytorch/rl/2213#25928809137) ([gh](https://github.com/pytorch/rl/actions/runs/9412952318/job/25928809137)) `##[error]The operation was canceled.` * [Wheels / test-wheel (linux, ubuntu-20.04, 3.8)](https://hud.pytorch.org/pr/pytorch/rl/2213#25928808646) ([gh](https://github.com/pytorch/rl/actions/runs/9412952318/job/25928808646)) `##[error]The operation was canceled.` * [Wheels / test-wheel (linux, ubuntu-20.04, 3.9)](https://hud.pytorch.org/pr/pytorch/rl/2213#25928808813) ([gh](https://github.com/pytorch/rl/actions/runs/9412952318/job/25928808813)) `##[error]The operation was canceled.` * [Wheels / test-wheel-windows (3.11)](https://hud.pytorch.org/pr/pytorch/rl/2213#25928878452) ([gh](https://github.com/pytorch/rl/actions/runs/9412952318/job/25928878452)) * [Wheels / test-wheel-windows (3.9)](https://hud.pytorch.org/pr/pytorch/rl/2213#25928877917) ([gh](https://github.com/pytorch/rl/actions/runs/9412952318/job/25928877917)) `ModuleNotFoundError: No module named 'dm_env'`

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

* [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2213#25928765573) ([gh](https://github.com/pytorch/rl/actions/runs/9412952355/job/25928765573)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/i/isl/libisl22_0.22.1-1_amd64.deb Undetermined Error [IP: 185.125.190.36 80]` * [Wheels / test-wheel-windows (3.10)](https://hud.pytorch.org/pr/pytorch/rl/2213#25928878200) ([gh](https://github.com/pytorch/rl/actions/runs/9412952318/job/25928878200)) (matched **win** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `##[error]The operation was canceled.` * [Wheels / test-wheel-windows (3.8)](https://hud.pytorch.org/pr/pytorch/rl/2213#25928877622) ([gh](https://github.com/pytorch/rl/actions/runs/9412952318/job/25928877622)) (matched **win** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `##[error]The operation was canceled.`

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2213#25928762921) ([gh](https://github.com/pytorch/rl/actions/runs/9412952349/job/25928762921)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/d9341531ec18848366fbb95b71ae893656a65a8b#25928751803)) `test/test_transforms.py::TestVecNorm::test_state_dict_vecnorm` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2213#25928759630) ([gh](https://github.com/pytorch/rl/actions/runs/9412952347/job/25928759630)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/d9341531ec18848366fbb95b71ae893656a65a8b#25928749886)) `ModuleNotFoundError: No module named 'dm_env'`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}13$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1273s | 61.6378ms | 16.2238 Ops/s | 16.9697 Ops/s | $\color{#d91a1a}-4.40\\%$ | | test_sync | 33.9014ms | 31.7941ms | 31.4523 Ops/s | 30.2913 Ops/s | $\color{#35bf28}+3.83\\%$ | | test_async | 54.6022ms | 28.9248ms | 34.5724 Ops/s | 35.1545 Ops/s | $\color{#d91a1a}-1.66\\%$ | | test_simple | 0.3845s | 0.3824s | 2.6149 Ops/s | 2.5576 Ops/s | $\color{#35bf28}+2.24\\%$ | | test_transformed | 0.5507s | 0.5486s | 1.8230 Ops/s | 1.7933 Ops/s | $\color{#35bf28}+1.66\\%$ | | test_serial | 1.3730s | 1.3066s | 0.7654 Ops/s | 0.7480 Ops/s | $\color{#35bf28}+2.32\\%$ | | test_parallel | 1.1542s | 1.0969s | 0.9116 Ops/s | 0.9110 Ops/s | $\color{#35bf28}+0.07\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.3020ms | 21.2885μs | 46.9736 KOps/s | 45.1753 KOps/s | $\color{#35bf28}+3.98\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 43.8510μs | 13.0868μs | 76.4131 KOps/s | 73.6620 KOps/s | $\color{#35bf28}+3.73\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 39.3930μs | 12.5692μs | 79.5595 KOps/s | 76.2979 KOps/s | $\color{#35bf28}+4.27\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 30.7770μs | 7.8166μs | 127.9330 KOps/s | 125.5477 KOps/s | $\color{#35bf28}+1.90\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 59.1490μs | 22.9454μs | 43.5816 KOps/s | 42.2584 KOps/s | $\color{#35bf28}+3.13\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 92.3930μs | 14.3211μs | 69.8269 KOps/s | 67.8041 KOps/s | $\color{#35bf28}+2.98\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 47.0470μs | 13.7714μs | 72.6140 KOps/s | 69.1031 KOps/s | $\textbf{\color{#35bf28}+5.08\\%}$ | | test_step_mdp_speed[True-True-False-False-False] | 32.7110μs | 8.9519μs | 111.7081 KOps/s | 108.2098 KOps/s | $\color{#35bf28}+3.23\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 61.3240μs | 23.9365μs | 41.7772 KOps/s | 39.6052 KOps/s | $\textbf{\color{#35bf28}+5.48\\%}$ | | test_step_mdp_speed[True-False-True-True-False] | 48.0190μs | 15.5897μs | 64.1450 KOps/s | 62.2116 KOps/s | $\color{#35bf28}+3.11\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 44.8330μs | 13.7698μs | 72.6227 KOps/s | 69.9171 KOps/s | $\color{#35bf28}+3.87\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 0.1058ms | 8.9293μs | 111.9911 KOps/s | 108.7954 KOps/s | $\color{#35bf28}+2.94\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 0.1367ms | 25.1914μs | 39.6960 KOps/s | 38.0973 KOps/s | $\color{#35bf28}+4.20\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 50.0530μs | 16.8391μs | 59.3857 KOps/s | 57.6915 KOps/s | $\color{#35bf28}+2.94\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 0.1646ms | 15.4933μs | 64.5440 KOps/s | 64.2999 KOps/s | $\color{#35bf28}+0.38\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 41.8680μs | 10.1204μs | 98.8104 KOps/s | 95.2860 KOps/s | $\color{#35bf28}+3.70\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 64.3190μs | 24.1806μs | 41.3554 KOps/s | 40.9444 KOps/s | $\color{#35bf28}+1.00\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 73.8660μs | 15.6569μs | 63.8695 KOps/s | 62.6864 KOps/s | $\color{#35bf28}+1.89\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 74.2080μs | 16.0289μs | 62.3874 KOps/s | 60.3634 KOps/s | $\color{#35bf28}+3.35\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 38.3200μs | 10.2149μs | 97.8960 KOps/s | 96.2116 KOps/s | $\color{#35bf28}+1.75\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 67.6550μs | 25.1934μs | 39.6929 KOps/s | 38.4684 KOps/s | $\color{#35bf28}+3.18\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 67.5550μs | 16.7564μs | 59.6788 KOps/s | 58.4260 KOps/s | $\color{#35bf28}+2.14\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 0.1628ms | 17.2026μs | 58.1307 KOps/s | 55.9054 KOps/s | $\color{#35bf28}+3.98\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 40.4150μs | 11.4891μs | 87.0389 KOps/s | 85.9548 KOps/s | $\color{#35bf28}+1.26\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 63.5580μs | 26.3167μs | 37.9987 KOps/s | 36.7875 KOps/s | $\color{#35bf28}+3.29\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 61.5240μs | 18.1854μs | 54.9890 KOps/s | 54.6437 KOps/s | $\color{#35bf28}+0.63\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 98.9120μs | 16.9642μs | 58.9477 KOps/s | 55.9012 KOps/s | $\textbf{\color{#35bf28}+5.45\\%}$ | | test_step_mdp_speed[False-False-True-False-False] | 51.9960μs | 11.3810μs | 87.8659 KOps/s | 86.2837 KOps/s | $\color{#35bf28}+1.83\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 39.7940μs | 27.7808μs | 35.9961 KOps/s | 34.1515 KOps/s | $\textbf{\color{#35bf28}+5.40\\%}$ | | test_step_mdp_speed[False-False-False-True-False] | 89.3210μs | 18.9964μs | 52.6415 KOps/s | 50.9931 KOps/s | $\color{#35bf28}+3.23\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 77.2030μs | 17.9173μs | 55.8119 KOps/s | 52.8646 KOps/s | $\textbf{\color{#35bf28}+5.58\\%}$ | | test_step_mdp_speed[False-False-False-False-False] | 40.8060μs | 12.1906μs | 82.0305 KOps/s | 74.3847 KOps/s | $\textbf{\color{#35bf28}+10.28\\%}$ | | test_values[generalized_advantage_estimate-True-True] | 10.0006ms | 9.6062ms | 104.0990 Ops/s | 97.5190 Ops/s | $\textbf{\color{#35bf28}+6.75\\%}$ | | test_values[vec_generalized_advantage_estimate-True-True] | 49.7217ms | 36.2459ms | 27.5893 Ops/s | 27.6314 Ops/s | $\color{#d91a1a}-0.15\\%$ | | test_values[td0_return_estimate-False-False] | 0.2537ms | 0.1919ms | 5.2112 KOps/s | 5.5146 KOps/s | $\textbf{\color{#d91a1a}-5.50\\%}$ | | test_values[td1_return_estimate-False-False] | 26.6172ms | 24.2722ms | 41.1994 Ops/s | 39.7098 Ops/s | $\color{#35bf28}+3.75\\%$ | | test_values[vec_td1_return_estimate-False-False] | 37.0113ms | 35.7788ms | 27.9495 Ops/s | 27.4600 Ops/s | $\color{#35bf28}+1.78\\%$ | | test_values[td_lambda_return_estimate-True-False] | 48.5476ms | 35.3362ms | 28.2996 Ops/s | 26.9512 Ops/s | $\textbf{\color{#35bf28}+5.00\\%}$ | | test_values[vec_td_lambda_return_estimate-True-False] | 37.3083ms | 36.0107ms | 27.7695 Ops/s | 27.7929 Ops/s | $\color{#d91a1a}-0.08\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.6209ms | 8.4040ms | 118.9909 Ops/s | 118.4691 Ops/s | $\color{#35bf28}+0.44\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.2092ms | 1.9820ms | 504.5318 Ops/s | 493.6425 Ops/s | $\color{#35bf28}+2.21\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.6408ms | 0.3616ms | 2.7658 KOps/s | 2.6997 KOps/s | $\color{#35bf28}+2.45\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 47.9656ms | 45.1958ms | 22.1259 Ops/s | 25.1560 Ops/s | $\textbf{\color{#d91a1a}-12.04\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.6670ms | 3.1163ms | 320.8979 Ops/s | 328.9329 Ops/s | $\color{#d91a1a}-2.44\\%$ | | test_dqn_speed | 6.4029ms | 1.4159ms | 706.2803 Ops/s | 701.2641 Ops/s | $\color{#35bf28}+0.72\\%$ | | test_ddpg_speed | 3.6015ms | 2.9390ms | 340.2484 Ops/s | 334.3968 Ops/s | $\color{#35bf28}+1.75\\%$ | | test_sac_speed | 10.3175ms | 9.0529ms | 110.4616 Ops/s | 111.9195 Ops/s | $\color{#d91a1a}-1.30\\%$ | | test_redq_speed | 15.5402ms | 13.8469ms | 72.2181 Ops/s | 64.6411 Ops/s | $\textbf{\color{#35bf28}+11.72\\%}$ | | test_redq_deprec_speed | 16.1005ms | 14.3830ms | 69.5263 Ops/s | 68.9652 Ops/s | $\color{#35bf28}+0.81\\%$ | | test_td3_speed | 17.7108ms | 8.8654ms | 112.7985 Ops/s | 112.5232 Ops/s | $\color{#35bf28}+0.24\\%$ | | test_cql_speed | 38.2901ms | 37.4294ms | 26.7170 Ops/s | 26.4762 Ops/s | $\color{#35bf28}+0.91\\%$ | | test_a2c_speed | 9.7020ms | 8.1341ms | 122.9399 Ops/s | 128.1722 Ops/s | $\color{#d91a1a}-4.08\\%$ | | test_ppo_speed | 9.9614ms | 8.5607ms | 116.8132 Ops/s | 123.3391 Ops/s | $\textbf{\color{#d91a1a}-5.29\\%}$ | | test_reinforce_speed | 7.5701ms | 6.8446ms | 146.1012 Ops/s | 146.0079 Ops/s | $\color{#35bf28}+0.06\\%$ | | test_iql_speed | 35.2787ms | 33.5875ms | 29.7730 Ops/s | 29.7358 Ops/s | $\color{#35bf28}+0.12\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.1197ms | 3.8092ms | 262.5199 Ops/s | 264.6965 Ops/s | $\color{#d91a1a}-0.82\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7095ms | 0.5198ms | 1.9237 KOps/s | 1.9314 KOps/s | $\color{#d91a1a}-0.40\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.9520ms | 0.4982ms | 2.0072 KOps/s | 2.0225 KOps/s | $\color{#d91a1a}-0.76\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.8833ms | 3.8780ms | 257.8630 Ops/s | 266.8992 Ops/s | $\color{#d91a1a}-3.39\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.9153ms | 0.5170ms | 1.9344 KOps/s | 1.9603 KOps/s | $\color{#d91a1a}-1.32\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6575ms | 0.4919ms | 2.0330 KOps/s | 2.0576 KOps/s | $\color{#d91a1a}-1.20\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.5372ms | 1.7454ms | 572.9329 Ops/s | 776.8886 Ops/s | $\textbf{\color{#d91a1a}-26.25\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.1716ms | 1.6435ms | 608.4440 Ops/s | 814.8573 Ops/s | $\textbf{\color{#d91a1a}-25.33\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.6014ms | 3.8930ms | 256.8703 Ops/s | 251.0587 Ops/s | $\color{#35bf28}+2.31\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.7393ms | 0.6327ms | 1.5806 KOps/s | 1.5645 KOps/s | $\color{#35bf28}+1.03\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.1214s | 0.7188ms | 1.3913 KOps/s | 1.6204 KOps/s | $\textbf{\color{#d91a1a}-14.14\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.5855ms | 3.9947ms | 250.3290 Ops/s | 251.8658 Ops/s | $\color{#d91a1a}-0.61\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6928ms | 0.5285ms | 1.8921 KOps/s | 1.8856 KOps/s | $\color{#35bf28}+0.34\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6861ms | 0.5042ms | 1.9834 KOps/s | 1.9901 KOps/s | $\color{#d91a1a}-0.34\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.5203ms | 3.8756ms | 258.0261 Ops/s | 260.4805 Ops/s | $\color{#d91a1a}-0.94\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.1211s | 0.6944ms | 1.4401 KOps/s | 1.9052 KOps/s | $\textbf{\color{#d91a1a}-24.41\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7329ms | 0.4946ms | 2.0219 KOps/s | 2.0083 KOps/s | $\color{#35bf28}+0.68\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.0140ms | 4.0446ms | 247.2457 Ops/s | 255.5925 Ops/s | $\color{#d91a1a}-3.27\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.7619ms | 0.6430ms | 1.5551 KOps/s | 1.5617 KOps/s | $\color{#d91a1a}-0.42\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.1239ms | 0.6208ms | 1.6109 KOps/s | 1.6266 KOps/s | $\color{#d91a1a}-0.97\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1226s | 8.3840ms | 119.2746 Ops/s | 115.4616 Ops/s | $\color{#35bf28}+3.30\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 16.0401ms | 13.1118ms | 76.2670 Ops/s | 76.8582 Ops/s | $\color{#d91a1a}-0.77\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 1.5704ms | 1.0736ms | 931.4059 Ops/s | 823.1971 Ops/s | $\textbf{\color{#35bf28}+13.14\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1252s | 6.2270ms | 160.5904 Ops/s | 161.0499 Ops/s | $\color{#d91a1a}-0.29\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 15.5191ms | 12.9020ms | 77.5076 Ops/s | 76.1623 Ops/s | $\color{#35bf28}+1.77\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.5846ms | 1.0762ms | 929.2096 Ops/s | 871.3799 Ops/s | $\textbf{\color{#35bf28}+6.64\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1183s | 8.6884ms | 115.0957 Ops/s | 107.9691 Ops/s | $\textbf{\color{#35bf28}+6.60\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 16.2894ms | 13.4916ms | 74.1200 Ops/s | 72.5821 Ops/s | $\color{#35bf28}+2.12\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.1623ms | 1.4078ms | 710.3059 Ops/s | 637.4480 Ops/s | $\textbf{\color{#35bf28}+11.43\\%}$ |
github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1196s | 0.1179s | 8.4812 Ops/s | 8.4687 Ops/s | $\color{#35bf28}+0.15\\%$ | | test_sync | 0.1046s | 0.1027s | 9.7339 Ops/s | 9.6764 Ops/s | $\color{#35bf28}+0.59\\%$ | | test_async | 0.2004s | 79.3112ms | 12.6086 Ops/s | 10.2384 Ops/s | $\textbf{\color{#35bf28}+23.15\\%}$ | | test_single_pixels | 0.1280s | 0.1277s | 7.8279 Ops/s | 7.7756 Ops/s | $\color{#35bf28}+0.67\\%$ | | test_sync_pixels | 85.7430ms | 84.1952ms | 11.8772 Ops/s | 12.0285 Ops/s | $\color{#d91a1a}-1.26\\%$ | | test_async_pixels | 0.1517s | 65.3425ms | 15.3040 Ops/s | 14.8685 Ops/s | $\color{#35bf28}+2.93\\%$ | | test_simple | 0.8849s | 0.8208s | 1.2183 Ops/s | 1.1897 Ops/s | $\color{#35bf28}+2.41\\%$ | | test_transformed | 1.1329s | 1.0723s | 0.9325 Ops/s | 0.9156 Ops/s | $\color{#35bf28}+1.86\\%$ | | test_serial | 2.5479s | 2.4910s | 0.4014 Ops/s | 0.3843 Ops/s | $\color{#35bf28}+4.46\\%$ | | test_parallel | 2.4254s | 2.3610s | 0.4235 Ops/s | 0.4238 Ops/s | $\color{#d91a1a}-0.05\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1006ms | 33.3500μs | 29.9850 KOps/s | 29.9364 KOps/s | $\color{#35bf28}+0.16\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 45.4420μs | 19.8768μs | 50.3099 KOps/s | 50.0547 KOps/s | $\color{#35bf28}+0.51\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 44.2320μs | 19.2135μs | 52.0468 KOps/s | 51.9065 KOps/s | $\color{#35bf28}+0.27\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 40.0420μs | 11.4261μs | 87.5187 KOps/s | 87.0043 KOps/s | $\color{#35bf28}+0.59\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 67.4930μs | 34.5041μs | 28.9821 KOps/s | 28.3618 KOps/s | $\color{#35bf28}+2.19\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 50.0620μs | 21.7103μs | 46.0612 KOps/s | 45.5604 KOps/s | $\color{#35bf28}+1.10\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 47.0620μs | 20.9371μs | 47.7621 KOps/s | 48.0434 KOps/s | $\color{#d91a1a}-0.59\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 34.9320μs | 13.3319μs | 75.0080 KOps/s | 75.0410 KOps/s | $\color{#d91a1a}-0.04\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 67.2530μs | 37.2138μs | 26.8718 KOps/s | 27.3799 KOps/s | $\color{#d91a1a}-1.86\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 57.5530μs | 23.7422μs | 42.1190 KOps/s | 42.3295 KOps/s | $\color{#d91a1a}-0.50\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 46.5120μs | 20.9671μs | 47.6937 KOps/s | 48.0539 KOps/s | $\color{#d91a1a}-0.75\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 40.8820μs | 13.5087μs | 74.0265 KOps/s | 75.8888 KOps/s | $\color{#d91a1a}-2.45\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 62.6930μs | 38.6299μs | 25.8867 KOps/s | 25.8017 KOps/s | $\color{#35bf28}+0.33\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 49.4030μs | 25.6395μs | 39.0024 KOps/s | 38.9010 KOps/s | $\color{#35bf28}+0.26\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 53.2020μs | 23.0773μs | 43.3326 KOps/s | 44.7229 KOps/s | $\color{#d91a1a}-3.11\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 39.6120μs | 15.4070μs | 64.9056 KOps/s | 66.1230 KOps/s | $\color{#d91a1a}-1.84\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 62.9630μs | 36.9828μs | 27.0396 KOps/s | 26.9066 KOps/s | $\color{#35bf28}+0.49\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 45.8320μs | 23.8627μs | 41.9064 KOps/s | 41.8326 KOps/s | $\color{#35bf28}+0.18\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 49.9830μs | 25.6065μs | 39.0526 KOps/s | 41.1531 KOps/s | $\textbf{\color{#d91a1a}-5.10\\%}$ | | test_step_mdp_speed[False-True-True-False-False] | 36.2420μs | 15.5392μs | 64.3532 KOps/s | 66.5050 KOps/s | $\color{#d91a1a}-3.24\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 74.2530μs | 39.0872μs | 25.5838 KOps/s | 25.8390 KOps/s | $\color{#d91a1a}-0.99\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 48.4530μs | 25.8199μs | 38.7298 KOps/s | 39.5047 KOps/s | $\color{#d91a1a}-1.96\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 56.4930μs | 27.3686μs | 36.5383 KOps/s | 37.8233 KOps/s | $\color{#d91a1a}-3.40\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 36.3320μs | 17.2421μs | 57.9974 KOps/s | 59.3290 KOps/s | $\color{#d91a1a}-2.24\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 71.2530μs | 40.7662μs | 24.5302 KOps/s | 24.7456 KOps/s | $\color{#d91a1a}-0.87\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 56.8630μs | 27.8674μs | 35.8842 KOps/s | 36.7914 KOps/s | $\color{#d91a1a}-2.47\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 55.4230μs | 26.6865μs | 37.4721 KOps/s | 38.3809 KOps/s | $\color{#d91a1a}-2.37\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 35.3220μs | 17.3605μs | 57.6019 KOps/s | 59.1622 KOps/s | $\color{#d91a1a}-2.64\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 68.7430μs | 42.9444μs | 23.2859 KOps/s | 23.3816 KOps/s | $\color{#d91a1a}-0.41\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 56.4520μs | 29.5110μs | 33.8856 KOps/s | 34.2991 KOps/s | $\color{#d91a1a}-1.21\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 49.8720μs | 28.1491μs | 35.5252 KOps/s | 35.3257 KOps/s | $\color{#35bf28}+0.56\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 38.0120μs | 19.1404μs | 52.2454 KOps/s | 53.3102 KOps/s | $\color{#d91a1a}-2.00\\%$ | | test_values[generalized_advantage_estimate-True-True] | 25.5837ms | 25.0859ms | 39.8631 Ops/s | 40.0622 Ops/s | $\color{#d91a1a}-0.50\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 88.8007ms | 3.3453ms | 298.9289 Ops/s | 298.8011 Ops/s | $\color{#35bf28}+0.04\\%$ | | test_values[td0_return_estimate-False-False] | 93.8540μs | 63.5160μs | 15.7441 KOps/s | 15.6610 KOps/s | $\color{#35bf28}+0.53\\%$ | | test_values[td1_return_estimate-False-False] | 53.7780ms | 53.0808ms | 18.8392 Ops/s | 18.9799 Ops/s | $\color{#d91a1a}-0.74\\%$ | | test_values[vec_td1_return_estimate-False-False] | 2.0709ms | 1.7668ms | 566.0021 Ops/s | 567.4811 Ops/s | $\color{#d91a1a}-0.26\\%$ | | test_values[td_lambda_return_estimate-True-False] | 87.7422ms | 85.0636ms | 11.7559 Ops/s | 11.9268 Ops/s | $\color{#d91a1a}-1.43\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 2.1352ms | 1.7625ms | 567.3893 Ops/s | 568.8171 Ops/s | $\color{#d91a1a}-0.25\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 24.1430ms | 23.8927ms | 41.8538 Ops/s | 42.0116 Ops/s | $\color{#d91a1a}-0.38\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9022ms | 0.6989ms | 1.4308 KOps/s | 1.4297 KOps/s | $\color{#35bf28}+0.08\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7200ms | 0.6517ms | 1.5344 KOps/s | 1.5488 KOps/s | $\color{#d91a1a}-0.93\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.4997ms | 1.4521ms | 688.6801 Ops/s | 690.2775 Ops/s | $\color{#d91a1a}-0.23\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.9339ms | 0.6688ms | 1.4952 KOps/s | 1.5098 KOps/s | $\color{#d91a1a}-0.97\\%$ | | test_dqn_speed | 1.8122ms | 1.4625ms | 683.7566 Ops/s | 690.1982 Ops/s | $\color{#d91a1a}-0.93\\%$ | | test_ddpg_speed | 3.2040ms | 2.9971ms | 333.6587 Ops/s | 334.8687 Ops/s | $\color{#d91a1a}-0.36\\%$ | | test_sac_speed | 8.7607ms | 8.4581ms | 118.2297 Ops/s | 117.5929 Ops/s | $\color{#35bf28}+0.54\\%$ | | test_redq_speed | 12.9666ms | 10.8669ms | 92.0226 Ops/s | 83.0516 Ops/s | $\textbf{\color{#35bf28}+10.80\\%}$ | | test_redq_deprec_speed | 12.0763ms | 11.6034ms | 86.1813 Ops/s | 83.5272 Ops/s | $\color{#35bf28}+3.18\\%$ | | test_td3_speed | 17.2898ms | 8.4726ms | 118.0274 Ops/s | 118.4847 Ops/s | $\color{#d91a1a}-0.39\\%$ | | test_cql_speed | 27.5326ms | 26.2427ms | 38.1059 Ops/s | 37.7725 Ops/s | $\color{#35bf28}+0.88\\%$ | | test_a2c_speed | 5.9969ms | 5.7967ms | 172.5127 Ops/s | 173.1282 Ops/s | $\color{#d91a1a}-0.36\\%$ | | test_ppo_speed | 6.3280ms | 6.1106ms | 163.6492 Ops/s | 163.6341 Ops/s | $+0.01\\%$ | | test_reinforce_speed | 5.6744ms | 4.7075ms | 212.4267 Ops/s | 209.8334 Ops/s | $\color{#35bf28}+1.24\\%$ | | test_iql_speed | 21.7756ms | 20.2394ms | 49.4085 Ops/s | 50.1799 Ops/s | $\color{#d91a1a}-1.54\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.7837ms | 4.6624ms | 214.4838 Ops/s | 215.0677 Ops/s | $\color{#d91a1a}-0.27\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.3756ms | 0.6097ms | 1.6401 KOps/s | 1.4255 KOps/s | $\textbf{\color{#35bf28}+15.05\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7979ms | 0.5861ms | 1.7063 KOps/s | 1.7007 KOps/s | $\color{#35bf28}+0.33\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.9018ms | 4.6525ms | 214.9405 Ops/s | 216.9898 Ops/s | $\color{#d91a1a}-0.94\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7302ms | 0.6030ms | 1.6583 KOps/s | 1.6572 KOps/s | $\color{#35bf28}+0.07\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.8554ms | 0.5792ms | 1.7264 KOps/s | 1.7200 KOps/s | $\color{#35bf28}+0.38\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.3320ms | 2.1638ms | 462.1568 Ops/s | 638.1223 Ops/s | $\textbf{\color{#d91a1a}-27.58\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 5.9717ms | 2.0615ms | 485.0812 Ops/s | 662.5717 Ops/s | $\textbf{\color{#d91a1a}-26.79\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.9189ms | 4.7308ms | 211.3821 Ops/s | 209.7324 Ops/s | $\color{#35bf28}+0.79\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8498ms | 0.7380ms | 1.3551 KOps/s | 1.3508 KOps/s | $\color{#35bf28}+0.32\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.5743ms | 0.7175ms | 1.3938 KOps/s | 1.3869 KOps/s | $\color{#35bf28}+0.49\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.7798ms | 4.6485ms | 215.1252 Ops/s | 213.8562 Ops/s | $\color{#35bf28}+0.59\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7483ms | 0.6143ms | 1.6277 KOps/s | 1.6294 KOps/s | $\color{#d91a1a}-0.10\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.8189ms | 0.5917ms | 1.6900 KOps/s | 1.6869 KOps/s | $\color{#35bf28}+0.18\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.4177ms | 4.6279ms | 216.0810 Ops/s | 215.2598 Ops/s | $\color{#35bf28}+0.38\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8832ms | 0.6069ms | 1.6476 KOps/s | 1.6603 KOps/s | $\color{#d91a1a}-0.77\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.8037ms | 0.5870ms | 1.7035 KOps/s | 1.7269 KOps/s | $\color{#d91a1a}-1.35\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.9484ms | 4.7885ms | 208.8351 Ops/s | 208.5676 Ops/s | $\color{#35bf28}+0.13\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9980ms | 0.7424ms | 1.3469 KOps/s | 1.3547 KOps/s | $\color{#d91a1a}-0.58\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8823ms | 0.7225ms | 1.3840 KOps/s | 1.4043 KOps/s | $\color{#d91a1a}-1.45\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1254s | 7.4483ms | 134.2588 Ops/s | 101.0458 Ops/s | $\textbf{\color{#35bf28}+32.87\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.2827ms | 15.9086ms | 62.8591 Ops/s | 63.2703 Ops/s | $\color{#d91a1a}-0.65\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.4201ms | 1.3356ms | 748.7016 Ops/s | 758.2140 Ops/s | $\color{#d91a1a}-1.25\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1200s | 9.5813ms | 104.3697 Ops/s | 137.8944 Ops/s | $\textbf{\color{#d91a1a}-24.31\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 18.2832ms | 15.9154ms | 62.8322 Ops/s | 62.9545 Ops/s | $\color{#d91a1a}-0.19\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.4683ms | 1.3206ms | 757.2282 Ops/s | 752.5143 Ops/s | $\color{#35bf28}+0.63\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1198s | 7.6381ms | 130.9227 Ops/s | 132.5431 Ops/s | $\color{#d91a1a}-1.22\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.5603ms | 16.2575ms | 61.5102 Ops/s | 62.0696 Ops/s | $\color{#d91a1a}-0.90\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.7711ms | 1.7076ms | 585.6088 Ops/s | 589.5748 Ops/s | $\color{#d91a1a}-0.67\\%$ |