search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.19k stars 289 forks source link

[Doc] Better doc for make_tensordict_primer #2324

Closed vmoens closed 1 month ago

pytorch-bot[bot] commented 1 month ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2324

Note: Links to docs will display an error until the docs builds have been completed.

:x: 1 New Failure, 1 Pending, 1 Unrelated Failure

As of commit e049f474178665e885a601e312e1f8021e4c3820 with merge base c6ef08017bf0f4e756dabd730530a3f3e564ed5f (image):

NEW FAILURE - The following job has failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2324#27969492358) ([gh](https://github.com/pytorch/rl/actions/runs/10113338983/job/27969492358)) `RuntimeError: Command docker exec -t 81f7ec9dec89f100855ab57e18b2f208f9af3a615cb4844d3c248b04bd2b492e /exec failed with exit code 139`

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2324#27969491556) ([gh](https://github.com/pytorch/rl/actions/runs/10113339016/job/27969491556)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/c6ef08017bf0f4e756dabd730530a3f3e564ed5f#27879545997)) `test/test_transforms.py::TestActionDiscretizer::test_trans_parallel_env_check[False]`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 59.3862ms | 58.3959ms | 17.1245 Ops/s | 17.3478 Ops/s | $\color{#d91a1a}-1.29\\%$ | | test_sync | 37.9244ms | 32.2590ms | 30.9991 Ops/s | 32.1161 Ops/s | $\color{#d91a1a}-3.48\\%$ | | test_async | 0.1062s | 30.3767ms | 32.9200 Ops/s | 33.3761 Ops/s | $\color{#d91a1a}-1.37\\%$ | | test_simple | 0.5052s | 0.4256s | 2.3495 Ops/s | 2.5506 Ops/s | $\textbf{\color{#d91a1a}-7.89\\%}$ | | test_transformed | 0.6583s | 0.5844s | 1.7112 Ops/s | 1.8053 Ops/s | $\textbf{\color{#d91a1a}-5.21\\%}$ | | test_serial | 1.3372s | 1.2617s | 0.7926 Ops/s | 0.7944 Ops/s | $\color{#d91a1a}-0.22\\%$ | | test_parallel | 1.1911s | 1.1151s | 0.8968 Ops/s | 0.8990 Ops/s | $\color{#d91a1a}-0.25\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.2024ms | 24.5378μs | 40.7534 KOps/s | 41.1964 KOps/s | $\color{#d91a1a}-1.08\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 39.5440μs | 14.2084μs | 70.3807 KOps/s | 71.8103 KOps/s | $\color{#d91a1a}-1.99\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 65.5120μs | 14.1885μs | 70.4796 KOps/s | 72.0904 KOps/s | $\color{#d91a1a}-2.23\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 33.8330μs | 8.2643μs | 121.0022 KOps/s | 124.3486 KOps/s | $\color{#d91a1a}-2.69\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 86.0010μs | 26.1179μs | 38.2880 KOps/s | 38.8463 KOps/s | $\color{#d91a1a}-1.44\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 48.8310μs | 15.7295μs | 63.5746 KOps/s | 64.5118 KOps/s | $\color{#d91a1a}-1.45\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 77.8250μs | 15.5751μs | 64.2052 KOps/s | 64.9102 KOps/s | $\color{#d91a1a}-1.09\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 31.5680μs | 9.7994μs | 102.0466 KOps/s | 103.0512 KOps/s | $\color{#d91a1a}-0.97\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 84.7990μs | 27.6977μs | 36.1041 KOps/s | 36.2966 KOps/s | $\color{#d91a1a}-0.53\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 60.3530μs | 17.3373μs | 57.6792 KOps/s | 57.8064 KOps/s | $\color{#d91a1a}-0.22\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 46.3370μs | 15.6772μs | 63.7869 KOps/s | 64.7028 KOps/s | $\color{#d91a1a}-1.42\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 34.7150μs | 9.7405μs | 102.6644 KOps/s | 105.2383 KOps/s | $\color{#d91a1a}-2.45\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 98.1470μs | 29.2251μs | 34.2171 KOps/s | 34.8117 KOps/s | $\color{#d91a1a}-1.71\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 93.6040μs | 18.5540μs | 53.8967 KOps/s | 53.6169 KOps/s | $\color{#35bf28}+0.52\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 72.7700μs | 17.1613μs | 58.2707 KOps/s | 59.1133 KOps/s | $\color{#d91a1a}-1.43\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 46.5870μs | 11.0742μs | 90.2996 KOps/s | 91.3482 KOps/s | $\color{#d91a1a}-1.15\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 82.9720μs | 27.5713μs | 36.2697 KOps/s | 36.6207 KOps/s | $\color{#d91a1a}-0.96\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 47.5390μs | 17.4208μs | 57.4027 KOps/s | 58.6565 KOps/s | $\color{#d91a1a}-2.14\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 59.0200μs | 18.2066μs | 54.9253 KOps/s | 55.5065 KOps/s | $\color{#d91a1a}-1.05\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 38.5120μs | 10.9379μs | 91.4254 KOps/s | 92.1960 KOps/s | $\color{#d91a1a}-0.84\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 79.1380μs | 28.9008μs | 34.6011 KOps/s | 35.1088 KOps/s | $\color{#d91a1a}-1.45\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 93.4740μs | 18.5971μs | 53.7719 KOps/s | 53.9009 KOps/s | $\color{#d91a1a}-0.24\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 79.8490μs | 19.8206μs | 50.4526 KOps/s | 51.8774 KOps/s | $\color{#d91a1a}-2.75\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 38.9520μs | 12.3854μs | 80.7405 KOps/s | 80.8507 KOps/s | $\color{#d91a1a}-0.14\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.7991ms | 29.7000μs | 33.6701 KOps/s | 32.7472 KOps/s | $\color{#35bf28}+2.82\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 93.4910μs | 19.6794μs | 50.8145 KOps/s | 50.2868 KOps/s | $\color{#35bf28}+1.05\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 52.9090μs | 19.4434μs | 51.4313 KOps/s | 52.2843 KOps/s | $\color{#d91a1a}-1.63\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 43.3520μs | 12.2671μs | 81.5187 KOps/s | 80.9043 KOps/s | $\color{#35bf28}+0.76\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 74.6200μs | 31.5189μs | 31.7270 KOps/s | 31.9367 KOps/s | $\color{#d91a1a}-0.66\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 55.1430μs | 21.2769μs | 46.9993 KOps/s | 47.3949 KOps/s | $\color{#d91a1a}-0.83\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 61.7250μs | 20.7052μs | 48.2971 KOps/s | 49.1083 KOps/s | $\color{#d91a1a}-1.65\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 63.3380μs | 13.5488μs | 73.8075 KOps/s | 75.1489 KOps/s | $\color{#d91a1a}-1.78\\%$ | | test_values[generalized_advantage_estimate-True-True] | 10.6787ms | 9.5585ms | 104.6194 Ops/s | 105.7537 Ops/s | $\color{#d91a1a}-1.07\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 39.3481ms | 35.8363ms | 27.9047 Ops/s | 27.9445 Ops/s | $\color{#d91a1a}-0.14\\%$ | | test_values[td0_return_estimate-False-False] | 0.2231ms | 0.1807ms | 5.5333 KOps/s | 5.3466 KOps/s | $\color{#35bf28}+3.49\\%$ | | test_values[td1_return_estimate-False-False] | 26.0715ms | 23.5012ms | 42.5511 Ops/s | 41.4163 Ops/s | $\color{#35bf28}+2.74\\%$ | | test_values[vec_td1_return_estimate-False-False] | 40.9525ms | 35.9745ms | 27.7975 Ops/s | 27.7976 Ops/s | $-0.00\\%$ | | test_values[td_lambda_return_estimate-True-False] | 39.5780ms | 34.2492ms | 29.1977 Ops/s | 29.3725 Ops/s | $\color{#d91a1a}-0.60\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 42.0569ms | 35.9930ms | 27.7832 Ops/s | 27.9471 Ops/s | $\color{#d91a1a}-0.59\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.6465ms | 8.2234ms | 121.6041 Ops/s | 122.1691 Ops/s | $\color{#d91a1a}-0.46\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.2940ms | 1.9609ms | 509.9699 Ops/s | 487.5781 Ops/s | $\color{#35bf28}+4.59\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.5979ms | 0.3614ms | 2.7670 KOps/s | 2.7448 KOps/s | $\color{#35bf28}+0.81\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 50.5012ms | 48.4200ms | 20.6526 Ops/s | 19.9768 Ops/s | $\color{#35bf28}+3.38\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 4.0674ms | 3.0610ms | 326.6918 Ops/s | 326.1081 Ops/s | $\color{#35bf28}+0.18\\%$ | | test_dqn_speed | 2.0124ms | 1.3093ms | 763.7537 Ops/s | 753.9722 Ops/s | $\color{#35bf28}+1.30\\%$ | | test_ddpg_speed | 4.1366ms | 2.7575ms | 362.6476 Ops/s | 362.4393 Ops/s | $\color{#35bf28}+0.06\\%$ | | test_sac_speed | 10.5203ms | 8.3367ms | 119.9511 Ops/s | 120.9100 Ops/s | $\color{#d91a1a}-0.79\\%$ | | test_redq_speed | 14.3579ms | 13.2442ms | 75.5048 Ops/s | 76.9205 Ops/s | $\color{#d91a1a}-1.84\\%$ | | test_redq_deprec_speed | 15.9078ms | 13.6511ms | 73.2539 Ops/s | 73.9168 Ops/s | $\color{#d91a1a}-0.90\\%$ | | test_td3_speed | 8.6892ms | 8.1712ms | 122.3810 Ops/s | 121.7355 Ops/s | $\color{#35bf28}+0.53\\%$ | | test_cql_speed | 38.3415ms | 36.2119ms | 27.6152 Ops/s | 27.1433 Ops/s | $\color{#35bf28}+1.74\\%$ | | test_a2c_speed | 8.5211ms | 7.5306ms | 132.7913 Ops/s | 130.7075 Ops/s | $\color{#35bf28}+1.59\\%$ | | test_ppo_speed | 9.3164ms | 7.7135ms | 129.6420 Ops/s | 127.1441 Ops/s | $\color{#35bf28}+1.96\\%$ | | test_reinforce_speed | 7.6507ms | 6.6209ms | 151.0363 Ops/s | 151.0772 Ops/s | $\color{#d91a1a}-0.03\\%$ | | test_iql_speed | 33.7241ms | 32.5884ms | 30.6857 Ops/s | 30.8380 Ops/s | $\color{#d91a1a}-0.49\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.6590ms | 5.0317ms | 198.7398 Ops/s | 199.7559 Ops/s | $\color{#d91a1a}-0.51\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.7987ms | 0.4920ms | 2.0324 KOps/s | 2.0296 KOps/s | $\color{#35bf28}+0.14\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6635ms | 0.4649ms | 2.1509 KOps/s | 2.1732 KOps/s | $\color{#d91a1a}-1.03\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 7.8559ms | 5.0433ms | 198.2813 Ops/s | 202.4262 Ops/s | $\color{#d91a1a}-2.05\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.3982ms | 0.4835ms | 2.0682 KOps/s | 2.1021 KOps/s | $\color{#d91a1a}-1.61\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6630ms | 0.4651ms | 2.1502 KOps/s | 2.1368 KOps/s | $\color{#35bf28}+0.63\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.4906ms | 1.7077ms | 585.5846 Ops/s | 586.3947 Ops/s | $\color{#d91a1a}-0.14\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.1717ms | 1.6179ms | 618.0666 Ops/s | 618.3568 Ops/s | $\color{#d91a1a}-0.05\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.5529ms | 5.1335ms | 194.7973 Ops/s | 199.4678 Ops/s | $\color{#d91a1a}-2.34\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8032ms | 0.6203ms | 1.6120 KOps/s | 1.5993 KOps/s | $\color{#35bf28}+0.80\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7459ms | 0.5920ms | 1.6891 KOps/s | 1.6572 KOps/s | $\color{#35bf28}+1.92\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.8862ms | 5.0043ms | 199.8266 Ops/s | 199.6231 Ops/s | $\color{#35bf28}+0.10\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.0177ms | 0.4912ms | 2.0358 KOps/s | 2.0413 KOps/s | $\color{#d91a1a}-0.27\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6173ms | 0.4624ms | 2.1625 KOps/s | 2.1151 KOps/s | $\color{#35bf28}+2.24\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.4562ms | 5.0613ms | 197.5769 Ops/s | 202.1064 Ops/s | $\color{#d91a1a}-2.24\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.6841ms | 0.4801ms | 2.0830 KOps/s | 1.5690 KOps/s | $\textbf{\color{#35bf28}+32.76\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6370ms | 0.4626ms | 2.1616 KOps/s | 2.1730 KOps/s | $\color{#d91a1a}-0.52\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 8.0594ms | 5.2035ms | 192.1796 Ops/s | 194.3638 Ops/s | $\color{#d91a1a}-1.12\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.6747ms | 0.6260ms | 1.5976 KOps/s | 1.5943 KOps/s | $\color{#35bf28}+0.20\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8002ms | 0.6050ms | 1.6529 KOps/s | 1.6381 KOps/s | $\color{#35bf28}+0.90\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1429s | 8.8687ms | 112.7563 Ops/s | 158.7217 Ops/s | $\textbf{\color{#d91a1a}-28.96\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 17.2606ms | 13.1909ms | 75.8098 Ops/s | 77.2283 Ops/s | $\color{#d91a1a}-1.84\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 4.8342ms | 1.1791ms | 848.0975 Ops/s | 831.8613 Ops/s | $\color{#35bf28}+1.95\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1213s | 6.1103ms | 163.6580 Ops/s | 120.5422 Ops/s | $\textbf{\color{#35bf28}+35.77\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 17.5930ms | 13.3162ms | 75.0966 Ops/s | 78.0055 Ops/s | $\color{#d91a1a}-3.73\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 5.1242ms | 1.2064ms | 828.8902 Ops/s | 833.1534 Ops/s | $\color{#d91a1a}-0.51\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1239s | 6.3106ms | 158.4647 Ops/s | 157.4294 Ops/s | $\color{#35bf28}+0.66\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 0.1380s | 15.8317ms | 63.1642 Ops/s | 77.1443 Ops/s | $\textbf{\color{#d91a1a}-18.12\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.8072ms | 1.2178ms | 821.1192 Ops/s | 774.3208 Ops/s | $\textbf{\color{#35bf28}+6.04\\%}$ |
github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1075s | 0.1074s | 9.3121 Ops/s | 8.6123 Ops/s | $\textbf{\color{#35bf28}+8.13\\%}$ | | test_sync | 95.7106ms | 95.0683ms | 10.5188 Ops/s | 10.5518 Ops/s | $\color{#d91a1a}-0.31\\%$ | | test_async | 0.2599s | 89.9309ms | 11.1197 Ops/s | 11.1424 Ops/s | $\color{#d91a1a}-0.20\\%$ | | test_single_pixels | 0.1195s | 0.1178s | 8.4860 Ops/s | 8.4704 Ops/s | $\color{#35bf28}+0.18\\%$ | | test_sync_pixels | 76.2088ms | 74.2845ms | 13.4618 Ops/s | 13.2639 Ops/s | $\color{#35bf28}+1.49\\%$ | | test_async_pixels | 0.1297s | 68.0015ms | 14.7056 Ops/s | 14.2634 Ops/s | $\color{#35bf28}+3.10\\%$ | | test_simple | 0.7820s | 0.7774s | 1.2864 Ops/s | 1.2405 Ops/s | $\color{#35bf28}+3.70\\%$ | | test_transformed | 1.0972s | 1.0270s | 0.9737 Ops/s | 0.9862 Ops/s | $\color{#d91a1a}-1.27\\%$ | | test_serial | 2.3305s | 2.2566s | 0.4431 Ops/s | 0.4404 Ops/s | $\color{#35bf28}+0.62\\%$ | | test_parallel | 2.0245s | 1.9573s | 0.5109 Ops/s | 0.5072 Ops/s | $\color{#35bf28}+0.74\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1138ms | 36.8171μs | 27.1613 KOps/s | 27.6324 KOps/s | $\color{#d91a1a}-1.70\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 49.0910μs | 20.8125μs | 48.0479 KOps/s | 47.6663 KOps/s | $\color{#35bf28}+0.80\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 45.7110μs | 20.6567μs | 48.4105 KOps/s | 48.9620 KOps/s | $\color{#d91a1a}-1.13\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 30.9710μs | 11.8640μs | 84.2888 KOps/s | 84.0330 KOps/s | $\color{#35bf28}+0.30\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 70.3410μs | 38.6999μs | 25.8399 KOps/s | 26.3201 KOps/s | $\color{#d91a1a}-1.82\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 54.2110μs | 22.8952μs | 43.6772 KOps/s | 43.3430 KOps/s | $\color{#35bf28}+0.77\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 49.2110μs | 22.7391μs | 43.9771 KOps/s | 44.1660 KOps/s | $\color{#d91a1a}-0.43\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 38.7310μs | 14.0804μs | 71.0206 KOps/s | 70.7044 KOps/s | $\color{#35bf28}+0.45\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 70.2110μs | 40.7406μs | 24.5456 KOps/s | 24.6153 KOps/s | $\color{#d91a1a}-0.28\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 51.4110μs | 25.3574μs | 39.4362 KOps/s | 39.1632 KOps/s | $\color{#35bf28}+0.70\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 47.2210μs | 22.4948μs | 44.4547 KOps/s | 44.1738 KOps/s | $\color{#35bf28}+0.64\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 39.5110μs | 14.0787μs | 71.0290 KOps/s | 70.8466 KOps/s | $\color{#35bf28}+0.26\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 81.7420μs | 42.8366μs | 23.3445 KOps/s | 23.4086 KOps/s | $\color{#d91a1a}-0.27\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 52.6410μs | 27.4632μs | 36.4124 KOps/s | 36.0178 KOps/s | $\color{#35bf28}+1.10\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 50.5500μs | 24.4408μs | 40.9153 KOps/s | 40.5790 KOps/s | $\color{#35bf28}+0.83\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 37.5800μs | 16.1810μs | 61.8007 KOps/s | 60.9312 KOps/s | $\color{#35bf28}+1.43\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 74.8120μs | 41.0581μs | 24.3557 KOps/s | 24.6223 KOps/s | $\color{#d91a1a}-1.08\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 51.7410μs | 25.3440μs | 39.4571 KOps/s | 39.2156 KOps/s | $\color{#35bf28}+0.62\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 66.1620μs | 27.0481μs | 36.9712 KOps/s | 36.7833 KOps/s | $\color{#35bf28}+0.51\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 39.7810μs | 15.8939μs | 62.9173 KOps/s | 61.7739 KOps/s | $\color{#35bf28}+1.85\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 70.6210μs | 42.9894μs | 23.2616 KOps/s | 23.3384 KOps/s | $\color{#d91a1a}-0.33\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 53.5210μs | 27.4141μs | 36.4776 KOps/s | 35.7441 KOps/s | $\color{#35bf28}+2.05\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 58.5310μs | 29.3773μs | 34.0399 KOps/s | 34.5036 KOps/s | $\color{#d91a1a}-1.34\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 37.1010μs | 17.9990μs | 55.5587 KOps/s | 54.4375 KOps/s | $\color{#35bf28}+2.06\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 4.0695ms | 45.4554μs | 21.9996 KOps/s | 22.0242 KOps/s | $\color{#d91a1a}-0.11\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 55.7610μs | 29.9497μs | 33.3894 KOps/s | 33.0464 KOps/s | $\color{#35bf28}+1.04\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 57.1910μs | 28.8829μs | 34.6225 KOps/s | 34.3310 KOps/s | $\color{#35bf28}+0.85\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 39.9800μs | 18.0551μs | 55.3859 KOps/s | 53.4838 KOps/s | $\color{#35bf28}+3.56\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 74.9910μs | 47.1466μs | 21.2104 KOps/s | 21.4059 KOps/s | $\color{#d91a1a}-0.91\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 57.8710μs | 31.4935μs | 31.7525 KOps/s | 31.3332 KOps/s | $\color{#35bf28}+1.34\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 59.9610μs | 30.8894μs | 32.3736 KOps/s | 32.2089 KOps/s | $\color{#35bf28}+0.51\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 43.6510μs | 20.1203μs | 49.7011 KOps/s | 49.6607 KOps/s | $\color{#35bf28}+0.08\\%$ | | test_values[generalized_advantage_estimate-True-True] | 25.7562ms | 24.8874ms | 40.1810 Ops/s | 40.5831 Ops/s | $\color{#d91a1a}-0.99\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 88.9029ms | 2.6808ms | 373.0231 Ops/s | 368.5821 Ops/s | $\color{#35bf28}+1.20\\%$ | | test_values[td0_return_estimate-False-False] | 95.3420μs | 66.4962μs | 15.0385 KOps/s | 15.2836 KOps/s | $\color{#d91a1a}-1.60\\%$ | | test_values[td1_return_estimate-False-False] | 56.0804ms | 55.9034ms | 17.8880 Ops/s | 18.3704 Ops/s | $\color{#d91a1a}-2.63\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.4010ms | 1.0922ms | 915.6097 Ops/s | 926.0953 Ops/s | $\color{#d91a1a}-1.13\\%$ | | test_values[td_lambda_return_estimate-True-False] | 89.0105ms | 88.6960ms | 11.2745 Ops/s | 11.5543 Ops/s | $\color{#d91a1a}-2.42\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.4063ms | 1.0913ms | 916.3407 Ops/s | 924.7071 Ops/s | $\color{#d91a1a}-0.90\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 25.2420ms | 25.1176ms | 39.8128 Ops/s | 40.0900 Ops/s | $\color{#d91a1a}-0.69\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9658ms | 0.7251ms | 1.3791 KOps/s | 1.3775 KOps/s | $\color{#35bf28}+0.11\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7713ms | 0.6800ms | 1.4706 KOps/s | 1.5058 KOps/s | $\color{#d91a1a}-2.34\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5117ms | 1.4744ms | 678.2519 Ops/s | 681.2599 Ops/s | $\color{#d91a1a}-0.44\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7290ms | 0.6887ms | 1.4519 KOps/s | 1.4722 KOps/s | $\color{#d91a1a}-1.38\\%$ | | test_dqn_speed | 92.8831ms | 1.5081ms | 663.0944 Ops/s | 729.7799 Ops/s | $\textbf{\color{#d91a1a}-9.14\\%}$ | | test_ddpg_speed | 3.0014ms | 2.7649ms | 361.6736 Ops/s | 360.6425 Ops/s | $\color{#35bf28}+0.29\\%$ | | test_sac_speed | 8.3128ms | 7.9576ms | 125.6662 Ops/s | 125.4412 Ops/s | $\color{#35bf28}+0.18\\%$ | | test_redq_speed | 11.7651ms | 10.2124ms | 97.9198 Ops/s | 97.6367 Ops/s | $\color{#35bf28}+0.29\\%$ | | test_redq_deprec_speed | 10.9193ms | 10.6274ms | 94.0968 Ops/s | 90.6683 Ops/s | $\color{#35bf28}+3.78\\%$ | | test_td3_speed | 8.0236ms | 7.8752ms | 126.9811 Ops/s | 126.2729 Ops/s | $\color{#35bf28}+0.56\\%$ | | test_cql_speed | 26.6279ms | 25.3368ms | 39.4682 Ops/s | 39.0879 Ops/s | $\color{#35bf28}+0.97\\%$ | | test_a2c_speed | 7.9909ms | 5.5064ms | 181.6082 Ops/s | 181.3022 Ops/s | $\color{#35bf28}+0.17\\%$ | | test_ppo_speed | 6.3914ms | 5.7677ms | 173.3802 Ops/s | 173.2688 Ops/s | $\color{#35bf28}+0.06\\%$ | | test_reinforce_speed | 4.5914ms | 4.4104ms | 226.7363 Ops/s | 226.5312 Ops/s | $\color{#35bf28}+0.09\\%$ | | test_iql_speed | 19.7724ms | 18.9078ms | 52.8883 Ops/s | 51.9272 Ops/s | $\color{#35bf28}+1.85\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.7898ms | 6.6116ms | 151.2501 Ops/s | 149.7829 Ops/s | $\color{#35bf28}+0.98\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.5016ms | 0.5157ms | 1.9393 KOps/s | 1.9383 KOps/s | $\color{#35bf28}+0.05\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7294ms | 0.4943ms | 2.0232 KOps/s | 2.0146 KOps/s | $\color{#35bf28}+0.43\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.8442ms | 6.5486ms | 152.7043 Ops/s | 152.1971 Ops/s | $\color{#35bf28}+0.33\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.9370ms | 0.5073ms | 1.9713 KOps/s | 1.9532 KOps/s | $\color{#35bf28}+0.93\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7308ms | 0.4901ms | 2.0403 KOps/s | 2.0330 KOps/s | $\color{#35bf28}+0.36\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.1323ms | 1.9718ms | 507.1485 Ops/s | 506.3003 Ops/s | $\color{#35bf28}+0.17\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.0380ms | 1.8790ms | 532.2061 Ops/s | 533.4890 Ops/s | $\color{#d91a1a}-0.24\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.9291ms | 6.7857ms | 147.3690 Ops/s | 146.1307 Ops/s | $\color{#35bf28}+0.85\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8611ms | 0.6613ms | 1.5121 KOps/s | 1.5013 KOps/s | $\color{#35bf28}+0.72\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8222ms | 0.6391ms | 1.5648 KOps/s | 1.5369 KOps/s | $\color{#35bf28}+1.82\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.7679ms | 6.6596ms | 150.1590 Ops/s | 150.2523 Ops/s | $\color{#d91a1a}-0.06\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.9712ms | 0.5158ms | 1.9388 KOps/s | 1.9351 KOps/s | $\color{#35bf28}+0.19\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6759ms | 0.4930ms | 2.0282 KOps/s | 1.5702 KOps/s | $\textbf{\color{#35bf28}+29.17\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.9995ms | 6.5369ms | 152.9766 Ops/s | 150.2508 Ops/s | $\color{#35bf28}+1.81\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6397ms | 0.5118ms | 1.9541 KOps/s | 1.9621 KOps/s | $\color{#d91a1a}-0.41\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7045ms | 0.4905ms | 2.0386 KOps/s | 2.0370 KOps/s | $\color{#35bf28}+0.08\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.8741ms | 6.7822ms | 147.4444 Ops/s | 147.1242 Ops/s | $\color{#35bf28}+0.22\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9315ms | 0.6637ms | 1.5067 KOps/s | 1.2100 KOps/s | $\textbf{\color{#35bf28}+24.52\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.3213ms | 0.6462ms | 1.5476 KOps/s | 1.5594 KOps/s | $\color{#d91a1a}-0.76\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1240s | 7.5342ms | 132.7278 Ops/s | 131.8571 Ops/s | $\color{#35bf28}+0.66\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.8612ms | 16.4143ms | 60.9224 Ops/s | 61.2988 Ops/s | $\color{#d91a1a}-0.61\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.3471ms | 1.2456ms | 802.8161 Ops/s | 726.6237 Ops/s | $\textbf{\color{#35bf28}+10.49\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1265s | 7.6243ms | 131.1590 Ops/s | 132.7684 Ops/s | $\color{#d91a1a}-1.21\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1360s | 18.7708ms | 53.2743 Ops/s | 61.2286 Ops/s | $\textbf{\color{#d91a1a}-12.99\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 7.4184ms | 1.3560ms | 737.4794 Ops/s | 842.7107 Ops/s | $\textbf{\color{#d91a1a}-12.49\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1254s | 7.7974ms | 128.2485 Ops/s | 129.3754 Ops/s | $\color{#d91a1a}-0.87\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 19.3081ms | 16.4506ms | 60.7880 Ops/s | 61.1869 Ops/s | $\color{#d91a1a}-0.65\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.5355ms | 1.3799ms | 724.6974 Ops/s | 661.0872 Ops/s | $\textbf{\color{#35bf28}+9.62\\%}$ |