search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.19k stars 289 forks source link

[BugFix] `done = done | truncated` in collector #2333

Closed vmoens closed 1 month ago

vmoens commented 1 month ago

Solved #2329

@matteobettini

pytorch-bot[bot] commented 1 month ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2333

Note: Links to docs will display an error until the docs builds have been completed.

:x: 3 New Failures, 1 Pending

As of commit 57e40513d45057d35b15f167f388481b99ef8938 with merge base 0063741839a3e5e1a527947945494d54f91bc629 (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2333#28063134100) ([gh](https://github.com/pytorch/rl/actions/runs/10149065169/job/28063134100)) `RuntimeError: Command docker exec -t 81fade4c6c4e250701392a51c31e44ca9dccde7d122853e1f001d07ea7f81476 /exec failed with exit code 139` * [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2333#28063137852) ([gh](https://github.com/pytorch/rl/actions/runs/10149065157/job/28063137852)) `test/test_tensordictmodules.py::TestGRUModule::test_gru_vmap_complex_model` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2333#28063138276) ([gh](https://github.com/pytorch/rl/actions/runs/10149065161/job/28063138276)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 58.2682ms | 57.7201ms | 17.3250 Ops/s | 17.3332 Ops/s | $\color{#d91a1a}-0.05\\%$ | | test_sync | 38.9921ms | 32.7087ms | 30.5729 Ops/s | 31.0508 Ops/s | $\color{#d91a1a}-1.54\\%$ | | test_async | 54.0355ms | 31.1677ms | 32.0845 Ops/s | 32.2214 Ops/s | $\color{#d91a1a}-0.43\\%$ | | test_simple | 0.5164s | 0.4216s | 2.3720 Ops/s | 2.5443 Ops/s | $\textbf{\color{#d91a1a}-6.77\\%}$ | | test_transformed | 0.6449s | 0.5722s | 1.7477 Ops/s | 1.8376 Ops/s | $\color{#d91a1a}-4.89\\%$ | | test_serial | 1.3262s | 1.2627s | 0.7920 Ops/s | 0.7980 Ops/s | $\color{#d91a1a}-0.76\\%$ | | test_parallel | 1.1714s | 1.1011s | 0.9082 Ops/s | 0.9054 Ops/s | $\color{#35bf28}+0.31\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1533ms | 24.5134μs | 40.7940 KOps/s | 40.4697 KOps/s | $\color{#35bf28}+0.80\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 48.4510μs | 14.4725μs | 69.0966 KOps/s | 69.4328 KOps/s | $\color{#d91a1a}-0.48\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 42.5410μs | 14.2628μs | 70.1127 KOps/s | 69.9997 KOps/s | $\color{#35bf28}+0.16\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 52.6600μs | 8.3447μs | 119.8366 KOps/s | 120.2243 KOps/s | $\color{#d91a1a}-0.32\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 58.8910μs | 26.1916μs | 38.1802 KOps/s | 38.0864 KOps/s | $\color{#35bf28}+0.25\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 43.3510μs | 16.0956μs | 62.1289 KOps/s | 62.4919 KOps/s | $\color{#d91a1a}-0.58\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 52.7800μs | 15.6647μs | 63.8377 KOps/s | 63.6273 KOps/s | $\color{#35bf28}+0.33\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 36.0680μs | 9.8333μs | 101.6951 KOps/s | 102.1530 KOps/s | $\color{#d91a1a}-0.45\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 63.8910μs | 27.6803μs | 36.1268 KOps/s | 36.2084 KOps/s | $\color{#d91a1a}-0.23\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 54.0020μs | 17.5289μs | 57.0485 KOps/s | 57.0801 KOps/s | $\color{#d91a1a}-0.06\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 42.0500μs | 15.7255μs | 63.5909 KOps/s | 63.3003 KOps/s | $\color{#35bf28}+0.46\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 35.0260μs | 9.8736μs | 101.2806 KOps/s | 96.3453 KOps/s | $\textbf{\color{#35bf28}+5.12\\%}$ | | test_step_mdp_speed[True-False-False-True-True] | 58.9510μs | 29.0794μs | 34.3885 KOps/s | 33.7138 KOps/s | $\color{#35bf28}+2.00\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 51.1360μs | 18.8699μs | 52.9945 KOps/s | 52.7756 KOps/s | $\color{#35bf28}+0.41\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 53.0100μs | 17.0346μs | 58.7041 KOps/s | 57.9211 KOps/s | $\color{#35bf28}+1.35\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 34.6050μs | 11.2528μs | 88.8671 KOps/s | 89.6627 KOps/s | $\color{#d91a1a}-0.89\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 56.7570μs | 27.7607μs | 36.0221 KOps/s | 35.6061 KOps/s | $\color{#35bf28}+1.17\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 57.7590μs | 17.4229μs | 57.3957 KOps/s | 56.5673 KOps/s | $\color{#35bf28}+1.46\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 46.0560μs | 18.7682μs | 53.2817 KOps/s | 54.5428 KOps/s | $\color{#d91a1a}-2.31\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 63.4910μs | 11.1461μs | 89.7174 KOps/s | 87.6156 KOps/s | $\color{#35bf28}+2.40\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 66.1340μs | 29.1966μs | 34.2505 KOps/s | 34.2388 KOps/s | $\color{#35bf28}+0.03\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 46.0370μs | 18.9839μs | 52.6761 KOps/s | 53.2415 KOps/s | $\color{#d91a1a}-1.06\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 67.6670μs | 19.9676μs | 50.0811 KOps/s | 51.1037 KOps/s | $\color{#d91a1a}-2.00\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 40.8170μs | 12.5893μs | 79.4324 KOps/s | 81.6348 KOps/s | $\color{#d91a1a}-2.70\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.1869ms | 30.8881μs | 32.3749 KOps/s | 30.7064 KOps/s | $\textbf{\color{#35bf28}+5.43\\%}$ | | test_step_mdp_speed[False-False-True-True-False] | 54.1720μs | 20.3621μs | 49.1108 KOps/s | 49.1508 KOps/s | $\color{#d91a1a}-0.08\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 83.5980μs | 19.7299μs | 50.6845 KOps/s | 51.3579 KOps/s | $\color{#d91a1a}-1.31\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 38.6130μs | 12.5734μs | 79.5331 KOps/s | 72.8458 KOps/s | $\textbf{\color{#35bf28}+9.18\\%}$ | | test_step_mdp_speed[False-False-False-True-True] | 86.4930μs | 31.7795μs | 31.4669 KOps/s | 29.5483 KOps/s | $\textbf{\color{#35bf28}+6.49\\%}$ | | test_step_mdp_speed[False-False-False-True-False] | 46.8890μs | 21.5778μs | 46.3439 KOps/s | 46.0473 KOps/s | $\color{#35bf28}+0.64\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 87.9060μs | 21.0941μs | 47.4067 KOps/s | 47.6777 KOps/s | $\color{#d91a1a}-0.57\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 49.6330μs | 13.6880μs | 73.0568 KOps/s | 72.8544 KOps/s | $\color{#35bf28}+0.28\\%$ | | test_values[generalized_advantage_estimate-True-True] | 10.0118ms | 9.5491ms | 104.7215 Ops/s | 106.6554 Ops/s | $\color{#d91a1a}-1.81\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 38.7313ms | 35.5951ms | 28.0938 Ops/s | 28.0843 Ops/s | $\color{#35bf28}+0.03\\%$ | | test_values[td0_return_estimate-False-False] | 0.2004ms | 0.1792ms | 5.5808 KOps/s | 5.9463 KOps/s | $\textbf{\color{#d91a1a}-6.15\\%}$ | | test_values[td1_return_estimate-False-False] | 39.3363ms | 24.5050ms | 40.8079 Ops/s | 43.3879 Ops/s | $\textbf{\color{#d91a1a}-5.95\\%}$ | | test_values[vec_td1_return_estimate-False-False] | 37.3837ms | 35.6434ms | 28.0557 Ops/s | 28.0238 Ops/s | $\color{#35bf28}+0.11\\%$ | | test_values[td_lambda_return_estimate-True-False] | 34.8344ms | 34.2007ms | 29.2392 Ops/s | 29.9063 Ops/s | $\color{#d91a1a}-2.23\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 38.5814ms | 35.7828ms | 27.9464 Ops/s | 28.0594 Ops/s | $\color{#d91a1a}-0.40\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.4802ms | 8.2883ms | 120.6524 Ops/s | 123.8882 Ops/s | $\color{#d91a1a}-2.61\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.4801ms | 2.0209ms | 494.8219 Ops/s | 498.7956 Ops/s | $\color{#d91a1a}-0.80\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4251ms | 0.3535ms | 2.8291 KOps/s | 2.8889 KOps/s | $\color{#d91a1a}-2.07\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 49.9546ms | 47.3934ms | 21.1000 Ops/s | 21.5967 Ops/s | $\color{#d91a1a}-2.30\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.8790ms | 3.0298ms | 330.0519 Ops/s | 329.1609 Ops/s | $\color{#35bf28}+0.27\\%$ | | test_dqn_speed | 1.7689ms | 1.2972ms | 770.8745 Ops/s | 764.3089 Ops/s | $\color{#35bf28}+0.86\\%$ | | test_ddpg_speed | 3.4857ms | 2.7020ms | 370.0926 Ops/s | 362.4203 Ops/s | $\color{#35bf28}+2.12\\%$ | | test_sac_speed | 9.5527ms | 7.9834ms | 125.2600 Ops/s | 126.8541 Ops/s | $\color{#d91a1a}-1.26\\%$ | | test_redq_speed | 16.5662ms | 12.8989ms | 77.5259 Ops/s | 79.1867 Ops/s | $\color{#d91a1a}-2.10\\%$ | | test_redq_deprec_speed | 13.6242ms | 12.5082ms | 79.9475 Ops/s | 79.2655 Ops/s | $\color{#35bf28}+0.86\\%$ | | test_td3_speed | 8.0768ms | 7.8435ms | 127.4937 Ops/s | 129.2277 Ops/s | $\color{#d91a1a}-1.34\\%$ | | test_cql_speed | 36.4411ms | 35.2359ms | 28.3801 Ops/s | 28.3911 Ops/s | $\color{#d91a1a}-0.04\\%$ | | test_a2c_speed | 9.2594ms | 7.2515ms | 137.9018 Ops/s | 138.1512 Ops/s | $\color{#d91a1a}-0.18\\%$ | | test_ppo_speed | 8.1775ms | 7.4792ms | 133.7044 Ops/s | 134.4606 Ops/s | $\color{#d91a1a}-0.56\\%$ | | test_reinforce_speed | 15.9634ms | 6.6718ms | 149.8836 Ops/s | 156.1048 Ops/s | $\color{#d91a1a}-3.99\\%$ | | test_iql_speed | 33.5061ms | 31.7971ms | 31.4494 Ops/s | 31.3829 Ops/s | $\color{#35bf28}+0.21\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 7.5619ms | 4.7770ms | 209.3361 Ops/s | 208.7312 Ops/s | $\color{#35bf28}+0.29\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 93.4754ms | 0.5310ms | 1.8833 KOps/s | 2.1066 KOps/s | $\textbf{\color{#d91a1a}-10.60\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6590ms | 0.4474ms | 2.2349 KOps/s | 2.1096 KOps/s | $\textbf{\color{#35bf28}+5.94\\%}$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 7.4543ms | 4.7052ms | 212.5319 Ops/s | 210.3604 Ops/s | $\color{#35bf28}+1.03\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.3913ms | 0.4649ms | 2.1509 KOps/s | 2.1500 KOps/s | $\color{#35bf28}+0.04\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 1.3619ms | 0.4426ms | 2.2592 KOps/s | 2.2350 KOps/s | $\color{#35bf28}+1.08\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 1.8422ms | 1.6662ms | 600.1608 Ops/s | 602.7514 Ops/s | $\color{#d91a1a}-0.43\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2798ms | 1.5900ms | 628.9285 Ops/s | 635.9370 Ops/s | $\color{#d91a1a}-1.10\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.1389ms | 4.8717ms | 205.2688 Ops/s | 203.0860 Ops/s | $\color{#35bf28}+1.07\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.4456ms | 0.6103ms | 1.6385 KOps/s | 1.6494 KOps/s | $\color{#d91a1a}-0.66\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8024ms | 0.5857ms | 1.7073 KOps/s | 1.7264 KOps/s | $\color{#d91a1a}-1.10\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 7.4454ms | 4.8148ms | 207.6935 Ops/s | 208.4881 Ops/s | $\color{#d91a1a}-0.38\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.3792ms | 0.4731ms | 2.1136 KOps/s | 2.0951 KOps/s | $\color{#35bf28}+0.88\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6176ms | 0.4516ms | 2.2143 KOps/s | 2.1986 KOps/s | $\color{#35bf28}+0.71\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.9827ms | 4.7814ms | 209.1434 Ops/s | 209.3904 Ops/s | $\color{#d91a1a}-0.12\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.9239ms | 0.4685ms | 2.1343 KOps/s | 2.1245 KOps/s | $\color{#35bf28}+0.46\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6170ms | 0.4489ms | 2.2277 KOps/s | 2.1927 KOps/s | $\color{#35bf28}+1.60\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.0518ms | 4.9947ms | 200.2106 Ops/s | 204.4546 Ops/s | $\color{#d91a1a}-2.08\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.5261ms | 0.6148ms | 1.6266 KOps/s | 1.6378 KOps/s | $\color{#d91a1a}-0.68\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8607ms | 0.5839ms | 1.7125 KOps/s | 1.5953 KOps/s | $\textbf{\color{#35bf28}+7.35\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1397s | 6.4513ms | 155.0077 Ops/s | 159.7449 Ops/s | $\color{#d91a1a}-2.97\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 16.8486ms | 12.8197ms | 78.0048 Ops/s | 77.8088 Ops/s | $\color{#35bf28}+0.25\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 5.2566ms | 1.1993ms | 833.7925 Ops/s | 907.4908 Ops/s | $\textbf{\color{#d91a1a}-8.12\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1100s | 7.9706ms | 125.4618 Ops/s | 174.8311 Ops/s | $\textbf{\color{#d91a1a}-28.24\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 22.6082ms | 12.9894ms | 76.9860 Ops/s | 78.5196 Ops/s | $\color{#d91a1a}-1.95\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 4.6928ms | 1.2049ms | 829.9340 Ops/s | 813.4756 Ops/s | $\color{#35bf28}+2.02\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1085s | 5.9980ms | 166.7234 Ops/s | 122.3864 Ops/s | $\textbf{\color{#35bf28}+36.23\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 17.3861ms | 13.0238ms | 76.7824 Ops/s | 78.0535 Ops/s | $\color{#d91a1a}-1.63\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.0036ms | 1.2734ms | 785.2990 Ops/s | 802.7707 Ops/s | $\color{#d91a1a}-2.18\\%$ |
github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1071s | 0.1063s | 9.4080 Ops/s | 9.2346 Ops/s | $\color{#35bf28}+1.88\\%$ | | test_sync | 95.8157ms | 95.3427ms | 10.4885 Ops/s | 10.5148 Ops/s | $\color{#d91a1a}-0.25\\%$ | | test_async | 0.1795s | 90.8605ms | 11.0059 Ops/s | 11.1272 Ops/s | $\color{#d91a1a}-1.09\\%$ | | test_single_pixels | 0.1176s | 0.1170s | 8.5472 Ops/s | 8.5528 Ops/s | $\color{#d91a1a}-0.06\\%$ | | test_sync_pixels | 76.2023ms | 73.2369ms | 13.6543 Ops/s | 13.2133 Ops/s | $\color{#35bf28}+3.34\\%$ | | test_async_pixels | 0.2288s | 72.0934ms | 13.8709 Ops/s | 14.2168 Ops/s | $\color{#d91a1a}-2.43\\%$ | | test_simple | 0.7655s | 0.7644s | 1.3083 Ops/s | 1.3055 Ops/s | $\color{#35bf28}+0.21\\%$ | | test_transformed | 1.1005s | 1.0165s | 0.9838 Ops/s | 0.9865 Ops/s | $\color{#d91a1a}-0.28\\%$ | | test_serial | 2.2776s | 2.2006s | 0.4544 Ops/s | 0.4550 Ops/s | $\color{#d91a1a}-0.13\\%$ | | test_parallel | 2.0116s | 1.9607s | 0.5100 Ops/s | 0.5120 Ops/s | $\color{#d91a1a}-0.38\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.2195ms | 36.1838μs | 27.6367 KOps/s | 27.3084 KOps/s | $\color{#35bf28}+1.20\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 0.2131ms | 20.3721μs | 49.0867 KOps/s | 48.6954 KOps/s | $\color{#35bf28}+0.80\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 0.2150ms | 19.9546μs | 50.1137 KOps/s | 48.5481 KOps/s | $\color{#35bf28}+3.22\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 32.7210μs | 11.5235μs | 86.7794 KOps/s | 85.6085 KOps/s | $\color{#35bf28}+1.37\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 99.5920μs | 37.6585μs | 26.5544 KOps/s | 26.0437 KOps/s | $\color{#35bf28}+1.96\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 0.2227ms | 22.5782μs | 44.2905 KOps/s | 44.2848 KOps/s | $\color{#35bf28}+0.01\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 41.6610μs | 22.1207μs | 45.2065 KOps/s | 43.3706 KOps/s | $\color{#35bf28}+4.23\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 32.2000μs | 13.6518μs | 73.2504 KOps/s | 72.2638 KOps/s | $\color{#35bf28}+1.37\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 73.4710μs | 39.9728μs | 25.0170 KOps/s | 24.6370 KOps/s | $\color{#35bf28}+1.54\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 41.5800μs | 24.8978μs | 40.1641 KOps/s | 40.2794 KOps/s | $\color{#d91a1a}-0.29\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 87.8220μs | 22.1144μs | 45.2194 KOps/s | 43.8972 KOps/s | $\color{#35bf28}+3.01\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 37.1710μs | 13.6391μs | 73.3187 KOps/s | 71.7995 KOps/s | $\color{#35bf28}+2.12\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 73.0710μs | 41.7556μs | 23.9489 KOps/s | 23.3529 KOps/s | $\color{#35bf28}+2.55\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 51.4510μs | 26.8641μs | 37.2244 KOps/s | 37.0504 KOps/s | $\color{#35bf28}+0.47\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 0.2009ms | 24.0593μs | 41.5639 KOps/s | 40.0056 KOps/s | $\color{#35bf28}+3.90\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 72.3110μs | 15.7708μs | 63.4083 KOps/s | 62.5053 KOps/s | $\color{#35bf28}+1.44\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 67.1210μs | 39.9590μs | 25.0257 KOps/s | 24.6529 KOps/s | $\color{#35bf28}+1.51\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 42.6410μs | 24.6531μs | 40.5628 KOps/s | 40.4785 KOps/s | $\color{#35bf28}+0.21\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 50.6310μs | 26.5312μs | 37.6915 KOps/s | 36.8709 KOps/s | $\color{#35bf28}+2.23\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 0.1223ms | 15.6992μs | 63.6975 KOps/s | 63.0433 KOps/s | $\color{#35bf28}+1.04\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 66.9010μs | 42.0162μs | 23.8003 KOps/s | 23.4188 KOps/s | $\color{#35bf28}+1.63\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 45.2610μs | 27.0456μs | 36.9746 KOps/s | 37.2539 KOps/s | $\color{#d91a1a}-0.75\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 54.9810μs | 28.6258μs | 34.9335 KOps/s | 34.7142 KOps/s | $\color{#35bf28}+0.63\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 36.5010μs | 17.7656μs | 56.2884 KOps/s | 55.5137 KOps/s | $\color{#35bf28}+1.40\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 4.3861ms | 44.9561μs | 22.2439 KOps/s | 22.2065 KOps/s | $\color{#35bf28}+0.17\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 55.0410μs | 29.4658μs | 33.9376 KOps/s | 34.2954 KOps/s | $\color{#d91a1a}-1.04\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 85.8120μs | 28.4378μs | 35.1645 KOps/s | 34.5637 KOps/s | $\color{#35bf28}+1.74\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 41.5110μs | 17.8377μs | 56.0609 KOps/s | 54.7608 KOps/s | $\color{#35bf28}+2.37\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 64.5510μs | 46.0800μs | 21.7014 KOps/s | 21.6514 KOps/s | $\color{#35bf28}+0.23\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 70.9810μs | 30.8750μs | 32.3887 KOps/s | 32.1790 KOps/s | $\color{#35bf28}+0.65\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 0.1317ms | 30.3142μs | 32.9879 KOps/s | 32.8030 KOps/s | $\color{#35bf28}+0.56\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 40.0300μs | 19.6086μs | 50.9980 KOps/s | 49.7584 KOps/s | $\color{#35bf28}+2.49\\%$ | | test_values[generalized_advantage_estimate-True-True] | 24.3704ms | 23.7408ms | 42.1215 Ops/s | 41.9824 Ops/s | $\color{#35bf28}+0.33\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 91.1016ms | 2.7150ms | 368.3268 Ops/s | 353.1535 Ops/s | $\color{#35bf28}+4.30\\%$ | | test_values[td0_return_estimate-False-False] | 89.8220μs | 64.7722μs | 15.4387 KOps/s | 15.6809 KOps/s | $\color{#d91a1a}-1.54\\%$ | | test_values[td1_return_estimate-False-False] | 54.0862ms | 53.1975ms | 18.7979 Ops/s | 18.8854 Ops/s | $\color{#d91a1a}-0.46\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.3939ms | 1.0764ms | 928.9844 Ops/s | 927.2865 Ops/s | $\color{#35bf28}+0.18\\%$ | | test_values[td_lambda_return_estimate-True-False] | 85.9313ms | 84.8368ms | 11.7873 Ops/s | 11.7385 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.4030ms | 1.0817ms | 924.4521 Ops/s | 934.5505 Ops/s | $\color{#d91a1a}-1.08\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 24.5101ms | 23.7497ms | 42.1058 Ops/s | 42.3697 Ops/s | $\color{#d91a1a}-0.62\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9553ms | 0.7116ms | 1.4054 KOps/s | 1.3706 KOps/s | $\color{#35bf28}+2.54\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.8229ms | 0.6632ms | 1.5079 KOps/s | 1.5166 KOps/s | $\color{#d91a1a}-0.57\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.6675ms | 1.4562ms | 686.6987 Ops/s | 686.5824 Ops/s | $\color{#35bf28}+0.02\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.8444ms | 0.6735ms | 1.4849 KOps/s | 1.4892 KOps/s | $\color{#d91a1a}-0.29\\%$ | | test_dqn_speed | 8.1540ms | 1.3764ms | 726.5411 Ops/s | 733.7462 Ops/s | $\color{#d91a1a}-0.98\\%$ | | test_ddpg_speed | 3.1092ms | 2.7652ms | 361.6407 Ops/s | 365.1733 Ops/s | $\color{#d91a1a}-0.97\\%$ | | test_sac_speed | 8.2861ms | 7.9035ms | 126.5258 Ops/s | 126.0114 Ops/s | $\color{#35bf28}+0.41\\%$ | | test_redq_speed | 12.4911ms | 10.6664ms | 93.7526 Ops/s | 94.8880 Ops/s | $\color{#d91a1a}-1.20\\%$ | | test_redq_deprec_speed | 11.4264ms | 10.8077ms | 92.5267 Ops/s | 88.9475 Ops/s | $\color{#35bf28}+4.02\\%$ | | test_td3_speed | 7.9473ms | 7.8803ms | 126.8991 Ops/s | 127.4314 Ops/s | $\color{#d91a1a}-0.42\\%$ | | test_cql_speed | 27.5194ms | 25.9982ms | 38.4642 Ops/s | 38.3066 Ops/s | $\color{#35bf28}+0.41\\%$ | | test_a2c_speed | 6.0266ms | 5.6549ms | 176.8383 Ops/s | 177.4844 Ops/s | $\color{#d91a1a}-0.36\\%$ | | test_ppo_speed | 6.7246ms | 5.9817ms | 167.1762 Ops/s | 168.2243 Ops/s | $\color{#d91a1a}-0.62\\%$ | | test_reinforce_speed | 4.9117ms | 4.5227ms | 221.1072 Ops/s | 219.9169 Ops/s | $\color{#35bf28}+0.54\\%$ | | test_iql_speed | 20.7447ms | 20.0521ms | 49.8702 Ops/s | 50.1454 Ops/s | $\color{#d91a1a}-0.55\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.9823ms | 6.5893ms | 151.7619 Ops/s | 149.5230 Ops/s | $\color{#35bf28}+1.50\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.1753ms | 0.5290ms | 1.8905 KOps/s | 1.8583 KOps/s | $\color{#35bf28}+1.73\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.8393ms | 0.5095ms | 1.9627 KOps/s | 1.9297 KOps/s | $\color{#35bf28}+1.71\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.9628ms | 6.5387ms | 152.9347 Ops/s | 152.5440 Ops/s | $\color{#35bf28}+0.26\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.9910ms | 0.5238ms | 1.9093 KOps/s | 1.8975 KOps/s | $\color{#35bf28}+0.62\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7005ms | 0.5028ms | 1.9887 KOps/s | 1.9707 KOps/s | $\color{#35bf28}+0.92\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.3009ms | 2.0228ms | 494.3678 Ops/s | 495.0281 Ops/s | $\color{#d91a1a}-0.13\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2142ms | 1.9241ms | 519.7203 Ops/s | 514.9528 Ops/s | $\color{#35bf28}+0.93\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 7.0410ms | 6.7481ms | 148.1900 Ops/s | 147.9929 Ops/s | $\color{#35bf28}+0.13\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.1366s | 0.8023ms | 1.2464 KOps/s | 1.2455 KOps/s | $\color{#35bf28}+0.08\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 1.0166ms | 0.6886ms | 1.4523 KOps/s | 1.4967 KOps/s | $\color{#d91a1a}-2.97\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 7.0381ms | 6.6104ms | 151.2775 Ops/s | 146.3002 Ops/s | $\color{#35bf28}+3.40\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.2700ms | 0.5319ms | 1.8800 KOps/s | 1.8813 KOps/s | $\color{#d91a1a}-0.07\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7368ms | 0.5163ms | 1.9368 KOps/s | 1.9557 KOps/s | $\color{#d91a1a}-0.97\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 7.2319ms | 6.5732ms | 152.1335 Ops/s | 152.5530 Ops/s | $\color{#d91a1a}-0.28\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.1452s | 0.7296ms | 1.3707 KOps/s | 1.8872 KOps/s | $\textbf{\color{#d91a1a}-27.37\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7720ms | 0.5053ms | 1.9789 KOps/s | 1.9393 KOps/s | $\color{#35bf28}+2.04\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 7.2556ms | 6.7942ms | 147.1837 Ops/s | 146.2944 Ops/s | $\color{#35bf28}+0.61\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8675ms | 0.6864ms | 1.4570 KOps/s | 1.4685 KOps/s | $\color{#d91a1a}-0.79\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8971ms | 0.6710ms | 1.4903 KOps/s | 1.4941 KOps/s | $\color{#d91a1a}-0.26\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1364s | 7.8758ms | 126.9717 Ops/s | 121.4874 Ops/s | $\color{#35bf28}+4.51\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.4505ms | 15.8457ms | 63.1087 Ops/s | 62.4643 Ops/s | $\color{#35bf28}+1.03\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 7.7046ms | 1.4145ms | 706.9424 Ops/s | 779.2424 Ops/s | $\textbf{\color{#d91a1a}-9.28\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1398s | 10.6092ms | 94.2582 Ops/s | 125.6071 Ops/s | $\textbf{\color{#d91a1a}-24.96\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 18.5954ms | 15.8572ms | 63.0627 Ops/s | 53.4797 Ops/s | $\textbf{\color{#35bf28}+17.92\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.4351ms | 1.2131ms | 824.3160 Ops/s | 793.4239 Ops/s | $\color{#35bf28}+3.89\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1367s | 8.0768ms | 123.8109 Ops/s | 122.9382 Ops/s | $\color{#35bf28}+0.71\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.9675ms | 16.0323ms | 62.3743 Ops/s | 61.6939 Ops/s | $\color{#35bf28}+1.10\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 7.7326ms | 1.5524ms | 644.1825 Ops/s | 661.1605 Ops/s | $\color{#d91a1a}-2.57\\%$ |