search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.01k stars 269 forks source link

[Quality] Warn if the sampler is not prioritized but update_priority is called #2253

Closed vmoens closed 1 week ago

pytorch-bot[bot] commented 1 week ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2253

Note: Links to docs will display an error until the docs builds have been completed.

:x: 3 New Failures

As of commit aa2a5e60484189a59b31b234a07fb0aa61e36e89 with merge base 353c897f73f7d20b09bd6c6d491cf288bbac9e19 (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2253#26764145718) ([gh](https://github.com/pytorch/rl/actions/runs/9698185687/job/26764145718)) `RuntimeError: Command docker exec -t 9506590e24a1db4a4d6a207ae442018a550c96502abc2dc5dcf5d0945af9a79b /exec failed with exit code 139` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2253#26764151181) ([gh](https://github.com/pytorch/rl/actions/runs/9698185690/job/26764151181)) `RuntimeError: Command docker exec -t da8cbd0e56e7965f2e64347d799654563237e59013823b3e63bf5d9b455ebba6 /exec failed with exit code 1` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2253#26764144454) ([gh](https://github.com/pytorch/rl/actions/runs/9698185674/job/26764144454)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 week ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}11$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1100s | 58.6114ms | 17.0615 Ops/s | 18.0410 Ops/s | $\textbf{\color{#d91a1a}-5.43\\%}$ | | test_sync | 34.6252ms | 31.0398ms | 32.2167 Ops/s | 32.5734 Ops/s | $\color{#d91a1a}-1.09\\%$ | | test_async | 55.8387ms | 27.7699ms | 36.0103 Ops/s | 34.9048 Ops/s | $\color{#35bf28}+3.17\\%$ | | test_simple | 0.3778s | 0.3771s | 2.6520 Ops/s | 2.5399 Ops/s | $\color{#35bf28}+4.41\\%$ | | test_transformed | 0.5739s | 0.5443s | 1.8374 Ops/s | 1.8898 Ops/s | $\color{#d91a1a}-2.77\\%$ | | test_serial | 1.3428s | 1.2892s | 0.7757 Ops/s | 0.7917 Ops/s | $\color{#d91a1a}-2.02\\%$ | | test_parallel | 1.1345s | 1.0743s | 0.9308 Ops/s | 0.9246 Ops/s | $\color{#35bf28}+0.67\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1910ms | 22.4699μs | 44.5039 KOps/s | 43.3388 KOps/s | $\color{#35bf28}+2.69\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 60.7230μs | 13.1871μs | 75.8320 KOps/s | 73.6839 KOps/s | $\color{#35bf28}+2.92\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 45.2350μs | 13.1940μs | 75.7919 KOps/s | 73.8093 KOps/s | $\color{#35bf28}+2.69\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 45.0840μs | 7.6477μs | 130.7575 KOps/s | 122.3894 KOps/s | $\textbf{\color{#35bf28}+6.84\\%}$ | | test_step_mdp_speed[True-True-False-True-True] | 62.0150μs | 24.0756μs | 41.5358 KOps/s | 40.7960 KOps/s | $\color{#35bf28}+1.81\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 52.7480μs | 14.5798μs | 68.5880 KOps/s | 66.8435 KOps/s | $\color{#35bf28}+2.61\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 51.1050μs | 14.5252μs | 68.8460 KOps/s | 66.9065 KOps/s | $\color{#35bf28}+2.90\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 45.1540μs | 8.9369μs | 111.8956 KOps/s | 106.7197 KOps/s | $\color{#35bf28}+4.85\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 61.4040μs | 25.4017μs | 39.3675 KOps/s | 38.4331 KOps/s | $\color{#35bf28}+2.43\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 53.0790μs | 15.9839μs | 62.5628 KOps/s | 61.0990 KOps/s | $\color{#35bf28}+2.40\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 41.7170μs | 14.5438μs | 68.7577 KOps/s | 67.2406 KOps/s | $\color{#35bf28}+2.26\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 45.8760μs | 9.0380μs | 110.6441 KOps/s | 107.5192 KOps/s | $\color{#35bf28}+2.91\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 63.0980μs | 26.5088μs | 37.7234 KOps/s | 36.9575 KOps/s | $\color{#35bf28}+2.07\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 79.8780μs | 16.8572μs | 59.3218 KOps/s | 56.7304 KOps/s | $\color{#35bf28}+4.57\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 52.1970μs | 15.6887μs | 63.7403 KOps/s | 61.4982 KOps/s | $\color{#35bf28}+3.65\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 36.3580μs | 10.2196μs | 97.8515 KOps/s | 94.5521 KOps/s | $\color{#35bf28}+3.49\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 59.9420μs | 25.5981μs | 39.0653 KOps/s | 38.7478 KOps/s | $\color{#35bf28}+0.82\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 41.3470μs | 16.0348μs | 62.3643 KOps/s | 61.0350 KOps/s | $\color{#35bf28}+2.18\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 55.2120μs | 16.9981μs | 58.8301 KOps/s | 58.4039 KOps/s | $\color{#35bf28}+0.73\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 48.7610μs | 10.3110μs | 96.9838 KOps/s | 95.0614 KOps/s | $\color{#35bf28}+2.02\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 63.4280μs | 26.7698μs | 37.3555 KOps/s | 36.8604 KOps/s | $\color{#35bf28}+1.34\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 60.8340μs | 17.0447μs | 58.6693 KOps/s | 56.6097 KOps/s | $\color{#35bf28}+3.64\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 52.2570μs | 18.2049μs | 54.9302 KOps/s | 54.6448 KOps/s | $\color{#35bf28}+0.52\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 46.6170μs | 11.4191μs | 87.5727 KOps/s | 85.3180 KOps/s | $\color{#35bf28}+2.64\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 91.7100μs | 27.7887μs | 35.9858 KOps/s | 35.0357 KOps/s | $\color{#35bf28}+2.71\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 53.9310μs | 18.3490μs | 54.4990 KOps/s | 52.6639 KOps/s | $\color{#35bf28}+3.48\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 57.9780μs | 18.0529μs | 55.3927 KOps/s | 55.0865 KOps/s | $\color{#35bf28}+0.56\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 52.1170μs | 11.4498μs | 87.3376 KOps/s | 85.1040 KOps/s | $\color{#35bf28}+2.62\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 44.1020μs | 29.9442μs | 33.3954 KOps/s | 32.9643 KOps/s | $\color{#35bf28}+1.31\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 53.7190μs | 19.6079μs | 50.9998 KOps/s | 49.9746 KOps/s | $\color{#35bf28}+2.05\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 50.5740μs | 19.3908μs | 51.5709 KOps/s | 51.9415 KOps/s | $\color{#d91a1a}-0.71\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 46.4370μs | 12.6288μs | 79.1839 KOps/s | 76.5259 KOps/s | $\color{#35bf28}+3.47\\%$ | | test_values[generalized_advantage_estimate-True-True] | 12.3193ms | 9.3159ms | 107.3438 Ops/s | 104.4868 Ops/s | $\color{#35bf28}+2.73\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 49.5501ms | 34.2058ms | 29.2348 Ops/s | 28.0384 Ops/s | $\color{#35bf28}+4.27\\%$ | | test_values[td0_return_estimate-False-False] | 0.2319ms | 0.1768ms | 5.6572 KOps/s | 5.7849 KOps/s | $\color{#d91a1a}-2.21\\%$ | | test_values[td1_return_estimate-False-False] | 26.1095ms | 23.0322ms | 43.4174 Ops/s | 42.4066 Ops/s | $\color{#35bf28}+2.38\\%$ | | test_values[vec_td1_return_estimate-False-False] | 34.6583ms | 33.4271ms | 29.9159 Ops/s | 28.3059 Ops/s | $\textbf{\color{#35bf28}+5.69\\%}$ | | test_values[td_lambda_return_estimate-True-False] | 36.9840ms | 33.4296ms | 29.9136 Ops/s | 29.4243 Ops/s | $\color{#35bf28}+1.66\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 35.2806ms | 33.4688ms | 29.8786 Ops/s | 28.3427 Ops/s | $\textbf{\color{#35bf28}+5.42\\%}$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 14.5023ms | 8.4468ms | 118.3877 Ops/s | 121.3100 Ops/s | $\color{#d91a1a}-2.41\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.6084ms | 1.9897ms | 502.5957 Ops/s | 554.8951 Ops/s | $\textbf{\color{#d91a1a}-9.43\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4341ms | 0.3506ms | 2.8522 KOps/s | 2.8367 KOps/s | $\color{#35bf28}+0.55\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 49.3425ms | 44.0849ms | 22.6835 Ops/s | 23.0131 Ops/s | $\color{#d91a1a}-1.43\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.7167ms | 3.0265ms | 330.4138 Ops/s | 328.6911 Ops/s | $\color{#35bf28}+0.52\\%$ | | test_dqn_speed | 6.6964ms | 1.3504ms | 740.5047 Ops/s | 737.4164 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_ddpg_speed | 4.5881ms | 2.8581ms | 349.8829 Ops/s | 350.8306 Ops/s | $\color{#d91a1a}-0.27\\%$ | | test_sac_speed | 9.1147ms | 8.5035ms | 117.5992 Ops/s | 117.1011 Ops/s | $\color{#35bf28}+0.43\\%$ | | test_redq_speed | 15.1380ms | 13.6272ms | 73.3826 Ops/s | 66.1017 Ops/s | $\textbf{\color{#35bf28}+11.01\\%}$ | | test_redq_deprec_speed | 21.0523ms | 13.6990ms | 72.9979 Ops/s | 70.8936 Ops/s | $\color{#35bf28}+2.97\\%$ | | test_td3_speed | 10.9054ms | 8.5488ms | 116.9754 Ops/s | 116.8328 Ops/s | $\color{#35bf28}+0.12\\%$ | | test_cql_speed | 39.4819ms | 37.2365ms | 26.8554 Ops/s | 26.8198 Ops/s | $\color{#35bf28}+0.13\\%$ | | test_a2c_speed | 8.3248ms | 7.7033ms | 129.8141 Ops/s | 129.8313 Ops/s | $\color{#d91a1a}-0.01\\%$ | | test_ppo_speed | 9.1751ms | 8.0889ms | 123.6262 Ops/s | 126.7840 Ops/s | $\color{#d91a1a}-2.49\\%$ | | test_reinforce_speed | 7.8910ms | 6.7761ms | 147.5772 Ops/s | 147.3224 Ops/s | $\color{#35bf28}+0.17\\%$ | | test_iql_speed | 34.2930ms | 32.7693ms | 30.5164 Ops/s | 29.8561 Ops/s | $\color{#35bf28}+2.21\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.2264ms | 3.4601ms | 289.0080 Ops/s | 272.1849 Ops/s | $\textbf{\color{#35bf28}+6.18\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 97.9372ms | 0.5598ms | 1.7865 KOps/s | 1.7243 KOps/s | $\color{#35bf28}+3.61\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6557ms | 0.4755ms | 2.1030 KOps/s | 2.0647 KOps/s | $\color{#35bf28}+1.85\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.7998ms | 3.5051ms | 285.3024 Ops/s | 265.5916 Ops/s | $\textbf{\color{#35bf28}+7.42\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6822ms | 0.4928ms | 2.0293 KOps/s | 1.9842 KOps/s | $\color{#35bf28}+2.27\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 3.5019ms | 0.4756ms | 2.1025 KOps/s | 2.0745 KOps/s | $\color{#35bf28}+1.35\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.3509ms | 1.7214ms | 580.9311 Ops/s | 577.1449 Ops/s | $\color{#35bf28}+0.66\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.0525ms | 1.6292ms | 613.8089 Ops/s | 605.4157 Ops/s | $\color{#35bf28}+1.39\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.8954ms | 3.6317ms | 275.3545 Ops/s | 260.3710 Ops/s | $\textbf{\color{#35bf28}+5.75\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.0805ms | 0.6339ms | 1.5774 KOps/s | 1.5314 KOps/s | $\color{#35bf28}+3.01\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7838ms | 0.6102ms | 1.6389 KOps/s | 1.6037 KOps/s | $\color{#35bf28}+2.19\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.2638ms | 3.4932ms | 286.2680 Ops/s | 263.8808 Ops/s | $\textbf{\color{#35bf28}+8.48\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6167ms | 0.5010ms | 1.9959 KOps/s | 1.9235 KOps/s | $\color{#35bf28}+3.76\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.6140ms | 0.4846ms | 2.0636 KOps/s | 2.0270 KOps/s | $\color{#35bf28}+1.81\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.8618ms | 3.4891ms | 286.6044 Ops/s | 270.1607 Ops/s | $\textbf{\color{#35bf28}+6.09\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6378ms | 0.4936ms | 2.0261 KOps/s | 1.9434 KOps/s | $\color{#35bf28}+4.25\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7137ms | 0.4769ms | 2.0969 KOps/s | 2.0383 KOps/s | $\color{#35bf28}+2.87\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.4745ms | 3.6747ms | 272.1337 Ops/s | 261.5069 Ops/s | $\color{#35bf28}+4.06\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.1402ms | 0.6340ms | 1.5772 KOps/s | 1.5558 KOps/s | $\color{#35bf28}+1.38\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8441ms | 0.6145ms | 1.6273 KOps/s | 1.6221 KOps/s | $\color{#35bf28}+0.32\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1175s | 8.1513ms | 122.6791 Ops/s | 157.3461 Ops/s | $\textbf{\color{#d91a1a}-22.03\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 15.0764ms | 12.4746ms | 80.1627 Ops/s | 67.1141 Ops/s | $\textbf{\color{#35bf28}+19.44\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 4.3146ms | 1.1659ms | 857.6802 Ops/s | 954.4458 Ops/s | $\textbf{\color{#d91a1a}-10.14\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1104s | 5.8786ms | 170.1099 Ops/s | 165.0918 Ops/s | $\color{#35bf28}+3.04\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 15.2500ms | 12.4589ms | 80.2636 Ops/s | 79.2199 Ops/s | $\color{#35bf28}+1.32\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 4.2721ms | 1.1331ms | 882.5188 Ops/s | 913.9434 Ops/s | $\color{#d91a1a}-3.44\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1100s | 6.0338ms | 165.7320 Ops/s | 165.3426 Ops/s | $\color{#35bf28}+0.24\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.7444ms | 12.5966ms | 79.3866 Ops/s | 77.2690 Ops/s | $\color{#35bf28}+2.74\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.8793ms | 1.2154ms | 822.7821 Ops/s | 780.1967 Ops/s | $\textbf{\color{#35bf28}+5.46\\%}$ |
github-actions[bot] commented 1 week ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}19$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1179s | 0.1151s | 8.6851 Ops/s | 8.4161 Ops/s | $\color{#35bf28}+3.20\\%$ | | test_sync | 0.1059s | 0.1041s | 9.6090 Ops/s | 9.7763 Ops/s | $\color{#d91a1a}-1.71\\%$ | | test_async | 0.1977s | 99.7344ms | 10.0266 Ops/s | 10.0854 Ops/s | $\color{#d91a1a}-0.58\\%$ | | test_single_pixels | 0.1267s | 0.1265s | 7.9067 Ops/s | 7.7443 Ops/s | $\color{#35bf28}+2.10\\%$ | | test_sync_pixels | 84.0710ms | 80.1713ms | 12.4733 Ops/s | 12.2275 Ops/s | $\color{#35bf28}+2.01\\%$ | | test_async_pixels | 0.1515s | 67.6383ms | 14.7845 Ops/s | 14.3878 Ops/s | $\color{#35bf28}+2.76\\%$ | | test_simple | 0.8651s | 0.8030s | 1.2453 Ops/s | 1.2314 Ops/s | $\color{#35bf28}+1.13\\%$ | | test_transformed | 1.1187s | 1.0575s | 0.9456 Ops/s | 0.9175 Ops/s | $\color{#35bf28}+3.07\\%$ | | test_serial | 2.4886s | 2.4322s | 0.4112 Ops/s | 0.3966 Ops/s | $\color{#35bf28}+3.68\\%$ | | test_parallel | 2.3870s | 2.3611s | 0.4235 Ops/s | 0.4163 Ops/s | $\color{#35bf28}+1.74\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 95.2510μs | 33.9937μs | 29.4172 KOps/s | 29.0731 KOps/s | $\color{#35bf28}+1.18\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 46.7100μs | 19.8270μs | 50.4362 KOps/s | 50.0779 KOps/s | $\color{#35bf28}+0.72\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 59.0010μs | 18.8520μs | 53.0449 KOps/s | 49.7423 KOps/s | $\textbf{\color{#35bf28}+6.64\\%}$ | | test_step_mdp_speed[True-True-True-False-False] | 32.6200μs | 10.9355μs | 91.4454 KOps/s | 85.7670 KOps/s | $\textbf{\color{#35bf28}+6.62\\%}$ | | test_step_mdp_speed[True-True-False-True-True] | 66.8900μs | 35.1155μs | 28.4774 KOps/s | 26.9110 KOps/s | $\textbf{\color{#35bf28}+5.82\\%}$ | | test_step_mdp_speed[True-True-False-True-False] | 44.6200μs | 21.0646μs | 47.4731 KOps/s | 45.8749 KOps/s | $\color{#35bf28}+3.48\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 53.9700μs | 20.9218μs | 47.7970 KOps/s | 46.0508 KOps/s | $\color{#35bf28}+3.79\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 61.5710μs | 13.0283μs | 76.7559 KOps/s | 73.7241 KOps/s | $\color{#35bf28}+4.11\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 60.1510μs | 37.3128μs | 26.8005 KOps/s | 25.8831 KOps/s | $\color{#35bf28}+3.54\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 42.9710μs | 23.5502μs | 42.4625 KOps/s | 41.4666 KOps/s | $\color{#35bf28}+2.40\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 44.3200μs | 21.0295μs | 47.5522 KOps/s | 45.7104 KOps/s | $\color{#35bf28}+4.03\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 38.0410μs | 13.0448μs | 76.6592 KOps/s | 74.3419 KOps/s | $\color{#35bf28}+3.12\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 58.6500μs | 39.0960μs | 25.5780 KOps/s | 24.5785 KOps/s | $\color{#35bf28}+4.07\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 50.2610μs | 24.9124μs | 40.1406 KOps/s | 38.7374 KOps/s | $\color{#35bf28}+3.62\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 39.8910μs | 22.7019μs | 44.0491 KOps/s | 41.5924 KOps/s | $\textbf{\color{#35bf28}+5.91\\%}$ | | test_step_mdp_speed[True-False-False-False-False] | 32.7910μs | 14.6703μs | 68.1650 KOps/s | 65.4281 KOps/s | $\color{#35bf28}+4.18\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 0.1022ms | 37.3130μs | 26.8003 KOps/s | 26.0904 KOps/s | $\color{#35bf28}+2.72\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 49.3910μs | 23.1374μs | 43.2201 KOps/s | 42.0539 KOps/s | $\color{#35bf28}+2.77\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 55.7420μs | 24.3132μs | 41.1299 KOps/s | 38.4246 KOps/s | $\textbf{\color{#35bf28}+7.04\\%}$ | | test_step_mdp_speed[False-True-True-False-False] | 32.9300μs | 14.6136μs | 68.4292 KOps/s | 64.8896 KOps/s | $\textbf{\color{#35bf28}+5.45\\%}$ | | test_step_mdp_speed[False-True-False-True-True] | 70.0310μs | 38.8819μs | 25.7189 KOps/s | 24.4784 KOps/s | $\textbf{\color{#35bf28}+5.07\\%}$ | | test_step_mdp_speed[False-True-False-True-False] | 58.9110μs | 24.5091μs | 40.8012 KOps/s | 39.0232 KOps/s | $\color{#35bf28}+4.56\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 51.8900μs | 26.3288μs | 37.9812 KOps/s | 36.0384 KOps/s | $\textbf{\color{#35bf28}+5.39\\%}$ | | test_step_mdp_speed[False-True-False-False-False] | 39.5000μs | 16.3909μs | 61.0093 KOps/s | 57.8753 KOps/s | $\textbf{\color{#35bf28}+5.42\\%}$ | | test_step_mdp_speed[False-False-True-True-True] | 76.2110μs | 40.6426μs | 24.6047 KOps/s | 23.5276 KOps/s | $\color{#35bf28}+4.58\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 76.0910μs | 26.5554μs | 37.6572 KOps/s | 35.8443 KOps/s | $\textbf{\color{#35bf28}+5.06\\%}$ | | test_step_mdp_speed[False-False-True-False-True] | 55.4410μs | 26.1123μs | 38.2961 KOps/s | 35.7822 KOps/s | $\textbf{\color{#35bf28}+7.03\\%}$ | | test_step_mdp_speed[False-False-True-False-False] | 40.3010μs | 16.4647μs | 60.7360 KOps/s | 58.8473 KOps/s | $\color{#35bf28}+3.21\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 55.7800μs | 42.6930μs | 23.4230 KOps/s | 22.3695 KOps/s | $\color{#35bf28}+4.71\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 69.7410μs | 28.3682μs | 35.2507 KOps/s | 33.6047 KOps/s | $\color{#35bf28}+4.90\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 57.0110μs | 28.0257μs | 35.6815 KOps/s | 34.7646 KOps/s | $\color{#35bf28}+2.64\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 40.8410μs | 17.9077μs | 55.8419 KOps/s | 53.0067 KOps/s | $\textbf{\color{#35bf28}+5.35\\%}$ | | test_values[generalized_advantage_estimate-True-True] | 24.7184ms | 24.0594ms | 41.5638 Ops/s | 39.8876 Ops/s | $\color{#35bf28}+4.20\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 96.2370ms | 2.8153ms | 355.1980 Ops/s | 372.5015 Ops/s | $\color{#d91a1a}-4.65\\%$ | | test_values[td0_return_estimate-False-False] | 89.9710μs | 65.2359μs | 15.3290 KOps/s | 15.1081 KOps/s | $\color{#35bf28}+1.46\\%$ | | test_values[td1_return_estimate-False-False] | 56.4148ms | 54.7106ms | 18.2780 Ops/s | 17.7491 Ops/s | $\color{#35bf28}+2.98\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.2884ms | 1.0710ms | 933.7398 Ops/s | 921.5710 Ops/s | $\color{#35bf28}+1.32\\%$ | | test_values[td_lambda_return_estimate-True-False] | 89.0448ms | 86.5745ms | 11.5507 Ops/s | 11.2469 Ops/s | $\color{#35bf28}+2.70\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.2069ms | 1.0667ms | 937.4751 Ops/s | 913.6002 Ops/s | $\color{#35bf28}+2.61\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 25.4571ms | 24.7880ms | 40.3422 Ops/s | 39.2901 Ops/s | $\color{#35bf28}+2.68\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9398ms | 0.7077ms | 1.4130 KOps/s | 1.3746 KOps/s | $\color{#35bf28}+2.80\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7343ms | 0.6617ms | 1.5114 KOps/s | 1.4845 KOps/s | $\color{#35bf28}+1.81\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5249ms | 1.4577ms | 686.0287 Ops/s | 677.1597 Ops/s | $\color{#35bf28}+1.31\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7303ms | 0.6758ms | 1.4798 KOps/s | 1.4490 KOps/s | $\color{#35bf28}+2.13\\%$ | | test_dqn_speed | 1.7952ms | 1.4358ms | 696.4635 Ops/s | 667.2706 Ops/s | $\color{#35bf28}+4.37\\%$ | | test_ddpg_speed | 3.2975ms | 2.9314ms | 341.1311 Ops/s | 334.8524 Ops/s | $\color{#35bf28}+1.88\\%$ | | test_sac_speed | 8.7693ms | 8.3849ms | 119.2615 Ops/s | 115.8418 Ops/s | $\color{#35bf28}+2.95\\%$ | | test_redq_speed | 0.1017s | 11.6389ms | 85.9188 Ops/s | 91.9750 Ops/s | $\textbf{\color{#d91a1a}-6.58\\%}$ | | test_redq_deprec_speed | 12.1716ms | 11.6438ms | 85.8825 Ops/s | 85.7554 Ops/s | $\color{#35bf28}+0.15\\%$ | | test_td3_speed | 8.4235ms | 8.3193ms | 120.2021 Ops/s | 114.4008 Ops/s | $\textbf{\color{#35bf28}+5.07\\%}$ | | test_cql_speed | 26.4218ms | 25.6333ms | 39.0117 Ops/s | 38.1453 Ops/s | $\color{#35bf28}+2.27\\%$ | | test_a2c_speed | 6.0087ms | 5.7107ms | 175.1101 Ops/s | 173.5501 Ops/s | $\color{#35bf28}+0.90\\%$ | | test_ppo_speed | 6.7630ms | 6.0410ms | 165.5359 Ops/s | 164.0745 Ops/s | $\color{#35bf28}+0.89\\%$ | | test_reinforce_speed | 4.9286ms | 4.6494ms | 215.0797 Ops/s | 210.8745 Ops/s | $\color{#35bf28}+1.99\\%$ | | test_iql_speed | 20.3664ms | 19.6008ms | 51.0183 Ops/s | 49.8055 Ops/s | $\color{#35bf28}+2.44\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.6815ms | 4.5681ms | 218.9115 Ops/s | 214.4710 Ops/s | $\color{#35bf28}+2.07\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.1063s | 0.6770ms | 1.4770 KOps/s | 1.6579 KOps/s | $\textbf{\color{#d91a1a}-10.91\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.8115ms | 0.5671ms | 1.7633 KOps/s | 1.7075 KOps/s | $\color{#35bf28}+3.27\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.7147ms | 4.5159ms | 221.4379 Ops/s | 217.7875 Ops/s | $\color{#35bf28}+1.68\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7310ms | 0.5833ms | 1.7144 KOps/s | 1.6670 KOps/s | $\color{#35bf28}+2.84\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6996ms | 0.5598ms | 1.7863 KOps/s | 1.7361 KOps/s | $\color{#35bf28}+2.89\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.3146ms | 2.0939ms | 477.5748 Ops/s | 454.4562 Ops/s | $\textbf{\color{#35bf28}+5.09\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2219ms | 2.0063ms | 498.4411 Ops/s | 470.3023 Ops/s | $\textbf{\color{#35bf28}+5.98\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.8274ms | 4.6961ms | 212.9414 Ops/s | 210.8142 Ops/s | $\color{#35bf28}+1.01\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.3902ms | 0.7383ms | 1.3544 KOps/s | 1.2993 KOps/s | $\color{#35bf28}+4.24\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9203ms | 0.7195ms | 1.3899 KOps/s | 1.3442 KOps/s | $\color{#35bf28}+3.40\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.6488ms | 4.5493ms | 219.8146 Ops/s | 215.3208 Ops/s | $\color{#35bf28}+2.09\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7680ms | 0.5908ms | 1.6926 KOps/s | 1.6214 KOps/s | $\color{#35bf28}+4.39\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 4.5367ms | 0.5742ms | 1.7415 KOps/s | 1.6884 KOps/s | $\color{#35bf28}+3.15\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.6801ms | 4.5157ms | 221.4489 Ops/s | 216.4806 Ops/s | $\color{#35bf28}+2.30\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8281ms | 0.5859ms | 1.7067 KOps/s | 1.6450 KOps/s | $\color{#35bf28}+3.75\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7235ms | 0.5656ms | 1.7681 KOps/s | 1.7301 KOps/s | $\color{#35bf28}+2.20\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.9657ms | 4.7160ms | 212.0445 Ops/s | 206.9357 Ops/s | $\color{#35bf28}+2.47\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8552ms | 0.7433ms | 1.3454 KOps/s | 1.3049 KOps/s | $\color{#35bf28}+3.10\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.6377ms | 0.7309ms | 1.3681 KOps/s | 1.3404 KOps/s | $\color{#35bf28}+2.07\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1272s | 7.3826ms | 135.4536 Ops/s | 133.8737 Ops/s | $\color{#35bf28}+1.18\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 17.7147ms | 15.5298ms | 64.3923 Ops/s | 61.1752 Ops/s | $\textbf{\color{#35bf28}+5.26\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.3138ms | 1.3073ms | 764.9619 Ops/s | 738.2958 Ops/s | $\color{#35bf28}+3.61\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1234s | 7.3194ms | 136.6223 Ops/s | 137.0114 Ops/s | $\color{#d91a1a}-0.28\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1359s | 17.9543ms | 55.6969 Ops/s | 53.5733 Ops/s | $\color{#35bf28}+3.96\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.3452ms | 1.3210ms | 756.9835 Ops/s | 707.6893 Ops/s | $\textbf{\color{#35bf28}+6.97\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1236s | 7.5395ms | 132.6340 Ops/s | 133.0183 Ops/s | $\color{#d91a1a}-0.29\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.1440ms | 15.6683ms | 63.8230 Ops/s | 60.4758 Ops/s | $\textbf{\color{#35bf28}+5.53\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.5536ms | 1.4751ms | 677.9367 Ops/s | 639.6469 Ops/s | $\textbf{\color{#35bf28}+5.99\\%}$ |