search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.19k stars 289 forks source link

[Feature] TensorDictMap hashing functions #2304

Open vmoens opened 1 month ago

vmoens commented 1 month ago

Stack from ghstack (oldest at bottom):

pytorch-bot[bot] commented 1 month ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2304

Note: Links to docs will display an error until the docs builds have been completed.

:x: 1 New Failure, 1 Cancelled Job, 31 Pending, 2 Unrelated Failures

As of commit abc2da12d313fdf8bbcb6dbe30c37e6441e40441 with merge base 35a1c5b5782dd918b7f857c44eb6c019a9ec8c41 (image):

NEW FAILURE - The following job has failed:

* [Wheels / build-wheel-windows (3.9, 3.9)](https://hud.pytorch.org/pr/pytorch/rl/2304#28324243911) ([gh](https://github.com/pytorch/rl/actions/runs/10239077898/job/28324243911)) `Process completed with exit code 1.`

CANCELLED JOB - The following job was cancelled. Please retry:

* [Wheels / build-wheel-windows (3.8, 3.8)](https://hud.pytorch.org/pr/pytorch/rl/2304#28324243366) ([gh](https://github.com/pytorch/rl/actions/runs/10239077898/job/28324243366))

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

* [Wheels / build-wheel-windows (3.10, 3.10.3)](https://hud.pytorch.org/pr/pytorch/rl/2304#28324244467) ([gh](https://github.com/pytorch/rl/actions/runs/10239077898/job/28324244467)) (matched **win** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `##[error]The operation was canceled.` * [Wheels / build-wheel-windows (3.12, 3.12)](https://hud.pytorch.org/pr/pytorch/rl/2304#28324246006) ([gh](https://github.com/pytorch/rl/actions/runs/10239077898/job/28324246006)) (matched **win** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `##[error]The operation was canceled.`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 58.7223ms | 57.6399ms | 17.3491 Ops/s | 17.2580 Ops/s | $\color{#35bf28}+0.53\\%$ | | test_sync | 45.8462ms | 36.4040ms | 27.4695 Ops/s | 31.5842 Ops/s | $\textbf{\color{#d91a1a}-13.03\\%}$ | | test_async | 61.4313ms | 30.9522ms | 32.3079 Ops/s | 33.3842 Ops/s | $\color{#d91a1a}-3.22\\%$ | | test_simple | 0.4789s | 0.4131s | 2.4206 Ops/s | 2.4305 Ops/s | $\color{#d91a1a}-0.41\\%$ | | test_transformed | 0.6271s | 0.5742s | 1.7415 Ops/s | 1.7623 Ops/s | $\color{#d91a1a}-1.18\\%$ | | test_serial | 1.3191s | 1.2737s | 0.7851 Ops/s | 0.7919 Ops/s | $\color{#d91a1a}-0.86\\%$ | | test_parallel | 1.1796s | 1.1069s | 0.9034 Ops/s | 0.9072 Ops/s | $\color{#d91a1a}-0.42\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1959ms | 24.4296μs | 40.9340 KOps/s | 40.9363 KOps/s | $-0.01\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 46.9780μs | 14.1934μs | 70.4555 KOps/s | 69.9757 KOps/s | $\color{#35bf28}+0.69\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 0.2079ms | 14.2259μs | 70.2942 KOps/s | 65.8879 KOps/s | $\textbf{\color{#35bf28}+6.69\\%}$ | | test_step_mdp_speed[True-True-True-False-False] | 32.5610μs | 8.0938μs | 123.5517 KOps/s | 122.3885 KOps/s | $\color{#35bf28}+0.95\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 64.3410μs | 25.9269μs | 38.5700 KOps/s | 38.0141 KOps/s | $\color{#35bf28}+1.46\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 0.1849ms | 15.7552μs | 63.4711 KOps/s | 63.4672 KOps/s | $+0.01\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 92.1330μs | 15.5019μs | 64.5083 KOps/s | 64.2728 KOps/s | $\color{#35bf28}+0.37\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 35.8880μs | 9.5814μs | 104.3685 KOps/s | 103.8832 KOps/s | $\color{#35bf28}+0.47\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 66.4040μs | 27.3695μs | 36.5370 KOps/s | 36.0164 KOps/s | $\color{#35bf28}+1.45\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 43.9920μs | 17.1667μs | 58.2524 KOps/s | 57.2119 KOps/s | $\color{#35bf28}+1.82\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 0.3218ms | 16.9329μs | 59.0566 KOps/s | 63.9302 KOps/s | $\textbf{\color{#d91a1a}-7.62\\%}$ | | test_step_mdp_speed[True-False-True-False-False] | 28.7040μs | 9.5217μs | 105.0235 KOps/s | 104.2133 KOps/s | $\color{#35bf28}+0.78\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 64.5210μs | 28.8104μs | 34.7097 KOps/s | 34.4996 KOps/s | $\color{#35bf28}+0.61\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 0.1688ms | 18.5921μs | 53.7864 KOps/s | 52.9479 KOps/s | $\color{#35bf28}+1.58\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 0.3132ms | 18.3058μs | 54.6275 KOps/s | 59.2519 KOps/s | $\textbf{\color{#d91a1a}-7.80\\%}$ | | test_step_mdp_speed[True-False-False-False-False] | 38.1520μs | 10.9360μs | 91.4414 KOps/s | 90.5345 KOps/s | $\color{#35bf28}+1.00\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 64.5410μs | 27.5006μs | 36.3629 KOps/s | 36.0965 KOps/s | $\color{#35bf28}+0.74\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 44.5430μs | 17.3254μs | 57.7186 KOps/s | 57.7253 KOps/s | $\color{#d91a1a}-0.01\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 42.9310μs | 17.9159μs | 55.8163 KOps/s | 55.8002 KOps/s | $\color{#35bf28}+0.03\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 39.8740μs | 10.8249μs | 92.3793 KOps/s | 92.4663 KOps/s | $\color{#d91a1a}-0.09\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 59.5310μs | 28.6796μs | 34.8680 KOps/s | 34.3936 KOps/s | $\color{#35bf28}+1.38\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 0.2855ms | 18.6518μs | 53.6142 KOps/s | 53.5205 KOps/s | $\color{#35bf28}+0.18\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 57.8380μs | 19.4210μs | 51.4907 KOps/s | 51.8768 KOps/s | $\color{#d91a1a}-0.74\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 67.0960μs | 12.0737μs | 82.8248 KOps/s | 81.5027 KOps/s | $\color{#35bf28}+1.62\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 77.4621ms | 33.2973μs | 30.0324 KOps/s | 32.6084 KOps/s | $\textbf{\color{#d91a1a}-7.90\\%}$ | | test_step_mdp_speed[False-False-True-True-False] | 0.2825ms | 20.0846μs | 49.7893 KOps/s | 49.5656 KOps/s | $\color{#35bf28}+0.45\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 91.7220μs | 19.2324μs | 51.9955 KOps/s | 51.7242 KOps/s | $\color{#35bf28}+0.52\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 0.2862ms | 12.8334μs | 77.9215 KOps/s | 81.3482 KOps/s | $\color{#d91a1a}-4.21\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 65.0130μs | 31.2290μs | 32.0215 KOps/s | 31.4299 KOps/s | $\color{#35bf28}+1.88\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 50.0540μs | 21.1005μs | 47.3923 KOps/s | 46.8080 KOps/s | $\color{#35bf28}+1.25\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 45.4650μs | 20.2634μs | 49.3500 KOps/s | 48.7865 KOps/s | $\color{#35bf28}+1.15\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 31.6190μs | 13.3624μs | 74.8368 KOps/s | 73.8001 KOps/s | $\color{#35bf28}+1.40\\%$ | | test_values[generalized_advantage_estimate-True-True] | 10.4168ms | 9.4411ms | 105.9193 Ops/s | 102.3776 Ops/s | $\color{#35bf28}+3.46\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 36.4900ms | 33.4069ms | 29.9339 Ops/s | 27.1962 Ops/s | $\textbf{\color{#35bf28}+10.07\\%}$ | | test_values[td0_return_estimate-False-False] | 0.2202ms | 0.1693ms | 5.9074 KOps/s | 5.9799 KOps/s | $\color{#d91a1a}-1.21\\%$ | | test_values[td1_return_estimate-False-False] | 27.7750ms | 23.8586ms | 41.9137 Ops/s | 41.6896 Ops/s | $\color{#35bf28}+0.54\\%$ | | test_values[vec_td1_return_estimate-False-False] | 35.8862ms | 33.3181ms | 30.0137 Ops/s | 28.0111 Ops/s | $\textbf{\color{#35bf28}+7.15\\%}$ | | test_values[td_lambda_return_estimate-True-False] | 38.3344ms | 34.6790ms | 28.8359 Ops/s | 29.3264 Ops/s | $\color{#d91a1a}-1.67\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 36.1485ms | 33.3049ms | 30.0256 Ops/s | 27.9589 Ops/s | $\textbf{\color{#35bf28}+7.39\\%}$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.4438ms | 8.2327ms | 121.4663 Ops/s | 122.4257 Ops/s | $\color{#d91a1a}-0.78\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.4986ms | 1.9786ms | 505.4078 Ops/s | 522.9331 Ops/s | $\color{#d91a1a}-3.35\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.5379ms | 0.3562ms | 2.8073 KOps/s | 2.8011 KOps/s | $\color{#35bf28}+0.22\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 48.0576ms | 46.0562ms | 21.7126 Ops/s | 21.6240 Ops/s | $\color{#35bf28}+0.41\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 4.2399ms | 3.0374ms | 329.2241 Ops/s | 326.1425 Ops/s | $\color{#35bf28}+0.94\\%$ | | test_dqn_speed | 1.8290ms | 1.2738ms | 785.0332 Ops/s | 781.4850 Ops/s | $\color{#35bf28}+0.45\\%$ | | test_ddpg_speed | 3.0083ms | 2.6797ms | 373.1756 Ops/s | 369.6973 Ops/s | $\color{#35bf28}+0.94\\%$ | | test_sac_speed | 9.4736ms | 7.9562ms | 125.6880 Ops/s | 125.4702 Ops/s | $\color{#35bf28}+0.17\\%$ | | test_redq_speed | 13.9763ms | 12.4717ms | 80.1815 Ops/s | 80.2134 Ops/s | $\color{#d91a1a}-0.04\\%$ | | test_redq_deprec_speed | 15.1398ms | 12.4760ms | 80.1538 Ops/s | 78.3952 Ops/s | $\color{#35bf28}+2.24\\%$ | | test_td3_speed | 8.2196ms | 7.8325ms | 127.6736 Ops/s | 125.1035 Ops/s | $\color{#35bf28}+2.05\\%$ | | test_cql_speed | 36.8652ms | 35.1167ms | 28.4765 Ops/s | 27.7072 Ops/s | $\color{#35bf28}+2.78\\%$ | | test_a2c_speed | 8.8325ms | 7.2201ms | 138.5018 Ops/s | 137.3473 Ops/s | $\color{#35bf28}+0.84\\%$ | | test_ppo_speed | 9.1648ms | 7.5106ms | 133.1450 Ops/s | 131.3815 Ops/s | $\color{#35bf28}+1.34\\%$ | | test_reinforce_speed | 7.5371ms | 6.3909ms | 156.4717 Ops/s | 154.4773 Ops/s | $\color{#35bf28}+1.29\\%$ | | test_iql_speed | 33.2497ms | 31.6862ms | 31.5594 Ops/s | 31.0843 Ops/s | $\color{#35bf28}+1.53\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.9656ms | 4.7255ms | 211.6187 Ops/s | 205.8530 Ops/s | $\color{#35bf28}+2.80\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6852ms | 0.4713ms | 2.1217 KOps/s | 2.1016 KOps/s | $\color{#35bf28}+0.96\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6175ms | 0.4476ms | 2.2343 KOps/s | 2.2226 KOps/s | $\color{#35bf28}+0.53\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 7.8272ms | 4.6981ms | 212.8513 Ops/s | 212.6420 Ops/s | $\color{#35bf28}+0.10\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6713ms | 0.4650ms | 2.1504 KOps/s | 2.1282 KOps/s | $\color{#35bf28}+1.05\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6805ms | 0.4437ms | 2.2536 KOps/s | 2.2597 KOps/s | $\color{#d91a1a}-0.27\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.5797ms | 1.6917ms | 591.1249 Ops/s | 587.0259 Ops/s | $\color{#35bf28}+0.70\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.3220ms | 1.6022ms | 624.1482 Ops/s | 623.1855 Ops/s | $\color{#35bf28}+0.15\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 7.9634ms | 4.8995ms | 204.1029 Ops/s | 201.9616 Ops/s | $\color{#35bf28}+1.06\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.1514s | 0.7348ms | 1.3610 KOps/s | 1.6176 KOps/s | $\textbf{\color{#d91a1a}-15.87\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8462ms | 0.5851ms | 1.7091 KOps/s | 1.7065 KOps/s | $\color{#35bf28}+0.15\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.3392ms | 4.7382ms | 211.0494 Ops/s | 208.1065 Ops/s | $\color{#35bf28}+1.41\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6648ms | 0.4735ms | 2.1118 KOps/s | 2.1070 KOps/s | $\color{#35bf28}+0.23\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 7.7560ms | 0.4669ms | 2.1419 KOps/s | 2.1660 KOps/s | $\color{#d91a1a}-1.11\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 7.3007ms | 4.7056ms | 212.5123 Ops/s | 206.0087 Ops/s | $\color{#35bf28}+3.16\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8853ms | 0.4689ms | 2.1327 KOps/s | 2.1157 KOps/s | $\color{#35bf28}+0.80\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.5582ms | 0.4432ms | 2.2561 KOps/s | 2.2409 KOps/s | $\color{#35bf28}+0.68\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0634ms | 4.8732ms | 205.2035 Ops/s | 201.2797 Ops/s | $\color{#35bf28}+1.95\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.3401ms | 0.6138ms | 1.6291 KOps/s | 1.6055 KOps/s | $\color{#35bf28}+1.47\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7300ms | 0.5816ms | 1.7194 KOps/s | 1.6784 KOps/s | $\color{#35bf28}+2.44\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1364s | 6.3616ms | 157.1926 Ops/s | 160.8304 Ops/s | $\color{#d91a1a}-2.26\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 19.0172ms | 12.9239ms | 77.3757 Ops/s | 77.4545 Ops/s | $\color{#d91a1a}-0.10\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 4.7598ms | 1.1619ms | 860.6500 Ops/s | 903.7594 Ops/s | $\color{#d91a1a}-4.77\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1088s | 7.8738ms | 127.0029 Ops/s | 163.4147 Ops/s | $\textbf{\color{#d91a1a}-22.28\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 16.8641ms | 12.7240ms | 78.5914 Ops/s | 65.9833 Ops/s | $\textbf{\color{#35bf28}+19.11\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 4.6949ms | 1.1915ms | 839.2908 Ops/s | 944.6129 Ops/s | $\textbf{\color{#d91a1a}-11.15\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1110s | 5.9992ms | 166.6877 Ops/s | 163.0227 Ops/s | $\color{#35bf28}+2.25\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 16.6575ms | 12.8342ms | 77.9168 Ops/s | 76.7778 Ops/s | $\color{#35bf28}+1.48\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.7578ms | 1.2425ms | 804.8172 Ops/s | 842.9433 Ops/s | $\color{#d91a1a}-4.52\\%$ |
github-actions[bot] commented 1 month ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1056s | 0.1053s | 9.4998 Ops/s | 9.4326 Ops/s | $\color{#35bf28}+0.71\\%$ | | test_sync | 95.1689ms | 93.5875ms | 10.6852 Ops/s | 10.6621 Ops/s | $\color{#35bf28}+0.22\\%$ | | test_async | 0.1774s | 89.6617ms | 11.1530 Ops/s | 11.5845 Ops/s | $\color{#d91a1a}-3.72\\%$ | | test_single_pixels | 0.1160s | 0.1158s | 8.6387 Ops/s | 8.5047 Ops/s | $\color{#35bf28}+1.58\\%$ | | test_sync_pixels | 76.5159ms | 75.8464ms | 13.1845 Ops/s | 13.1069 Ops/s | $\color{#35bf28}+0.59\\%$ | | test_async_pixels | 0.1332s | 70.0746ms | 14.2705 Ops/s | 14.3990 Ops/s | $\color{#d91a1a}-0.89\\%$ | | test_simple | 0.7826s | 0.7803s | 1.2815 Ops/s | 1.2826 Ops/s | $\color{#d91a1a}-0.09\\%$ | | test_transformed | 1.0917s | 1.0049s | 0.9951 Ops/s | 0.9995 Ops/s | $\color{#d91a1a}-0.44\\%$ | | test_serial | 2.3144s | 2.2097s | 0.4525 Ops/s | 0.4554 Ops/s | $\color{#d91a1a}-0.62\\%$ | | test_parallel | 1.9375s | 1.8796s | 0.5320 Ops/s | 0.5260 Ops/s | $\color{#35bf28}+1.16\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 88.6620μs | 36.6430μs | 27.2903 KOps/s | 27.0734 KOps/s | $\color{#35bf28}+0.80\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 48.0310μs | 20.8497μs | 47.9623 KOps/s | 47.2305 KOps/s | $\color{#35bf28}+1.55\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 45.6010μs | 20.4341μs | 48.9379 KOps/s | 47.3786 KOps/s | $\color{#35bf28}+3.29\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 31.2410μs | 11.7617μs | 85.0220 KOps/s | 81.7839 KOps/s | $\color{#35bf28}+3.96\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 67.1320μs | 38.7327μs | 25.8179 KOps/s | 26.0460 KOps/s | $\color{#d91a1a}-0.88\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 43.5010μs | 22.9668μs | 43.5410 KOps/s | 42.9586 KOps/s | $\color{#35bf28}+1.36\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 44.3810μs | 22.3655μs | 44.7117 KOps/s | 44.3149 KOps/s | $\color{#35bf28}+0.90\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 42.1310μs | 13.9960μs | 71.4489 KOps/s | 69.8337 KOps/s | $\color{#35bf28}+2.31\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 82.9120μs | 40.8674μs | 24.4694 KOps/s | 24.0751 KOps/s | $\color{#35bf28}+1.64\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 43.4610μs | 25.2725μs | 39.5687 KOps/s | 38.8751 KOps/s | $\color{#35bf28}+1.78\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 45.9310μs | 22.6855μs | 44.0810 KOps/s | 43.4673 KOps/s | $\color{#35bf28}+1.41\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 36.2710μs | 14.0184μs | 71.3348 KOps/s | 69.4357 KOps/s | $\color{#35bf28}+2.74\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 74.4610μs | 42.6022μs | 23.4729 KOps/s | 23.0594 KOps/s | $\color{#35bf28}+1.79\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 53.7210μs | 27.4844μs | 36.3843 KOps/s | 35.7506 KOps/s | $\color{#35bf28}+1.77\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 49.8520μs | 24.2039μs | 41.3156 KOps/s | 39.1859 KOps/s | $\textbf{\color{#35bf28}+5.43\\%}$ | | test_step_mdp_speed[True-False-False-False-False] | 39.5410μs | 16.0200μs | 62.4218 KOps/s | 60.0793 KOps/s | $\color{#35bf28}+3.90\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 72.4920μs | 40.8287μs | 24.4926 KOps/s | 24.1892 KOps/s | $\color{#35bf28}+1.25\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 53.0210μs | 25.4325μs | 39.3197 KOps/s | 38.9389 KOps/s | $\color{#35bf28}+0.98\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 55.6510μs | 27.1239μs | 36.8679 KOps/s | 36.3396 KOps/s | $\color{#35bf28}+1.45\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 34.7610μs | 15.8529μs | 63.0800 KOps/s | 61.0304 KOps/s | $\color{#35bf28}+3.36\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 70.8910μs | 42.6827μs | 23.4287 KOps/s | 22.8941 KOps/s | $\color{#35bf28}+2.34\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 48.0810μs | 27.5135μs | 36.3457 KOps/s | 35.9626 KOps/s | $\color{#35bf28}+1.07\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 56.3510μs | 29.4942μs | 33.9050 KOps/s | 33.6725 KOps/s | $\color{#35bf28}+0.69\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 43.6010μs | 18.0302μs | 55.4624 KOps/s | 53.8895 KOps/s | $\color{#35bf28}+2.92\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.9756ms | 44.9122μs | 22.2656 KOps/s | 21.7150 KOps/s | $\color{#35bf28}+2.54\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 48.8910μs | 29.7921μs | 33.5660 KOps/s | 32.9088 KOps/s | $\color{#35bf28}+2.00\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 58.4910μs | 28.7913μs | 34.7327 KOps/s | 34.1197 KOps/s | $\color{#35bf28}+1.80\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 40.0010μs | 17.9554μs | 55.6937 KOps/s | 53.6682 KOps/s | $\color{#35bf28}+3.77\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 76.7920μs | 46.8780μs | 21.3320 KOps/s | 20.8061 KOps/s | $\color{#35bf28}+2.53\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 60.7510μs | 31.7027μs | 31.5431 KOps/s | 31.1717 KOps/s | $\color{#35bf28}+1.19\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 90.4720μs | 31.2043μs | 32.0468 KOps/s | 31.5339 KOps/s | $\color{#35bf28}+1.63\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 39.7600μs | 20.0435μs | 49.8914 KOps/s | 48.8199 KOps/s | $\color{#35bf28}+2.19\\%$ | | test_values[generalized_advantage_estimate-True-True] | 25.2971ms | 24.0838ms | 41.5216 Ops/s | 41.2133 Ops/s | $\color{#35bf28}+0.75\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 89.8745ms | 2.6926ms | 371.3902 Ops/s | 365.1448 Ops/s | $\color{#35bf28}+1.71\\%$ | | test_values[td0_return_estimate-False-False] | 92.8020μs | 65.3177μs | 15.3098 KOps/s | 15.0262 KOps/s | $\color{#35bf28}+1.89\\%$ | | test_values[td1_return_estimate-False-False] | 56.6104ms | 54.7643ms | 18.2601 Ops/s | 18.2700 Ops/s | $\color{#d91a1a}-0.05\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.4273ms | 1.0826ms | 923.6907 Ops/s | 919.9025 Ops/s | $\color{#35bf28}+0.41\\%$ | | test_values[td_lambda_return_estimate-True-False] | 88.5222ms | 86.3585ms | 11.5796 Ops/s | 11.5869 Ops/s | $\color{#d91a1a}-0.06\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.4344ms | 1.0802ms | 925.7414 Ops/s | 921.8437 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 25.6936ms | 25.4131ms | 39.3497 Ops/s | 41.1419 Ops/s | $\color{#d91a1a}-4.36\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9529ms | 0.7148ms | 1.3990 KOps/s | 1.3412 KOps/s | $\color{#35bf28}+4.31\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7517ms | 0.6658ms | 1.5019 KOps/s | 1.4904 KOps/s | $\color{#35bf28}+0.77\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5051ms | 1.4628ms | 683.6276 Ops/s | 679.4155 Ops/s | $\color{#35bf28}+0.62\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7716ms | 0.6838ms | 1.4624 KOps/s | 1.4610 KOps/s | $\color{#35bf28}+0.09\\%$ | | test_dqn_speed | 7.4117ms | 1.3999ms | 714.3512 Ops/s | 716.6125 Ops/s | $\color{#d91a1a}-0.32\\%$ | | test_ddpg_speed | 3.0860ms | 2.8156ms | 355.1620 Ops/s | 355.8494 Ops/s | $\color{#d91a1a}-0.19\\%$ | | test_sac_speed | 8.4446ms | 7.9752ms | 125.3890 Ops/s | 121.2947 Ops/s | $\color{#35bf28}+3.38\\%$ | | test_redq_speed | 11.3966ms | 10.2434ms | 97.6237 Ops/s | 95.9311 Ops/s | $\color{#35bf28}+1.76\\%$ | | test_redq_deprec_speed | 11.0568ms | 10.7702ms | 92.8488 Ops/s | 89.3355 Ops/s | $\color{#35bf28}+3.93\\%$ | | test_td3_speed | 8.2553ms | 7.9633ms | 125.5753 Ops/s | 123.6066 Ops/s | $\color{#35bf28}+1.59\\%$ | | test_cql_speed | 26.9405ms | 25.5232ms | 39.1800 Ops/s | 39.7923 Ops/s | $\color{#d91a1a}-1.54\\%$ | | test_a2c_speed | 6.2479ms | 5.6338ms | 177.5010 Ops/s | 180.2098 Ops/s | $\color{#d91a1a}-1.50\\%$ | | test_ppo_speed | 6.2412ms | 6.0242ms | 165.9981 Ops/s | 174.3569 Ops/s | $\color{#d91a1a}-4.79\\%$ | | test_reinforce_speed | 4.8155ms | 4.5038ms | 222.0360 Ops/s | 227.2698 Ops/s | $\color{#d91a1a}-2.30\\%$ | | test_iql_speed | 19.9483ms | 19.4241ms | 51.4824 Ops/s | 51.5555 Ops/s | $\color{#d91a1a}-0.14\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.7909ms | 6.6686ms | 149.9575 Ops/s | 149.3686 Ops/s | $\color{#35bf28}+0.39\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.7073ms | 0.5227ms | 1.9130 KOps/s | 1.9236 KOps/s | $\color{#d91a1a}-0.55\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6666ms | 0.4958ms | 2.0170 KOps/s | 2.0036 KOps/s | $\color{#35bf28}+0.67\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.8415ms | 6.5427ms | 152.8414 Ops/s | 151.5624 Ops/s | $\color{#35bf28}+0.84\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8525ms | 0.5118ms | 1.9539 KOps/s | 1.9268 KOps/s | $\color{#35bf28}+1.41\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6699ms | 0.4911ms | 2.0362 KOps/s | 2.0101 KOps/s | $\color{#35bf28}+1.30\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.1622ms | 1.9966ms | 500.8456 Ops/s | 503.4378 Ops/s | $\color{#d91a1a}-0.51\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.0864ms | 1.8802ms | 531.8634 Ops/s | 524.4950 Ops/s | $\color{#35bf28}+1.40\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.9369ms | 6.7346ms | 148.4863 Ops/s | 146.2256 Ops/s | $\color{#35bf28}+1.55\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.1263s | 0.7713ms | 1.2965 KOps/s | 1.4772 KOps/s | $\textbf{\color{#d91a1a}-12.23\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8364ms | 0.6462ms | 1.5476 KOps/s | 1.5332 KOps/s | $\color{#35bf28}+0.94\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.8272ms | 6.6659ms | 150.0162 Ops/s | 149.0442 Ops/s | $\color{#35bf28}+0.65\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7654ms | 0.5199ms | 1.9235 KOps/s | 1.9203 KOps/s | $\color{#35bf28}+0.17\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 4.1580ms | 0.5071ms | 1.9719 KOps/s | 1.9386 KOps/s | $\color{#35bf28}+1.72\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 9.4981ms | 6.6803ms | 149.6948 Ops/s | 150.6466 Ops/s | $\color{#d91a1a}-0.63\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.1276s | 0.6860ms | 1.4577 KOps/s | 1.9441 KOps/s | $\textbf{\color{#d91a1a}-25.02\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6223ms | 0.4914ms | 2.0350 KOps/s | 2.0111 KOps/s | $\color{#35bf28}+1.19\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.9769ms | 6.8054ms | 146.9415 Ops/s | 146.4843 Ops/s | $\color{#35bf28}+0.31\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.9211ms | 0.6681ms | 1.4968 KOps/s | 1.4882 KOps/s | $\color{#35bf28}+0.58\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8175ms | 0.6425ms | 1.5565 KOps/s | 1.5404 KOps/s | $\color{#35bf28}+1.04\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1286s | 7.6302ms | 131.0584 Ops/s | 98.8899 Ops/s | $\textbf{\color{#35bf28}+32.53\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.5341ms | 16.1858ms | 61.7824 Ops/s | 60.4914 Ops/s | $\color{#35bf28}+2.13\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.4297ms | 1.1793ms | 847.9855 Ops/s | 773.1233 Ops/s | $\textbf{\color{#35bf28}+9.68\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1255s | 9.9824ms | 100.1762 Ops/s | 130.7815 Ops/s | $\textbf{\color{#d91a1a}-23.40\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 18.6071ms | 16.1274ms | 62.0062 Ops/s | 60.1415 Ops/s | $\color{#35bf28}+3.10\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 6.4342ms | 1.3494ms | 741.0558 Ops/s | 778.1095 Ops/s | $\color{#d91a1a}-4.76\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1239s | 7.7176ms | 129.5733 Ops/s | 129.1888 Ops/s | $\color{#35bf28}+0.30\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.7121ms | 16.2199ms | 61.6526 Ops/s | 60.1273 Ops/s | $\color{#35bf28}+2.54\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 6.5143ms | 1.5380ms | 650.2095 Ops/s | 704.5622 Ops/s | $\textbf{\color{#d91a1a}-7.71\\%}$ |