search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.01k stars 269 forks source link

[Doc] Edit README for local installs #2255

Closed vmoens closed 1 week ago

pytorch-bot[bot] commented 1 week ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2255

Note: Links to docs will display an error until the docs builds have been completed.

:x: 8 New Failures

As of commit 52201f4d60b076d65befcef80dd8131e0c3a27ec with merge base 1083b35ef9733b2335bd88d587cb282e180267c4 (image):

NEW FAILURES - The following jobs have failed:

* [Examples Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2255#26799435869) ([gh](https://github.com/pytorch/rl/actions/runs/9709882101/job/26799435869)) `RuntimeError: Command docker exec -t 9523d26bb02da9b0512a88dc658b1e816fa24297dcbd1dd2f230da8b165eeb3b /exec failed with exit code 8` * [Generate documentation / build-docs (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2255#26799436555) ([gh](https://github.com/pytorch/rl/actions/runs/9709882107/job/26799436555)) `RuntimeError: Command docker exec -t 6dd9d5114e4f01626b2dd57923ce65569c0e56cce77d38a07c86545035235270 /exec failed with exit code 8` * [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2255#26799436776) ([gh](https://github.com/pytorch/rl/actions/runs/9709882098/job/26799436776)) `RuntimeError: Command docker exec -t 0fe463f0e8e8f5606a82585debc73be240fd3b2c1ac526e62b9997212c63feb7 /exec failed with exit code 8` * [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2255#26799443032) ([gh](https://github.com/pytorch/rl/actions/runs/9709882141/job/26799443032)) `RuntimeError: Command docker exec -t 11aa5420389a604853ee4b5b8a089545ca409a528adce1ff609551c0a398cd2a /exec failed with exit code 8` * [Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2255#26799443314) ([gh](https://github.com/pytorch/rl/actions/runs/9709882141/job/26799443314)) `RuntimeError: Command docker exec -t 0dc5180d9bf4c8b21b217fd8413f580bae6b9bdb82b661b597d7e5918d3ad6f4 /exec failed with exit code 8` * [RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2255#26799436584) ([gh](https://github.com/pytorch/rl/actions/runs/9709882132/job/26799436584)) `RuntimeError: vmap: It looks like you're attempting to use a Tensor in some data-dependent control flow. We don't support that yet, please shout over at https://github.com/pytorch/functorch/issues/257 .` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2255#26799440554) ([gh](https://github.com/pytorch/rl/actions/runs/9709882126/job/26799440554)) `RuntimeError: Command docker exec -t 01fef29ac9465be98420079e06fd9e95a53e1a58172896ff137dbc736cfa6063 /exec failed with exit code 8` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2255#26799435236) ([gh](https://github.com/pytorch/rl/actions/runs/9709882095/job/26799435236)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 1 week ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1171s | 59.9114ms | 16.6913 Ops/s | 17.9006 Ops/s | $\textbf{\color{#d91a1a}-6.76\\%}$ | | test_sync | 39.5396ms | 32.3238ms | 30.9369 Ops/s | 32.4530 Ops/s | $\color{#d91a1a}-4.67\\%$ | | test_async | 54.3160ms | 28.4848ms | 35.1065 Ops/s | 35.4533 Ops/s | $\color{#d91a1a}-0.98\\%$ | | test_simple | 0.3954s | 0.3854s | 2.5947 Ops/s | 2.4797 Ops/s | $\color{#35bf28}+4.64\\%$ | | test_transformed | 0.5394s | 0.5377s | 1.8597 Ops/s | 1.8600 Ops/s | $\color{#d91a1a}-0.02\\%$ | | test_serial | 1.3804s | 1.2942s | 0.7727 Ops/s | 0.7865 Ops/s | $\color{#d91a1a}-1.76\\%$ | | test_parallel | 1.1308s | 1.0761s | 0.9293 Ops/s | 0.9207 Ops/s | $\color{#35bf28}+0.94\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1519ms | 22.7743μs | 43.9091 KOps/s | 44.0623 KOps/s | $\color{#d91a1a}-0.35\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 38.0710μs | 13.4706μs | 74.2358 KOps/s | 74.6135 KOps/s | $\color{#d91a1a}-0.51\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 46.0860μs | 13.4301μs | 74.4595 KOps/s | 75.4831 KOps/s | $\color{#d91a1a}-1.36\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 38.3210μs | 7.9518μs | 125.7579 KOps/s | 127.0104 KOps/s | $\color{#d91a1a}-0.99\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 56.9360μs | 24.1436μs | 41.4188 KOps/s | 41.3187 KOps/s | $\color{#35bf28}+0.24\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 40.2650μs | 14.7500μs | 67.7968 KOps/s | 67.8340 KOps/s | $\color{#d91a1a}-0.05\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 44.2230μs | 14.6384μs | 68.3134 KOps/s | 68.7602 KOps/s | $\color{#d91a1a}-0.65\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 28.4940μs | 9.0709μs | 110.2430 KOps/s | 111.1164 KOps/s | $\color{#d91a1a}-0.79\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 58.1190μs | 25.4322μs | 39.3202 KOps/s | 39.0791 KOps/s | $\color{#35bf28}+0.62\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 48.1190μs | 15.9087μs | 62.8586 KOps/s | 62.7784 KOps/s | $\color{#35bf28}+0.13\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 58.1790μs | 14.4652μs | 69.1315 KOps/s | 68.9584 KOps/s | $\color{#35bf28}+0.25\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 31.7190μs | 9.1201μs | 109.6475 KOps/s | 110.6741 KOps/s | $\color{#d91a1a}-0.93\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 59.0300μs | 26.7379μs | 37.4002 KOps/s | 37.4961 KOps/s | $\color{#d91a1a}-0.26\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 40.6950μs | 17.0939μs | 58.5004 KOps/s | 57.8619 KOps/s | $\color{#35bf28}+1.10\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 65.7330μs | 15.7705μs | 63.4095 KOps/s | 64.6220 KOps/s | $\color{#d91a1a}-1.88\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 36.5980μs | 10.2757μs | 97.3166 KOps/s | 99.1890 KOps/s | $\color{#d91a1a}-1.89\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 56.3950μs | 25.6908μs | 38.9245 KOps/s | 39.4274 KOps/s | $\color{#d91a1a}-1.28\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 59.6420μs | 16.0380μs | 62.3520 KOps/s | 62.8488 KOps/s | $\color{#d91a1a}-0.79\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 46.1470μs | 16.9154μs | 59.1178 KOps/s | 59.5383 KOps/s | $\color{#d91a1a}-0.71\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 40.0250μs | 10.2998μs | 97.0890 KOps/s | 97.6611 KOps/s | $\color{#d91a1a}-0.59\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 64.4800μs | 26.6415μs | 37.5354 KOps/s | 37.7926 KOps/s | $\color{#d91a1a}-0.68\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 47.4190μs | 17.1500μs | 58.3090 KOps/s | 58.4383 KOps/s | $\color{#d91a1a}-0.22\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 47.1880μs | 18.1659μs | 55.0480 KOps/s | 55.6926 KOps/s | $\color{#d91a1a}-1.16\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 0.2834ms | 12.4531μs | 80.3013 KOps/s | 88.6976 KOps/s | $\textbf{\color{#d91a1a}-9.47\\%}$ | | test_step_mdp_speed[False-False-True-True-True] | 67.3460μs | 27.9986μs | 35.7160 KOps/s | 35.8760 KOps/s | $\color{#d91a1a}-0.45\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 74.8900μs | 18.3734μs | 54.4265 KOps/s | 54.6166 KOps/s | $\color{#d91a1a}-0.35\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 49.9630μs | 18.2348μs | 54.8403 KOps/s | 55.2485 KOps/s | $\color{#d91a1a}-0.74\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 40.0550μs | 11.5559μs | 86.5357 KOps/s | 87.8387 KOps/s | $\color{#d91a1a}-1.48\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 52.5580μs | 29.6249μs | 33.7554 KOps/s | 33.9830 KOps/s | $\color{#d91a1a}-0.67\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 48.5800μs | 19.5131μs | 51.2478 KOps/s | 51.1041 KOps/s | $\color{#35bf28}+0.28\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 63.8790μs | 19.6160μs | 50.9789 KOps/s | 52.8893 KOps/s | $\color{#d91a1a}-3.61\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 53.9110μs | 12.5267μs | 79.8295 KOps/s | 80.7923 KOps/s | $\color{#d91a1a}-1.19\\%$ | | test_values[generalized_advantage_estimate-True-True] | 12.5175ms | 9.8069ms | 101.9694 Ops/s | 105.1203 Ops/s | $\color{#d91a1a}-3.00\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 37.3953ms | 35.6494ms | 28.0510 Ops/s | 28.0275 Ops/s | $\color{#35bf28}+0.08\\%$ | | test_values[td0_return_estimate-False-False] | 0.2262ms | 0.1902ms | 5.2564 KOps/s | 5.2854 KOps/s | $\color{#d91a1a}-0.55\\%$ | | test_values[td1_return_estimate-False-False] | 27.4289ms | 24.2529ms | 41.2322 Ops/s | 42.2205 Ops/s | $\color{#d91a1a}-2.34\\%$ | | test_values[vec_td1_return_estimate-False-False] | 38.1072ms | 35.8561ms | 27.8893 Ops/s | 28.2191 Ops/s | $\color{#d91a1a}-1.17\\%$ | | test_values[td_lambda_return_estimate-True-False] | 37.4355ms | 34.5576ms | 28.9372 Ops/s | 29.1926 Ops/s | $\color{#d91a1a}-0.87\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 36.8646ms | 35.5971ms | 28.0922 Ops/s | 28.1725 Ops/s | $\color{#d91a1a}-0.29\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 10.4983ms | 8.3610ms | 119.6034 Ops/s | 119.3551 Ops/s | $\color{#35bf28}+0.21\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.5900ms | 1.9334ms | 517.2280 Ops/s | 496.5999 Ops/s | $\color{#35bf28}+4.15\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4485ms | 0.3597ms | 2.7803 KOps/s | 2.8300 KOps/s | $\color{#d91a1a}-1.75\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 53.1164ms | 47.9904ms | 20.8375 Ops/s | 22.2348 Ops/s | $\textbf{\color{#d91a1a}-6.28\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 4.0270ms | 3.0907ms | 323.5495 Ops/s | 329.2826 Ops/s | $\color{#d91a1a}-1.74\\%$ | | test_dqn_speed | 7.0742ms | 1.3813ms | 723.9569 Ops/s | 738.0308 Ops/s | $\color{#d91a1a}-1.91\\%$ | | test_ddpg_speed | 3.8489ms | 2.9163ms | 342.8991 Ops/s | 349.8871 Ops/s | $\color{#d91a1a}-2.00\\%$ | | test_sac_speed | 10.6671ms | 8.6762ms | 115.2582 Ops/s | 114.2088 Ops/s | $\color{#35bf28}+0.92\\%$ | | test_redq_speed | 22.1500ms | 13.8798ms | 72.0472 Ops/s | 65.2408 Ops/s | $\textbf{\color{#35bf28}+10.43\\%}$ | | test_redq_deprec_speed | 14.5799ms | 13.9019ms | 71.9325 Ops/s | 72.6907 Ops/s | $\color{#d91a1a}-1.04\\%$ | | test_td3_speed | 9.3686ms | 8.5490ms | 116.9723 Ops/s | 118.1076 Ops/s | $\color{#d91a1a}-0.96\\%$ | | test_cql_speed | 38.2961ms | 37.0542ms | 26.9875 Ops/s | 27.2067 Ops/s | $\color{#d91a1a}-0.81\\%$ | | test_a2c_speed | 8.6264ms | 7.5225ms | 132.9354 Ops/s | 130.9700 Ops/s | $\color{#35bf28}+1.50\\%$ | | test_ppo_speed | 8.8531ms | 7.8248ms | 127.7983 Ops/s | 126.9432 Ops/s | $\color{#35bf28}+0.67\\%$ | | test_reinforce_speed | 7.8323ms | 6.7229ms | 148.7445 Ops/s | 147.2762 Ops/s | $\color{#35bf28}+1.00\\%$ | | test_iql_speed | 34.3208ms | 33.0815ms | 30.2284 Ops/s | 30.4315 Ops/s | $\color{#d91a1a}-0.67\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.2077ms | 3.4761ms | 287.6757 Ops/s | 287.4017 Ops/s | $\color{#35bf28}+0.10\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.0263ms | 0.5080ms | 1.9684 KOps/s | 1.9796 KOps/s | $\color{#d91a1a}-0.56\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7138ms | 0.4937ms | 2.0253 KOps/s | 2.1119 KOps/s | $\color{#d91a1a}-4.10\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.6782ms | 3.8227ms | 261.5935 Ops/s | 292.9100 Ops/s | $\textbf{\color{#d91a1a}-10.69\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.1150ms | 0.5170ms | 1.9344 KOps/s | 2.0260 KOps/s | $\color{#d91a1a}-4.52\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7220ms | 0.4995ms | 2.0022 KOps/s | 2.1091 KOps/s | $\textbf{\color{#d91a1a}-5.07\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.3256ms | 1.7632ms | 567.1464 Ops/s | 576.9419 Ops/s | $\color{#d91a1a}-1.70\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2958ms | 1.6529ms | 604.9962 Ops/s | 608.5119 Ops/s | $\color{#d91a1a}-0.58\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.7916ms | 3.8102ms | 262.4535 Ops/s | 280.2409 Ops/s | $\textbf{\color{#d91a1a}-6.35\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.1428s | 0.7762ms | 1.2884 KOps/s | 1.5693 KOps/s | $\textbf{\color{#d91a1a}-17.90\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8265ms | 0.6314ms | 1.5838 KOps/s | 1.6228 KOps/s | $\color{#d91a1a}-2.40\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.0911ms | 3.7917ms | 263.7313 Ops/s | 281.2637 Ops/s | $\textbf{\color{#d91a1a}-6.23\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6597ms | 0.5127ms | 1.9505 KOps/s | 1.9890 KOps/s | $\color{#d91a1a}-1.94\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.9017ms | 0.4945ms | 2.0221 KOps/s | 2.0637 KOps/s | $\color{#d91a1a}-2.02\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.2162ms | 3.7039ms | 269.9825 Ops/s | 284.3238 Ops/s | $\textbf{\color{#d91a1a}-5.04\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6979ms | 0.5137ms | 1.9467 KOps/s | 2.0080 KOps/s | $\color{#d91a1a}-3.05\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6010ms | 0.4871ms | 2.0529 KOps/s | 2.0555 KOps/s | $\color{#d91a1a}-0.13\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.0589ms | 3.8235ms | 261.5409 Ops/s | 268.8072 Ops/s | $\color{#d91a1a}-2.70\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.2205ms | 0.6521ms | 1.5335 KOps/s | 1.5283 KOps/s | $\color{#35bf28}+0.34\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7959ms | 0.6247ms | 1.6007 KOps/s | 1.6240 KOps/s | $\color{#d91a1a}-1.43\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1412s | 6.6073ms | 151.3487 Ops/s | 122.1334 Ops/s | $\textbf{\color{#35bf28}+23.92\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 15.9994ms | 12.7760ms | 78.2720 Ops/s | 79.8220 Ops/s | $\color{#d91a1a}-1.94\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 3.9422ms | 1.1718ms | 853.4061 Ops/s | 976.4249 Ops/s | $\textbf{\color{#d91a1a}-12.60\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1198s | 8.2973ms | 120.5212 Ops/s | 161.3062 Ops/s | $\textbf{\color{#d91a1a}-25.28\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 15.0281ms | 12.7169ms | 78.6353 Ops/s | 80.8185 Ops/s | $\color{#d91a1a}-2.70\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 3.6741ms | 1.1210ms | 892.0906 Ops/s | 959.8719 Ops/s | $\textbf{\color{#d91a1a}-7.06\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1119s | 6.1432ms | 162.7829 Ops/s | 163.1763 Ops/s | $\color{#d91a1a}-0.24\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 15.4260ms | 12.8339ms | 77.9187 Ops/s | 77.8639 Ops/s | $\color{#35bf28}+0.07\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.8621ms | 1.2386ms | 807.3310 Ops/s | 824.4349 Ops/s | $\color{#d91a1a}-2.07\\%$ |
github-actions[bot] commented 1 week ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1212s | 0.1202s | 8.3162 Ops/s | 8.3525 Ops/s | $\color{#d91a1a}-0.43\\%$ | | test_sync | 0.1042s | 0.1014s | 9.8637 Ops/s | 9.6678 Ops/s | $\color{#35bf28}+2.03\\%$ | | test_async | 0.1988s | 99.4707ms | 10.0532 Ops/s | 10.1447 Ops/s | $\color{#d91a1a}-0.90\\%$ | | test_single_pixels | 0.1302s | 0.1301s | 7.6856 Ops/s | 7.8259 Ops/s | $\color{#d91a1a}-1.79\\%$ | | test_sync_pixels | 84.2656ms | 82.6295ms | 12.1022 Ops/s | 12.5238 Ops/s | $\color{#d91a1a}-3.37\\%$ | | test_async_pixels | 0.1612s | 68.2023ms | 14.6623 Ops/s | 14.3225 Ops/s | $\color{#35bf28}+2.37\\%$ | | test_simple | 0.8964s | 0.8326s | 1.2011 Ops/s | 1.1885 Ops/s | $\color{#35bf28}+1.06\\%$ | | test_transformed | 1.1579s | 1.0941s | 0.9140 Ops/s | 0.9149 Ops/s | $\color{#d91a1a}-0.10\\%$ | | test_serial | 2.5853s | 2.5230s | 0.3964 Ops/s | 0.3844 Ops/s | $\color{#35bf28}+3.10\\%$ | | test_parallel | 2.4382s | 2.3816s | 0.4199 Ops/s | 0.4179 Ops/s | $\color{#35bf28}+0.47\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1177ms | 34.0616μs | 29.3585 KOps/s | 28.9133 KOps/s | $\color{#35bf28}+1.54\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 56.6300μs | 19.5302μs | 51.2029 KOps/s | 49.5280 KOps/s | $\color{#35bf28}+3.38\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 37.1910μs | 19.1934μs | 52.1013 KOps/s | 50.4768 KOps/s | $\color{#35bf28}+3.22\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 30.5610μs | 11.1360μs | 89.7988 KOps/s | 85.9893 KOps/s | $\color{#35bf28}+4.43\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 84.6210μs | 35.6175μs | 28.0761 KOps/s | 27.6332 KOps/s | $\color{#35bf28}+1.60\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 45.8910μs | 21.3072μs | 46.9324 KOps/s | 46.0926 KOps/s | $\color{#35bf28}+1.82\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 46.8110μs | 20.9925μs | 47.6360 KOps/s | 46.3030 KOps/s | $\color{#35bf28}+2.88\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 35.0210μs | 13.0607μs | 76.5657 KOps/s | 74.3149 KOps/s | $\color{#35bf28}+3.03\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 73.6020μs | 37.4311μs | 26.7157 KOps/s | 26.2760 KOps/s | $\color{#35bf28}+1.67\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 46.8600μs | 23.2726μs | 42.9689 KOps/s | 42.1566 KOps/s | $\color{#35bf28}+1.93\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 53.4410μs | 20.7913μs | 48.0969 KOps/s | 46.9415 KOps/s | $\color{#35bf28}+2.46\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 37.9310μs | 13.3416μs | 74.9535 KOps/s | 75.2863 KOps/s | $\color{#d91a1a}-0.44\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 60.8610μs | 40.1703μs | 24.8940 KOps/s | 25.2412 KOps/s | $\color{#d91a1a}-1.38\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 53.2510μs | 25.3151μs | 39.5022 KOps/s | 39.2552 KOps/s | $\color{#35bf28}+0.63\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 40.8200μs | 22.8958μs | 43.6762 KOps/s | 42.9849 KOps/s | $\color{#35bf28}+1.61\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 31.9910μs | 15.1617μs | 65.9558 KOps/s | 66.1906 KOps/s | $\color{#d91a1a}-0.35\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 55.8210μs | 37.9857μs | 26.3257 KOps/s | 26.2959 KOps/s | $\color{#35bf28}+0.11\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 49.1410μs | 23.6838μs | 42.2229 KOps/s | 42.1281 KOps/s | $\color{#35bf28}+0.22\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 42.9610μs | 25.6300μs | 39.0168 KOps/s | 39.9318 KOps/s | $\color{#d91a1a}-2.29\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 34.1610μs | 15.0595μs | 66.4031 KOps/s | 66.9126 KOps/s | $\color{#d91a1a}-0.76\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 63.8810μs | 39.9827μs | 25.0108 KOps/s | 25.3076 KOps/s | $\color{#d91a1a}-1.17\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 45.8010μs | 25.1845μs | 39.7070 KOps/s | 39.4375 KOps/s | $\color{#35bf28}+0.68\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 52.1910μs | 27.4245μs | 36.4637 KOps/s | 37.3708 KOps/s | $\color{#d91a1a}-2.43\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 36.9500μs | 16.8139μs | 59.4747 KOps/s | 59.5046 KOps/s | $\color{#d91a1a}-0.05\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 0.1055ms | 41.3428μs | 24.1880 KOps/s | 24.1857 KOps/s | $+0.01\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 49.9210μs | 27.4995μs | 36.3643 KOps/s | 36.6929 KOps/s | $\color{#d91a1a}-0.90\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 54.1910μs | 26.9586μs | 37.0939 KOps/s | 37.7015 KOps/s | $\color{#d91a1a}-1.61\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 33.0100μs | 16.9341μs | 59.0525 KOps/s | 59.4706 KOps/s | $\color{#d91a1a}-0.70\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 61.4710μs | 44.1553μs | 22.6473 KOps/s | 22.9169 KOps/s | $\color{#d91a1a}-1.18\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 54.5410μs | 29.1489μs | 34.3066 KOps/s | 33.8390 KOps/s | $\color{#35bf28}+1.38\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 49.7400μs | 28.9852μs | 34.5004 KOps/s | 35.0607 KOps/s | $\color{#d91a1a}-1.60\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 34.8400μs | 18.6752μs | 53.5469 KOps/s | 52.8809 KOps/s | $\color{#35bf28}+1.26\\%$ | | test_values[generalized_advantage_estimate-True-True] | 25.6312ms | 25.1402ms | 39.7769 Ops/s | 39.9960 Ops/s | $\color{#d91a1a}-0.55\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 92.1443ms | 2.7447ms | 364.3383 Ops/s | 367.1190 Ops/s | $\color{#d91a1a}-0.76\\%$ | | test_values[td0_return_estimate-False-False] | 94.4410μs | 66.9994μs | 14.9255 KOps/s | 14.7343 KOps/s | $\color{#35bf28}+1.30\\%$ | | test_values[td1_return_estimate-False-False] | 57.1029ms | 56.6863ms | 17.6409 Ops/s | 17.2911 Ops/s | $\color{#35bf28}+2.02\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.2750ms | 1.0881ms | 919.0699 Ops/s | 910.1381 Ops/s | $\color{#35bf28}+0.98\\%$ | | test_values[td_lambda_return_estimate-True-False] | 91.2621ms | 89.7141ms | 11.1465 Ops/s | 10.6989 Ops/s | $\color{#35bf28}+4.18\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.2466ms | 1.0829ms | 923.4853 Ops/s | 904.7910 Ops/s | $\color{#35bf28}+2.07\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 25.4696ms | 25.3388ms | 39.4652 Ops/s | 37.1752 Ops/s | $\textbf{\color{#35bf28}+6.16\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9754ms | 0.7325ms | 1.3651 KOps/s | 1.3289 KOps/s | $\color{#35bf28}+2.73\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7659ms | 0.6750ms | 1.4815 KOps/s | 1.4190 KOps/s | $\color{#35bf28}+4.40\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5038ms | 1.4782ms | 676.4868 Ops/s | 670.1377 Ops/s | $\color{#35bf28}+0.95\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7359ms | 0.6892ms | 1.4509 KOps/s | 1.3892 KOps/s | $\color{#35bf28}+4.44\\%$ | | test_dqn_speed | 1.8307ms | 1.5191ms | 658.2933 Ops/s | 675.1628 Ops/s | $\color{#d91a1a}-2.50\\%$ | | test_ddpg_speed | 3.3979ms | 3.0321ms | 329.8096 Ops/s | 335.7369 Ops/s | $\color{#d91a1a}-1.77\\%$ | | test_sac_speed | 8.9092ms | 8.6455ms | 115.6674 Ops/s | 117.9267 Ops/s | $\color{#d91a1a}-1.92\\%$ | | test_redq_speed | 0.1057s | 11.9609ms | 83.6058 Ops/s | 92.1913 Ops/s | $\textbf{\color{#d91a1a}-9.31\\%}$ | | test_redq_deprec_speed | 13.0724ms | 12.3416ms | 81.0266 Ops/s | 84.4544 Ops/s | $\color{#d91a1a}-4.06\\%$ | | test_td3_speed | 8.8203ms | 8.5724ms | 116.6538 Ops/s | 116.8005 Ops/s | $\color{#d91a1a}-0.13\\%$ | | test_cql_speed | 26.6526ms | 26.1385ms | 38.2578 Ops/s | 38.8548 Ops/s | $\color{#d91a1a}-1.54\\%$ | | test_a2c_speed | 6.2390ms | 5.9453ms | 168.1999 Ops/s | 174.2106 Ops/s | $\color{#d91a1a}-3.45\\%$ | | test_ppo_speed | 6.6744ms | 6.2102ms | 161.0259 Ops/s | 164.8687 Ops/s | $\color{#d91a1a}-2.33\\%$ | | test_reinforce_speed | 5.5599ms | 4.8403ms | 206.5995 Ops/s | 213.3642 Ops/s | $\color{#d91a1a}-3.17\\%$ | | test_iql_speed | 20.8810ms | 19.9383ms | 50.1546 Ops/s | 50.3696 Ops/s | $\color{#d91a1a}-0.43\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.8476ms | 4.6688ms | 214.1883 Ops/s | 215.5807 Ops/s | $\color{#d91a1a}-0.65\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.1086s | 0.6857ms | 1.4584 KOps/s | 1.6658 KOps/s | $\textbf{\color{#d91a1a}-12.45\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7767ms | 0.5699ms | 1.7548 KOps/s | 1.7399 KOps/s | $\color{#35bf28}+0.86\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.8380ms | 4.6336ms | 215.8142 Ops/s | 218.0458 Ops/s | $\color{#d91a1a}-1.02\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.2740ms | 0.5896ms | 1.6961 KOps/s | 1.7072 KOps/s | $\color{#d91a1a}-0.65\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6974ms | 0.5646ms | 1.7712 KOps/s | 1.7668 KOps/s | $\color{#35bf28}+0.25\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.8334ms | 2.1251ms | 470.5769 Ops/s | 474.3212 Ops/s | $\color{#d91a1a}-0.79\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2168ms | 2.0185ms | 495.4083 Ops/s | 496.9110 Ops/s | $\color{#d91a1a}-0.30\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0083ms | 4.8230ms | 207.3395 Ops/s | 210.5882 Ops/s | $\color{#d91a1a}-1.54\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8771ms | 0.7507ms | 1.3320 KOps/s | 1.3541 KOps/s | $\color{#d91a1a}-1.63\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.7849ms | 0.7321ms | 1.3660 KOps/s | 1.3897 KOps/s | $\color{#d91a1a}-1.70\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.9149ms | 4.7194ms | 211.8909 Ops/s | 215.8602 Ops/s | $\color{#d91a1a}-1.84\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7510ms | 0.5941ms | 1.6832 KOps/s | 1.6872 KOps/s | $\color{#d91a1a}-0.24\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7002ms | 0.5705ms | 1.7527 KOps/s | 1.7460 KOps/s | $\color{#35bf28}+0.39\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.9339ms | 4.6717ms | 214.0536 Ops/s | 219.0577 Ops/s | $\color{#d91a1a}-2.28\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.1402s | 0.7986ms | 1.2522 KOps/s | 1.7033 KOps/s | $\textbf{\color{#d91a1a}-26.49\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7195ms | 0.5639ms | 1.7733 KOps/s | 1.7539 KOps/s | $\color{#35bf28}+1.11\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.1171ms | 4.8554ms | 205.9562 Ops/s | 210.0482 Ops/s | $\color{#d91a1a}-1.95\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8925ms | 0.7542ms | 1.3259 KOps/s | 1.3365 KOps/s | $\color{#d91a1a}-0.79\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.7625ms | 0.7310ms | 1.3679 KOps/s | 1.3743 KOps/s | $\color{#d91a1a}-0.46\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1261s | 7.4135ms | 134.8898 Ops/s | 134.8145 Ops/s | $\color{#35bf28}+0.06\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.3929ms | 15.6952ms | 63.7137 Ops/s | 61.8023 Ops/s | $\color{#35bf28}+3.09\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.6266ms | 1.3416ms | 745.3947 Ops/s | 785.4190 Ops/s | $\textbf{\color{#d91a1a}-5.10\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1289s | 9.9827ms | 100.1734 Ops/s | 136.1996 Ops/s | $\textbf{\color{#d91a1a}-26.45\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 18.5135ms | 15.7014ms | 63.6887 Ops/s | 54.3392 Ops/s | $\textbf{\color{#35bf28}+17.21\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.4896ms | 1.3289ms | 752.4776 Ops/s | 722.6023 Ops/s | $\color{#35bf28}+4.13\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1272s | 7.6231ms | 131.1798 Ops/s | 132.4948 Ops/s | $\color{#d91a1a}-0.99\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.4164ms | 15.8717ms | 63.0053 Ops/s | 61.9703 Ops/s | $\color{#35bf28}+1.67\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.5627ms | 1.4937ms | 669.4687 Ops/s | 681.0836 Ops/s | $\color{#d91a1a}-1.71\\%$ |