search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.27k stars 302 forks source link

[Feature] single player for TicTacToe #2303

Closed vmoens closed 2 months ago

vmoens commented 2 months ago

Stack from ghstack (oldest at bottom):

pytorch-bot[bot] commented 2 months ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2303

Note: Links to docs will display an error until the docs builds have been completed.

:x: 2 New Failures, 1 Pending, 4 Unrelated Failures

As of commit e47e628f30df5ff1e9b81db84ca42c13a06b44ef with merge base c771e6e858faa84a9084c7e3993c3d0a134d8ea3 (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2303#27751691621) ([gh](https://github.com/pytorch/rl/actions/runs/10042088266/job/27751691621)) `RuntimeError: Command docker exec -t 1b61a2a0d8fb107cad9b2fee7656cc5b6d65adf7ac58fec046bb76215464c0ca /exec failed with exit code 139` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2303#27751697640) ([gh](https://github.com/pytorch/rl/actions/runs/10042088272/job/27751697640)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2303#27751698419) ([gh](https://github.com/pytorch/rl/actions/runs/10042088277/job/27751698419)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/c771e6e858faa84a9084c7e3993c3d0a134d8ea3#27674777898)) `AttributeError: module 'torch' has no attribute 'compiler'` * [Libs Tests on Linux / unittests-jumanji (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2303#27751698776) ([gh](https://github.com/pytorch/rl/actions/runs/10042088277/job/27751698776)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/c771e6e858faa84a9084c7e3993c3d0a134d8ea3#27674778270)) `test/test_libs.py::TestJumanji::test_jumanji_rendering[batch_size1-RubiksCube-partly-scrambled-v0]` * [Libs Tests on Linux / unittests-robohive (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2303#27751699142) ([gh](https://github.com/pytorch/rl/actions/runs/10042088277/job/27751699142)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/c771e6e858faa84a9084c7e3993c3d0a134d8ea3#27674778892)) `test/test_libs.py::TestRoboHive::test_robohive[franka_slide_random-v3-True-True]` * [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2303#27751696075) ([gh](https://github.com/pytorch/rl/actions/runs/10042088282/job/27751696075)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/c771e6e858faa84a9084c7e3993c3d0a134d8ea3#27674776256)) `AttributeError: module 'torch' has no attribute 'compiler'`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 2 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 58.9604ms | 57.6349ms | 17.3506 Ops/s | 17.0033 Ops/s | $\color{#35bf28}+2.04\\%$ | | test_sync | 33.7012ms | 31.6225ms | 31.6231 Ops/s | 31.1000 Ops/s | $\color{#35bf28}+1.68\\%$ | | test_async | 68.5445ms | 29.9608ms | 33.3770 Ops/s | 33.3000 Ops/s | $\color{#35bf28}+0.23\\%$ | | test_simple | 0.4768s | 0.4113s | 2.4313 Ops/s | 2.4126 Ops/s | $\color{#35bf28}+0.77\\%$ | | test_transformed | 0.6239s | 0.5675s | 1.7621 Ops/s | 1.7268 Ops/s | $\color{#35bf28}+2.05\\%$ | | test_serial | 1.3214s | 1.2628s | 0.7919 Ops/s | 0.7801 Ops/s | $\color{#35bf28}+1.51\\%$ | | test_parallel | 1.1708s | 1.1055s | 0.9045 Ops/s | 0.8934 Ops/s | $\color{#35bf28}+1.25\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 67.9270μs | 25.3005μs | 39.5250 KOps/s | 39.3849 KOps/s | $\color{#35bf28}+0.36\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 45.8460μs | 14.7593μs | 67.7540 KOps/s | 68.5259 KOps/s | $\color{#d91a1a}-1.13\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 57.5970μs | 14.5277μs | 68.8338 KOps/s | 68.1062 KOps/s | $\color{#35bf28}+1.07\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 29.0640μs | 8.5120μs | 117.4810 KOps/s | 117.4860 KOps/s | $-0.00\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 75.6120μs | 26.9700μs | 37.0782 KOps/s | 36.6883 KOps/s | $\color{#35bf28}+1.06\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 49.5430μs | 17.2999μs | 57.8039 KOps/s | 61.3037 KOps/s | $\textbf{\color{#d91a1a}-5.71\\%}$ | | test_step_mdp_speed[True-True-False-False-True] | 0.2623ms | 17.1658μs | 58.2554 KOps/s | 61.0659 KOps/s | $\color{#d91a1a}-4.60\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 52.3170μs | 10.0628μs | 99.3756 KOps/s | 99.4117 KOps/s | $\color{#d91a1a}-0.04\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 85.8410μs | 28.6919μs | 34.8530 KOps/s | 34.0851 KOps/s | $\color{#35bf28}+2.25\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 48.8820μs | 18.0262μs | 55.4747 KOps/s | 55.4811 KOps/s | $\color{#d91a1a}-0.01\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 49.3720μs | 16.1722μs | 61.8346 KOps/s | 60.9628 KOps/s | $\color{#35bf28}+1.43\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 58.4090μs | 10.0071μs | 99.9289 KOps/s | 99.7886 KOps/s | $\color{#35bf28}+0.14\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 76.8630μs | 30.1318μs | 33.1876 KOps/s | 33.4408 KOps/s | $\color{#d91a1a}-0.76\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 49.2520μs | 19.5508μs | 51.1488 KOps/s | 51.6259 KOps/s | $\color{#d91a1a}-0.92\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 59.0700μs | 17.7553μs | 56.3211 KOps/s | 55.7219 KOps/s | $\color{#35bf28}+1.08\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 41.9190μs | 11.4820μs | 87.0930 KOps/s | 86.4920 KOps/s | $\color{#35bf28}+0.69\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 62.6170μs | 28.5562μs | 35.0187 KOps/s | 34.5006 KOps/s | $\color{#35bf28}+1.50\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 47.8600μs | 18.0403μs | 55.4313 KOps/s | 56.0544 KOps/s | $\color{#d91a1a}-1.11\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 73.5370μs | 18.6479μs | 53.6253 KOps/s | 53.3656 KOps/s | $\color{#35bf28}+0.49\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 35.9170μs | 11.3128μs | 88.3958 KOps/s | 88.3066 KOps/s | $\color{#35bf28}+0.10\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 70.4120μs | 30.2276μs | 33.0824 KOps/s | 32.7293 KOps/s | $\color{#35bf28}+1.08\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 57.9080μs | 19.4858μs | 51.3193 KOps/s | 51.2873 KOps/s | $\color{#35bf28}+0.06\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 55.8650μs | 20.5439μs | 48.6762 KOps/s | 48.9529 KOps/s | $\color{#d91a1a}-0.57\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 33.9130μs | 12.8760μs | 77.6637 KOps/s | 78.2648 KOps/s | $\color{#d91a1a}-0.77\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.1993ms | 32.2437μs | 31.0138 KOps/s | 30.9201 KOps/s | $\color{#35bf28}+0.30\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 69.6400μs | 20.9541μs | 47.7233 KOps/s | 46.3246 KOps/s | $\color{#35bf28}+3.02\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 46.5970μs | 20.2129μs | 49.4734 KOps/s | 48.5520 KOps/s | $\color{#35bf28}+1.90\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 38.0010μs | 12.7967μs | 78.1454 KOps/s | 77.8746 KOps/s | $\color{#35bf28}+0.35\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 75.4310μs | 33.1888μs | 30.1307 KOps/s | 29.7594 KOps/s | $\color{#35bf28}+1.25\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 62.2460μs | 22.3404μs | 44.7620 KOps/s | 44.2116 KOps/s | $\color{#35bf28}+1.25\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 52.5480μs | 21.3268μs | 46.8895 KOps/s | 46.3321 KOps/s | $\color{#35bf28}+1.20\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 51.8170μs | 14.2950μs | 69.9546 KOps/s | 70.2276 KOps/s | $\color{#d91a1a}-0.39\\%$ | | test_values[generalized_advantage_estimate-True-True] | 11.1561ms | 9.6193ms | 103.9574 Ops/s | 104.1545 Ops/s | $\color{#d91a1a}-0.19\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 37.7334ms | 35.4131ms | 28.2381 Ops/s | 27.5439 Ops/s | $\color{#35bf28}+2.52\\%$ | | test_values[td0_return_estimate-False-False] | 0.2316ms | 0.1806ms | 5.5378 KOps/s | 5.4416 KOps/s | $\color{#35bf28}+1.77\\%$ | | test_values[td1_return_estimate-False-False] | 25.0955ms | 23.9504ms | 41.7530 Ops/s | 42.3639 Ops/s | $\color{#d91a1a}-1.44\\%$ | | test_values[vec_td1_return_estimate-False-False] | 37.6315ms | 35.5339ms | 28.1421 Ops/s | 27.6299 Ops/s | $\color{#35bf28}+1.85\\%$ | | test_values[td_lambda_return_estimate-True-False] | 38.4385ms | 34.3600ms | 29.1036 Ops/s | 29.3818 Ops/s | $\color{#d91a1a}-0.95\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 38.5922ms | 35.5962ms | 28.0929 Ops/s | 27.5596 Ops/s | $\color{#35bf28}+1.93\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 10.0039ms | 8.4460ms | 118.3992 Ops/s | 119.8481 Ops/s | $\color{#d91a1a}-1.21\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 1.9725ms | 1.8771ms | 532.7239 Ops/s | 420.3443 Ops/s | $\textbf{\color{#35bf28}+26.74\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.5578ms | 0.3546ms | 2.8203 KOps/s | 2.7923 KOps/s | $\color{#35bf28}+1.00\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 49.8847ms | 45.7675ms | 21.8496 Ops/s | 21.1811 Ops/s | $\color{#35bf28}+3.16\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 4.0308ms | 3.0307ms | 329.9607 Ops/s | 326.1093 Ops/s | $\color{#35bf28}+1.18\\%$ | | test_dqn_speed | 1.5961ms | 1.3569ms | 736.9512 Ops/s | 700.5757 Ops/s | $\textbf{\color{#35bf28}+5.19\\%}$ | | test_ddpg_speed | 3.3141ms | 2.8694ms | 348.5088 Ops/s | 334.8046 Ops/s | $\color{#35bf28}+4.09\\%$ | | test_sac_speed | 9.1175ms | 8.4754ms | 117.9884 Ops/s | 113.7190 Ops/s | $\color{#35bf28}+3.75\\%$ | | test_redq_speed | 15.0999ms | 13.3965ms | 74.6462 Ops/s | 72.5092 Ops/s | $\color{#35bf28}+2.95\\%$ | | test_redq_deprec_speed | 14.1660ms | 13.4283ms | 74.4697 Ops/s | 73.4349 Ops/s | $\color{#35bf28}+1.41\\%$ | | test_td3_speed | 11.3908ms | 8.5370ms | 117.1371 Ops/s | 114.1965 Ops/s | $\color{#35bf28}+2.58\\%$ | | test_cql_speed | 38.7690ms | 36.5152ms | 27.3858 Ops/s | 27.0860 Ops/s | $\color{#35bf28}+1.11\\%$ | | test_a2c_speed | 9.0356ms | 7.4611ms | 134.0278 Ops/s | 131.5528 Ops/s | $\color{#35bf28}+1.88\\%$ | | test_ppo_speed | 9.3986ms | 7.6965ms | 129.9288 Ops/s | 127.9829 Ops/s | $\color{#35bf28}+1.52\\%$ | | test_reinforce_speed | 7.4480ms | 6.5915ms | 151.7100 Ops/s | 148.7342 Ops/s | $\color{#35bf28}+2.00\\%$ | | test_iql_speed | 34.4942ms | 32.4656ms | 30.8018 Ops/s | 30.5345 Ops/s | $\color{#35bf28}+0.88\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.8794ms | 4.2772ms | 233.7980 Ops/s | 234.5517 Ops/s | $\color{#d91a1a}-0.32\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7741ms | 0.4711ms | 2.1225 KOps/s | 2.0804 KOps/s | $\color{#35bf28}+2.03\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6335ms | 0.4517ms | 2.2136 KOps/s | 2.1983 KOps/s | $\color{#35bf28}+0.70\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.6326ms | 4.2448ms | 235.5813 Ops/s | 233.4228 Ops/s | $\color{#35bf28}+0.92\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7517ms | 0.4667ms | 2.1426 KOps/s | 2.0981 KOps/s | $\color{#35bf28}+2.12\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7302ms | 0.4486ms | 2.2293 KOps/s | 2.2113 KOps/s | $\color{#35bf28}+0.81\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.0201ms | 1.6947ms | 590.0870 Ops/s | 585.6066 Ops/s | $\color{#35bf28}+0.77\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.0127ms | 1.6093ms | 621.4064 Ops/s | 619.0936 Ops/s | $\color{#35bf28}+0.37\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.9111ms | 4.4536ms | 224.5377 Ops/s | 227.2591 Ops/s | $\color{#d91a1a}-1.20\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.6273ms | 0.6185ms | 1.6169 KOps/s | 1.6242 KOps/s | $\color{#d91a1a}-0.45\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7804ms | 0.5826ms | 1.7166 KOps/s | 1.6963 KOps/s | $\color{#35bf28}+1.20\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.2262ms | 4.2722ms | 234.0717 Ops/s | 233.4752 Ops/s | $\color{#35bf28}+0.26\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.5974ms | 0.4716ms | 2.1204 KOps/s | 2.0829 KOps/s | $\color{#35bf28}+1.80\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 7.1958ms | 0.4637ms | 2.1564 KOps/s | 2.1877 KOps/s | $\color{#d91a1a}-1.43\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.5953ms | 4.2624ms | 234.6075 Ops/s | 236.3622 Ops/s | $\color{#d91a1a}-0.74\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8063ms | 0.4671ms | 2.1407 KOps/s | 1.5883 KOps/s | $\textbf{\color{#35bf28}+34.78\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 1.1355ms | 0.4529ms | 2.2079 KOps/s | 2.2147 KOps/s | $\color{#d91a1a}-0.31\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.9062ms | 4.4188ms | 226.3077 Ops/s | 226.2987 Ops/s | $+0.00\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.4151ms | 0.6160ms | 1.6232 KOps/s | 1.6095 KOps/s | $\color{#35bf28}+0.85\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.1214s | 0.7450ms | 1.3422 KOps/s | 1.6657 KOps/s | $\textbf{\color{#d91a1a}-19.42\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1060s | 5.9202ms | 168.9120 Ops/s | 165.5311 Ops/s | $\color{#35bf28}+2.04\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 16.6580ms | 12.8447ms | 77.8529 Ops/s | 76.4545 Ops/s | $\color{#35bf28}+1.83\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 4.6610ms | 1.1935ms | 837.9024 Ops/s | 816.2497 Ops/s | $\color{#35bf28}+2.65\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1075s | 5.8687ms | 170.3947 Ops/s | 125.0218 Ops/s | $\textbf{\color{#35bf28}+36.29\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 16.8084ms | 12.7963ms | 78.1477 Ops/s | 76.5569 Ops/s | $\color{#35bf28}+2.08\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 5.6059ms | 1.2777ms | 782.6770 Ops/s | 839.8162 Ops/s | $\textbf{\color{#d91a1a}-6.80\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1059s | 8.0492ms | 124.2366 Ops/s | 160.2051 Ops/s | $\textbf{\color{#d91a1a}-22.45\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 17.8376ms | 13.1771ms | 75.8891 Ops/s | 75.7110 Ops/s | $\color{#35bf28}+0.24\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 4.9532ms | 1.3289ms | 752.4846 Ops/s | 788.6578 Ops/s | $\color{#d91a1a}-4.59\\%$ |
github-actions[bot] commented 2 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1068s | 0.1048s | 9.5392 Ops/s | 8.7328 Ops/s | $\textbf{\color{#35bf28}+9.23\\%}$ | | test_sync | 95.0866ms | 94.5113ms | 10.5807 Ops/s | 10.5880 Ops/s | $\color{#d91a1a}-0.07\\%$ | | test_async | 0.1769s | 89.3798ms | 11.1882 Ops/s | 11.3223 Ops/s | $\color{#d91a1a}-1.18\\%$ | | test_single_pixels | 0.1152s | 0.1146s | 8.7275 Ops/s | 8.6472 Ops/s | $\color{#35bf28}+0.93\\%$ | | test_sync_pixels | 84.8568ms | 82.1620ms | 12.1711 Ops/s | 13.4972 Ops/s | $\textbf{\color{#d91a1a}-9.83\\%}$ | | test_async_pixels | 0.1400s | 69.2471ms | 14.4410 Ops/s | 14.4768 Ops/s | $\color{#d91a1a}-0.25\\%$ | | test_simple | 0.8562s | 0.7833s | 1.2766 Ops/s | 1.2924 Ops/s | $\color{#d91a1a}-1.23\\%$ | | test_transformed | 1.0790s | 1.0017s | 0.9983 Ops/s | 1.0173 Ops/s | $\color{#d91a1a}-1.87\\%$ | | test_serial | 2.2511s | 2.1791s | 0.4589 Ops/s | 0.4581 Ops/s | $\color{#35bf28}+0.18\\%$ | | test_parallel | 2.0028s | 1.9388s | 0.5158 Ops/s | 0.5099 Ops/s | $\color{#35bf28}+1.16\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 97.1820μs | 39.0591μs | 25.6022 KOps/s | 26.8312 KOps/s | $\color{#d91a1a}-4.58\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 47.3110μs | 21.8699μs | 45.7250 KOps/s | 47.4785 KOps/s | $\color{#d91a1a}-3.69\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 54.8110μs | 22.9867μs | 43.5033 KOps/s | 47.4454 KOps/s | $\textbf{\color{#d91a1a}-8.31\\%}$ | | test_step_mdp_speed[True-True-True-False-False] | 31.8500μs | 13.2512μs | 75.4649 KOps/s | 83.4415 KOps/s | $\textbf{\color{#d91a1a}-9.56\\%}$ | | test_step_mdp_speed[True-True-False-True-True] | 67.8910μs | 40.7733μs | 24.5259 KOps/s | 25.8010 KOps/s | $\color{#d91a1a}-4.94\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 48.8910μs | 24.0340μs | 41.6078 KOps/s | 43.4492 KOps/s | $\color{#d91a1a}-4.24\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 48.3710μs | 23.7992μs | 42.0183 KOps/s | 43.5856 KOps/s | $\color{#d91a1a}-3.60\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 34.3810μs | 14.8299μs | 67.4315 KOps/s | 70.6553 KOps/s | $\color{#d91a1a}-4.56\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 73.4620μs | 43.1214μs | 23.1903 KOps/s | 24.1521 KOps/s | $\color{#d91a1a}-3.98\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 44.2320μs | 25.7457μs | 38.8414 KOps/s | 38.8393 KOps/s | $+0.01\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 72.9810μs | 23.7694μs | 42.0709 KOps/s | 43.2458 KOps/s | $\color{#d91a1a}-2.72\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 41.3810μs | 15.3818μs | 65.0120 KOps/s | 71.3628 KOps/s | $\textbf{\color{#d91a1a}-8.90\\%}$ | | test_step_mdp_speed[True-False-False-True-True] | 74.4910μs | 44.1763μs | 22.6365 KOps/s | 23.2014 KOps/s | $\color{#d91a1a}-2.43\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 54.0410μs | 28.1281μs | 35.5517 KOps/s | 36.4350 KOps/s | $\color{#d91a1a}-2.42\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 49.9610μs | 26.4158μs | 37.8562 KOps/s | 39.6800 KOps/s | $\color{#d91a1a}-4.60\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 37.1010μs | 17.0299μs | 58.7203 KOps/s | 62.3069 KOps/s | $\textbf{\color{#d91a1a}-5.76\\%}$ | | test_step_mdp_speed[False-True-True-True-True] | 63.9210μs | 41.9290μs | 23.8499 KOps/s | 24.1317 KOps/s | $\color{#d91a1a}-1.17\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 44.8610μs | 25.4868μs | 39.2360 KOps/s | 38.9932 KOps/s | $\color{#35bf28}+0.62\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 53.0310μs | 27.3549μs | 36.5566 KOps/s | 36.3936 KOps/s | $\color{#35bf28}+0.45\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 40.6110μs | 16.2792μs | 61.4279 KOps/s | 61.3855 KOps/s | $\color{#35bf28}+0.07\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 87.4720μs | 44.1092μs | 22.6710 KOps/s | 23.2652 KOps/s | $\color{#d91a1a}-2.55\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 45.1610μs | 27.6359μs | 36.1849 KOps/s | 36.2712 KOps/s | $\color{#d91a1a}-0.24\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 49.2110μs | 29.0719μs | 34.3975 KOps/s | 33.6976 KOps/s | $\color{#35bf28}+2.08\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 37.9210μs | 18.3773μs | 54.4151 KOps/s | 55.0100 KOps/s | $\color{#d91a1a}-1.08\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 3.8388ms | 46.9270μs | 21.3097 KOps/s | 21.6547 KOps/s | $\color{#d91a1a}-1.59\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 65.9720μs | 31.3409μs | 31.9072 KOps/s | 33.0163 KOps/s | $\color{#d91a1a}-3.36\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 76.9610μs | 30.5081μs | 32.7782 KOps/s | 34.0524 KOps/s | $\color{#d91a1a}-3.74\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 44.5110μs | 19.4030μs | 51.5384 KOps/s | 54.5584 KOps/s | $\textbf{\color{#d91a1a}-5.54\\%}$ | | test_step_mdp_speed[False-False-False-True-True] | 69.6010μs | 47.9345μs | 20.8618 KOps/s | 20.8422 KOps/s | $\color{#35bf28}+0.09\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 59.1010μs | 32.6670μs | 30.6119 KOps/s | 30.7696 KOps/s | $\color{#d91a1a}-0.51\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 54.0410μs | 31.2778μs | 31.9715 KOps/s | 31.9024 KOps/s | $\color{#35bf28}+0.22\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 36.2210μs | 20.2928μs | 49.2787 KOps/s | 49.1804 KOps/s | $\color{#35bf28}+0.20\\%$ | | test_values[generalized_advantage_estimate-True-True] | 24.7382ms | 24.1969ms | 41.3277 Ops/s | 42.4300 Ops/s | $\color{#d91a1a}-2.60\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 87.4574ms | 2.6422ms | 378.4757 Ops/s | 370.7744 Ops/s | $\color{#35bf28}+2.08\\%$ | | test_values[td0_return_estimate-False-False] | 87.9820μs | 65.0132μs | 15.3815 KOps/s | 15.4170 KOps/s | $\color{#d91a1a}-0.23\\%$ | | test_values[td1_return_estimate-False-False] | 58.4633ms | 55.0832ms | 18.1544 Ops/s | 18.7097 Ops/s | $\color{#d91a1a}-2.97\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.4106ms | 1.0777ms | 927.9428 Ops/s | 931.7806 Ops/s | $\color{#d91a1a}-0.41\\%$ | | test_values[td_lambda_return_estimate-True-False] | 93.1335ms | 89.9484ms | 11.1175 Ops/s | 11.7156 Ops/s | $\textbf{\color{#d91a1a}-5.11\\%}$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.4359ms | 1.0744ms | 930.7240 Ops/s | 935.9698 Ops/s | $\color{#d91a1a}-0.56\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 24.6302ms | 24.5087ms | 40.8018 Ops/s | 42.0931 Ops/s | $\color{#d91a1a}-3.07\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9286ms | 0.7086ms | 1.4113 KOps/s | 1.4290 KOps/s | $\color{#d91a1a}-1.24\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7402ms | 0.6620ms | 1.5105 KOps/s | 1.5341 KOps/s | $\color{#d91a1a}-1.54\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.4943ms | 1.4593ms | 685.2759 Ops/s | 689.8976 Ops/s | $\color{#d91a1a}-0.67\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7074ms | 0.6775ms | 1.4760 KOps/s | 1.4939 KOps/s | $\color{#d91a1a}-1.20\\%$ | | test_dqn_speed | 7.1509ms | 1.4189ms | 704.7888 Ops/s | 705.5252 Ops/s | $\color{#d91a1a}-0.10\\%$ | | test_ddpg_speed | 3.2336ms | 2.8898ms | 346.0462 Ops/s | 345.3894 Ops/s | $\color{#35bf28}+0.19\\%$ | | test_sac_speed | 0.1003s | 9.1717ms | 109.0316 Ops/s | 118.9261 Ops/s | $\textbf{\color{#d91a1a}-8.32\\%}$ | | test_redq_speed | 11.2316ms | 10.3888ms | 96.2578 Ops/s | 94.3094 Ops/s | $\color{#35bf28}+2.07\\%$ | | test_redq_deprec_speed | 11.4707ms | 11.0616ms | 90.4031 Ops/s | 88.8263 Ops/s | $\color{#35bf28}+1.78\\%$ | | test_td3_speed | 8.4518ms | 8.2404ms | 121.3538 Ops/s | 120.0487 Ops/s | $\color{#35bf28}+1.09\\%$ | | test_cql_speed | 25.7863ms | 25.2153ms | 39.6584 Ops/s | 35.4041 Ops/s | $\textbf{\color{#35bf28}+12.02\\%}$ | | test_a2c_speed | 6.2896ms | 5.6128ms | 178.1644 Ops/s | 181.0428 Ops/s | $\color{#d91a1a}-1.59\\%$ | | test_ppo_speed | 6.0498ms | 5.8793ms | 170.0875 Ops/s | 171.4888 Ops/s | $\color{#d91a1a}-0.82\\%$ | | test_reinforce_speed | 4.9107ms | 4.5853ms | 218.0886 Ops/s | 220.5392 Ops/s | $\color{#d91a1a}-1.11\\%$ | | test_iql_speed | 20.2832ms | 19.4666ms | 51.3699 Ops/s | 51.5694 Ops/s | $\color{#d91a1a}-0.39\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.8893ms | 5.7614ms | 173.5689 Ops/s | 172.2761 Ops/s | $\color{#35bf28}+0.75\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7572ms | 0.5038ms | 1.9849 KOps/s | 1.9791 KOps/s | $\color{#35bf28}+0.29\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6653ms | 0.4820ms | 2.0746 KOps/s | 2.0598 KOps/s | $\color{#35bf28}+0.72\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.8919ms | 5.6674ms | 176.4486 Ops/s | 176.6678 Ops/s | $\color{#d91a1a}-0.12\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.6424ms | 0.5008ms | 1.9969 KOps/s | 2.0084 KOps/s | $\color{#d91a1a}-0.58\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6787ms | 0.4776ms | 2.0939 KOps/s | 2.0965 KOps/s | $\color{#d91a1a}-0.12\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.0271ms | 1.8937ms | 528.0574 Ops/s | 528.9399 Ops/s | $\color{#d91a1a}-0.17\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 6.5221ms | 1.8054ms | 553.8806 Ops/s | 556.8181 Ops/s | $\color{#d91a1a}-0.53\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.9627ms | 5.8504ms | 170.9290 Ops/s | 168.9672 Ops/s | $\color{#35bf28}+1.16\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.4427ms | 0.6456ms | 1.5490 KOps/s | 1.5500 KOps/s | $\color{#d91a1a}-0.06\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.1516s | 0.7525ms | 1.3289 KOps/s | 1.5983 KOps/s | $\textbf{\color{#d91a1a}-16.86\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.8562ms | 5.7327ms | 174.4392 Ops/s | 174.2173 Ops/s | $\color{#35bf28}+0.13\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.4969ms | 0.5051ms | 1.9796 KOps/s | 1.9748 KOps/s | $\color{#35bf28}+0.24\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6666ms | 0.4831ms | 2.0699 KOps/s | 2.0638 KOps/s | $\color{#35bf28}+0.30\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.9266ms | 5.6576ms | 176.7523 Ops/s | 175.1910 Ops/s | $\color{#35bf28}+0.89\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8677ms | 0.4994ms | 2.0024 KOps/s | 1.9922 KOps/s | $\color{#35bf28}+0.51\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.5991ms | 0.4768ms | 2.0972 KOps/s | 2.0863 KOps/s | $\color{#35bf28}+0.52\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.2247ms | 5.9051ms | 169.3459 Ops/s | 170.4197 Ops/s | $\color{#d91a1a}-0.63\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.4504ms | 0.6446ms | 1.5513 KOps/s | 1.5463 KOps/s | $\color{#35bf28}+0.32\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7817ms | 0.6235ms | 1.6039 KOps/s | 1.5800 KOps/s | $\color{#35bf28}+1.52\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1418s | 7.8381ms | 127.5819 Ops/s | 127.3674 Ops/s | $\color{#35bf28}+0.17\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.6509ms | 15.6158ms | 64.0378 Ops/s | 63.7095 Ops/s | $\color{#35bf28}+0.52\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 6.5254ms | 1.3135ms | 761.3222 Ops/s | 806.1539 Ops/s | $\textbf{\color{#d91a1a}-5.56\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1231s | 7.4453ms | 134.3129 Ops/s | 132.4577 Ops/s | $\color{#35bf28}+1.40\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1336s | 18.0708ms | 55.3379 Ops/s | 63.5759 Ops/s | $\textbf{\color{#d91a1a}-12.96\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.8454ms | 1.0743ms | 930.7998 Ops/s | 766.7500 Ops/s | $\textbf{\color{#35bf28}+21.40\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1240s | 7.6339ms | 130.9952 Ops/s | 130.5813 Ops/s | $\color{#35bf28}+0.32\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.5202ms | 15.8617ms | 63.0448 Ops/s | 63.7865 Ops/s | $\color{#d91a1a}-1.16\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 6.2279ms | 1.4668ms | 681.7591 Ops/s | 742.1718 Ops/s | $\textbf{\color{#d91a1a}-8.14\\%}$ |