search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.27k stars 302 forks source link

[Feature] TicTacToeEnv #2301

Closed vmoens closed 2 months ago

vmoens commented 2 months ago

Stack from ghstack (oldest at bottom):

pytorch-bot[bot] commented 2 months ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2301

Note: Links to docs will display an error until the docs builds have been completed.

:x: 3 New Failures, 1 Pending, 4 Unrelated Failures

As of commit a689eed371ae4a1d8e750dedfc8a1b8152ef70b6 with merge base c771e6e858faa84a9084c7e3993c3d0a134d8ea3 (image):

NEW FAILURES - The following jobs have failed:

* [Examples Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2301#27751529279) ([gh](https://github.com/pytorch/rl/actions/runs/10042041840/job/27751529279)) `RuntimeError: Command docker exec -t 0b47d2a9a9d24dc6425c3dd7943741896efc9c6b7a7296e98e829c2d23c6242a /exec failed with exit code 1` * [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2301#27751528824) ([gh](https://github.com/pytorch/rl/actions/runs/10042041837/job/27751528824)) `RuntimeError: Command docker exec -t 5cb0ff5aa728ad75c4dc9f809151b47efa1073cc5dbf7adfb8164e724091be62 /exec failed with exit code 139` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2301#27751534312) ([gh](https://github.com/pytorch/rl/actions/runs/10042041845/job/27751534312)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

FLAKY - The following job failed but was likely due to flakiness present on trunk:

* [Libs Tests on Linux / unittests-smacv2 (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2301#27751537681) ([gh](https://github.com/pytorch/rl/actions/runs/10042041834/job/27751537681)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128`

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2301#27751536098) ([gh](https://github.com/pytorch/rl/actions/runs/10042041834/job/27751536098)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/c771e6e858faa84a9084c7e3993c3d0a134d8ea3#27674777898)) `AttributeError: module 'torch' has no attribute 'compiler'` * [Libs Tests on Linux / unittests-robohive (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2301#27751536755) ([gh](https://github.com/pytorch/rl/actions/runs/10042041834/job/27751536755)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/c771e6e858faa84a9084c7e3993c3d0a134d8ea3#27674778892)) `test/test_libs.py::TestRoboHive::test_robohive[franka_slide_random-v3-True-True]` * [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2301#27751533586) ([gh](https://github.com/pytorch/rl/actions/runs/10042041838/job/27751533586)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/c771e6e858faa84a9084c7e3993c3d0a134d8ea3#27674776256)) `AttributeError: module 'torch' has no attribute 'compiler'`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 2 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 62.4776ms | 57.5570ms | 17.3741 Ops/s | 17.0386 Ops/s | $\color{#35bf28}+1.97\\%$ | | test_sync | 41.1893ms | 32.3467ms | 30.9151 Ops/s | 31.4336 Ops/s | $\color{#d91a1a}-1.65\\%$ | | test_async | 80.4314ms | 30.5672ms | 32.7149 Ops/s | 33.5576 Ops/s | $\color{#d91a1a}-2.51\\%$ | | test_simple | 0.4785s | 0.4080s | 2.4508 Ops/s | 2.4228 Ops/s | $\color{#35bf28}+1.16\\%$ | | test_transformed | 0.6204s | 0.5615s | 1.7810 Ops/s | 1.7650 Ops/s | $\color{#35bf28}+0.91\\%$ | | test_serial | 1.3296s | 1.2676s | 0.7889 Ops/s | 0.7921 Ops/s | $\color{#d91a1a}-0.41\\%$ | | test_parallel | 1.1961s | 1.1244s | 0.8894 Ops/s | 0.9124 Ops/s | $\color{#d91a1a}-2.52\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1984ms | 25.8444μs | 38.6931 KOps/s | 39.6762 KOps/s | $\color{#d91a1a}-2.48\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 42.8900μs | 15.0108μs | 66.6187 KOps/s | 68.1327 KOps/s | $\color{#d91a1a}-2.22\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 67.7970μs | 14.9634μs | 66.8296 KOps/s | 68.7335 KOps/s | $\color{#d91a1a}-2.77\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 0.1649ms | 8.7112μs | 114.7944 KOps/s | 118.7064 KOps/s | $\color{#d91a1a}-3.30\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 72.6560μs | 27.9415μs | 35.7891 KOps/s | 37.0239 KOps/s | $\color{#d91a1a}-3.34\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 58.7690μs | 16.7482μs | 59.7078 KOps/s | 61.7239 KOps/s | $\color{#d91a1a}-3.27\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 0.1610ms | 17.2060μs | 58.1191 KOps/s | 62.1020 KOps/s | $\textbf{\color{#d91a1a}-6.41\\%}$ | | test_step_mdp_speed[True-True-False-False-False] | 50.9650μs | 10.3388μs | 96.7233 KOps/s | 97.0235 KOps/s | $\color{#d91a1a}-0.31\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 0.1056ms | 29.7813μs | 33.5781 KOps/s | 35.0246 KOps/s | $\color{#d91a1a}-4.13\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 73.2270μs | 18.4365μs | 54.2401 KOps/s | 56.0388 KOps/s | $\color{#d91a1a}-3.21\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 44.1320μs | 16.7463μs | 59.7147 KOps/s | 61.9634 KOps/s | $\color{#d91a1a}-3.63\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 57.9680μs | 10.3409μs | 96.7034 KOps/s | 99.8452 KOps/s | $\color{#d91a1a}-3.15\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 83.0450μs | 31.1889μs | 32.0627 KOps/s | 33.2180 KOps/s | $\color{#d91a1a}-3.48\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 49.7820μs | 20.2927μs | 49.2787 KOps/s | 51.0039 KOps/s | $\color{#d91a1a}-3.38\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 67.1350μs | 17.9733μs | 55.6381 KOps/s | 56.5443 KOps/s | $\color{#d91a1a}-1.60\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 53.2790μs | 11.7662μs | 84.9889 KOps/s | 85.1250 KOps/s | $\color{#d91a1a}-0.16\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 64.3900μs | 29.8548μs | 33.4955 KOps/s | 34.9149 KOps/s | $\color{#d91a1a}-4.07\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 68.0770μs | 18.7079μs | 53.4535 KOps/s | 55.9959 KOps/s | $\color{#d91a1a}-4.54\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 51.3760μs | 19.7265μs | 50.6932 KOps/s | 52.5220 KOps/s | $\color{#d91a1a}-3.48\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 62.1570μs | 11.6113μs | 86.1232 KOps/s | 87.2242 KOps/s | $\color{#d91a1a}-1.26\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 66.7950μs | 31.1334μs | 32.1198 KOps/s | 33.2802 KOps/s | $\color{#d91a1a}-3.49\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 51.5570μs | 20.0220μs | 49.9451 KOps/s | 51.3252 KOps/s | $\color{#d91a1a}-2.69\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 72.3150μs | 21.1977μs | 47.1750 KOps/s | 49.6105 KOps/s | $\color{#d91a1a}-4.91\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 50.6740μs | 13.2831μs | 75.2837 KOps/s | 78.0063 KOps/s | $\color{#d91a1a}-3.49\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 4.2380ms | 33.3253μs | 30.0072 KOps/s | 32.1965 KOps/s | $\textbf{\color{#d91a1a}-6.80\\%}$ | | test_step_mdp_speed[False-False-True-True-False] | 59.5920μs | 21.6148μs | 46.2647 KOps/s | 47.9227 KOps/s | $\color{#d91a1a}-3.46\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 65.1020μs | 21.0405μs | 47.5275 KOps/s | 50.0554 KOps/s | $\textbf{\color{#d91a1a}-5.05\\%}$ | | test_step_mdp_speed[False-False-True-False-False] | 62.0160μs | 13.0984μs | 76.3451 KOps/s | 77.9869 KOps/s | $\color{#d91a1a}-2.11\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 81.1010μs | 34.5389μs | 28.9528 KOps/s | 30.5040 KOps/s | $\textbf{\color{#d91a1a}-5.09\\%}$ | | test_step_mdp_speed[False-False-False-True-False] | 58.4090μs | 23.3883μs | 42.7564 KOps/s | 44.7668 KOps/s | $\color{#d91a1a}-4.49\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 68.6280μs | 22.0245μs | 45.4040 KOps/s | 46.9732 KOps/s | $\color{#d91a1a}-3.34\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 62.9670μs | 14.4966μs | 68.9816 KOps/s | 70.2570 KOps/s | $\color{#d91a1a}-1.82\\%$ | | test_values[generalized_advantage_estimate-True-True] | 10.0674ms | 9.7179ms | 102.9024 Ops/s | 104.4063 Ops/s | $\color{#d91a1a}-1.44\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 37.1173ms | 33.4412ms | 29.9032 Ops/s | 29.7445 Ops/s | $\color{#35bf28}+0.53\\%$ | | test_values[td0_return_estimate-False-False] | 0.2200ms | 0.1719ms | 5.8181 KOps/s | 5.6423 KOps/s | $\color{#35bf28}+3.12\\%$ | | test_values[td1_return_estimate-False-False] | 25.1046ms | 23.7206ms | 42.1575 Ops/s | 41.8518 Ops/s | $\color{#35bf28}+0.73\\%$ | | test_values[vec_td1_return_estimate-False-False] | 43.7004ms | 33.9982ms | 29.4133 Ops/s | 29.6831 Ops/s | $\color{#d91a1a}-0.91\\%$ | | test_values[td_lambda_return_estimate-True-False] | 35.7521ms | 34.3362ms | 29.1238 Ops/s | 29.1782 Ops/s | $\color{#d91a1a}-0.19\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 35.8816ms | 33.6015ms | 29.7606 Ops/s | 29.6171 Ops/s | $\color{#35bf28}+0.48\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.6968ms | 8.5156ms | 117.4313 Ops/s | 116.9582 Ops/s | $\color{#35bf28}+0.40\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 3.1702ms | 2.0014ms | 499.6512 Ops/s | 498.2443 Ops/s | $\color{#35bf28}+0.28\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7809ms | 0.3675ms | 2.7213 KOps/s | 2.8050 KOps/s | $\color{#d91a1a}-2.98\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 40.0659ms | 38.7118ms | 25.8319 Ops/s | 21.9748 Ops/s | $\textbf{\color{#35bf28}+17.55\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 4.1517ms | 3.0490ms | 327.9802 Ops/s | 329.8161 Ops/s | $\color{#d91a1a}-0.56\\%$ | | test_dqn_speed | 1.9872ms | 1.3585ms | 736.1085 Ops/s | 726.1531 Ops/s | $\color{#35bf28}+1.37\\%$ | | test_ddpg_speed | 3.6412ms | 2.8787ms | 347.3809 Ops/s | 343.3732 Ops/s | $\color{#35bf28}+1.17\\%$ | | test_sac_speed | 11.2524ms | 8.5997ms | 116.2831 Ops/s | 117.1962 Ops/s | $\color{#d91a1a}-0.78\\%$ | | test_redq_speed | 15.0818ms | 13.1419ms | 76.0926 Ops/s | 75.7677 Ops/s | $\color{#35bf28}+0.43\\%$ | | test_redq_deprec_speed | 15.7474ms | 13.1181ms | 76.2306 Ops/s | 75.9297 Ops/s | $\color{#35bf28}+0.40\\%$ | | test_td3_speed | 8.8064ms | 8.4906ms | 117.7772 Ops/s | 117.2847 Ops/s | $\color{#35bf28}+0.42\\%$ | | test_cql_speed | 42.2459ms | 36.6594ms | 27.2781 Ops/s | 27.2780 Ops/s | $+0.00\\%$ | | test_a2c_speed | 9.5468ms | 7.5238ms | 132.9115 Ops/s | 134.3883 Ops/s | $\color{#d91a1a}-1.10\\%$ | | test_ppo_speed | 9.2569ms | 7.8003ms | 128.2003 Ops/s | 129.2930 Ops/s | $\color{#d91a1a}-0.85\\%$ | | test_reinforce_speed | 8.1709ms | 6.6717ms | 149.8862 Ops/s | 149.1322 Ops/s | $\color{#35bf28}+0.51\\%$ | | test_iql_speed | 39.4661ms | 33.1830ms | 30.1359 Ops/s | 28.1301 Ops/s | $\textbf{\color{#35bf28}+7.13\\%}$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.5676ms | 4.4312ms | 225.6733 Ops/s | 231.2031 Ops/s | $\color{#d91a1a}-2.39\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6778ms | 0.4769ms | 2.0968 KOps/s | 2.0919 KOps/s | $\color{#35bf28}+0.23\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6662ms | 0.4497ms | 2.2239 KOps/s | 2.2135 KOps/s | $\color{#35bf28}+0.47\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.7219ms | 4.3148ms | 231.7626 Ops/s | 236.5561 Ops/s | $\color{#d91a1a}-2.03\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6761ms | 0.4721ms | 2.1181 KOps/s | 2.1328 KOps/s | $\color{#d91a1a}-0.69\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6531ms | 0.4473ms | 2.2358 KOps/s | 2.2360 KOps/s | $-0.01\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 3.0711ms | 1.6942ms | 590.2376 Ops/s | 586.2938 Ops/s | $\color{#35bf28}+0.67\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2429ms | 1.6009ms | 624.6594 Ops/s | 618.1720 Ops/s | $\color{#35bf28}+1.05\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.8191ms | 4.4994ms | 222.2535 Ops/s | 220.1947 Ops/s | $\color{#35bf28}+0.94\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.6412ms | 0.6132ms | 1.6308 KOps/s | 1.6083 KOps/s | $\color{#35bf28}+1.40\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 1.4488ms | 0.5910ms | 1.6920 KOps/s | 1.6848 KOps/s | $\color{#35bf28}+0.42\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.5682ms | 4.1961ms | 238.3148 Ops/s | 231.2556 Ops/s | $\color{#35bf28}+3.05\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7409ms | 0.4723ms | 2.1172 KOps/s | 2.0577 KOps/s | $\color{#35bf28}+2.89\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6679ms | 0.4498ms | 2.2230 KOps/s | 2.1974 KOps/s | $\color{#35bf28}+1.16\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.3538ms | 4.1580ms | 240.5018 Ops/s | 230.3308 Ops/s | $\color{#35bf28}+4.42\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.7667ms | 0.4703ms | 2.1264 KOps/s | 1.9171 KOps/s | $\textbf{\color{#35bf28}+10.92\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6094ms | 0.4432ms | 2.2565 KOps/s | 2.1976 KOps/s | $\color{#35bf28}+2.68\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.8142ms | 4.3434ms | 230.2366 Ops/s | 228.9744 Ops/s | $\color{#35bf28}+0.55\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8847ms | 0.6118ms | 1.6345 KOps/s | 1.6364 KOps/s | $\color{#d91a1a}-0.11\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 7.4442ms | 0.5906ms | 1.6931 KOps/s | 1.6765 KOps/s | $\color{#35bf28}+0.99\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1248s | 8.2137ms | 121.7478 Ops/s | 167.4388 Ops/s | $\textbf{\color{#d91a1a}-27.29\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 17.5846ms | 13.3168ms | 75.0933 Ops/s | 76.6041 Ops/s | $\color{#d91a1a}-1.97\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 6.2363ms | 1.1885ms | 841.3759 Ops/s | 916.7436 Ops/s | $\textbf{\color{#d91a1a}-8.22\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1024s | 5.7410ms | 174.1861 Ops/s | 170.7212 Ops/s | $\color{#35bf28}+2.03\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 17.6850ms | 13.4217ms | 74.5062 Ops/s | 66.7969 Ops/s | $\textbf{\color{#35bf28}+11.54\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.7850ms | 1.0773ms | 928.2287 Ops/s | 851.5148 Ops/s | $\textbf{\color{#35bf28}+9.01\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1073s | 6.0025ms | 166.5985 Ops/s | 167.0302 Ops/s | $\color{#d91a1a}-0.26\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 17.9352ms | 13.5094ms | 74.0224 Ops/s | 76.3671 Ops/s | $\color{#d91a1a}-3.07\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 5.3336ms | 1.3034ms | 767.1957 Ops/s | 807.6681 Ops/s | $\textbf{\color{#d91a1a}-5.01\\%}$ |
github-actions[bot] commented 2 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1052s | 0.1045s | 9.5694 Ops/s | 9.4842 Ops/s | $\color{#35bf28}+0.90\\%$ | | test_sync | 94.7172ms | 94.1670ms | 10.6194 Ops/s | 10.6328 Ops/s | $\color{#d91a1a}-0.13\\%$ | | test_async | 0.1785s | 90.2401ms | 11.0816 Ops/s | 11.2102 Ops/s | $\color{#d91a1a}-1.15\\%$ | | test_single_pixels | 0.1156s | 0.1149s | 8.7070 Ops/s | 8.6084 Ops/s | $\color{#35bf28}+1.14\\%$ | | test_sync_pixels | 77.3392ms | 74.4042ms | 13.4401 Ops/s | 13.5161 Ops/s | $\color{#d91a1a}-0.56\\%$ | | test_async_pixels | 0.1424s | 70.6607ms | 14.1521 Ops/s | 14.2332 Ops/s | $\color{#d91a1a}-0.57\\%$ | | test_simple | 0.7610s | 0.7602s | 1.3155 Ops/s | 1.3043 Ops/s | $\color{#35bf28}+0.86\\%$ | | test_transformed | 1.0815s | 1.0047s | 0.9953 Ops/s | 1.0039 Ops/s | $\color{#d91a1a}-0.85\\%$ | | test_serial | 2.2718s | 2.1946s | 0.4557 Ops/s | 0.4537 Ops/s | $\color{#35bf28}+0.44\\%$ | | test_parallel | 2.0101s | 1.9368s | 0.5163 Ops/s | 0.5116 Ops/s | $\color{#35bf28}+0.92\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1178ms | 36.2457μs | 27.5895 KOps/s | 27.0654 KOps/s | $\color{#35bf28}+1.94\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 78.9810μs | 21.0392μs | 47.5303 KOps/s | 47.7099 KOps/s | $\color{#d91a1a}-0.38\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 0.1458ms | 20.6649μs | 48.3913 KOps/s | 47.3290 KOps/s | $\color{#35bf28}+2.24\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 36.6910μs | 11.8422μs | 84.4436 KOps/s | 84.6969 KOps/s | $\color{#d91a1a}-0.30\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 80.5010μs | 38.8656μs | 25.7297 KOps/s | 25.1477 KOps/s | $\color{#35bf28}+2.31\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 42.9410μs | 23.0825μs | 43.3229 KOps/s | 43.0805 KOps/s | $\color{#35bf28}+0.56\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 79.9710μs | 22.8106μs | 43.8393 KOps/s | 43.4505 KOps/s | $\color{#35bf28}+0.89\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 31.9210μs | 14.0868μs | 70.9885 KOps/s | 70.8796 KOps/s | $\color{#35bf28}+0.15\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 68.9010μs | 41.8458μs | 23.8972 KOps/s | 23.7058 KOps/s | $\color{#35bf28}+0.81\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 51.0410μs | 25.4799μs | 39.2466 KOps/s | 39.4654 KOps/s | $\color{#d91a1a}-0.55\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 0.1868ms | 22.7095μs | 44.0345 KOps/s | 42.9724 KOps/s | $\color{#35bf28}+2.47\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 0.1926ms | 14.0962μs | 70.9412 KOps/s | 71.5272 KOps/s | $\color{#d91a1a}-0.82\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 73.6920μs | 43.1931μs | 23.1518 KOps/s | 22.8016 KOps/s | $\color{#35bf28}+1.54\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 44.2500μs | 27.5705μs | 36.2706 KOps/s | 36.5082 KOps/s | $\color{#d91a1a}-0.65\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 50.3110μs | 24.9065μs | 40.1502 KOps/s | 39.4928 KOps/s | $\color{#35bf28}+1.66\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 33.1210μs | 16.1867μs | 61.7790 KOps/s | 61.8911 KOps/s | $\color{#d91a1a}-0.18\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 60.5110μs | 41.1086μs | 24.3258 KOps/s | 24.1202 KOps/s | $\color{#35bf28}+0.85\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 44.8210μs | 25.6994μs | 38.9115 KOps/s | 39.7322 KOps/s | $\color{#d91a1a}-2.07\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 43.8210μs | 26.9786μs | 37.0665 KOps/s | 36.9598 KOps/s | $\color{#35bf28}+0.29\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 83.6910μs | 15.9725μs | 62.6076 KOps/s | 62.7187 KOps/s | $\color{#d91a1a}-0.18\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 73.2710μs | 43.2464μs | 23.1233 KOps/s | 22.9409 KOps/s | $\color{#35bf28}+0.80\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 52.5510μs | 27.5592μs | 36.2855 KOps/s | 36.2551 KOps/s | $\color{#35bf28}+0.08\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 91.4410μs | 29.1384μs | 34.3190 KOps/s | 34.1378 KOps/s | $\color{#35bf28}+0.53\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 35.5610μs | 18.0830μs | 55.3005 KOps/s | 54.1674 KOps/s | $\color{#35bf28}+2.09\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 4.8987ms | 46.2856μs | 21.6050 KOps/s | 21.3591 KOps/s | $\color{#35bf28}+1.15\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 56.5710μs | 30.1124μs | 33.2089 KOps/s | 33.0666 KOps/s | $\color{#35bf28}+0.43\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 51.3720μs | 29.3563μs | 34.0642 KOps/s | 33.7506 KOps/s | $\color{#35bf28}+0.93\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 35.0010μs | 18.1430μs | 55.1176 KOps/s | 53.7970 KOps/s | $\color{#35bf28}+2.45\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 0.1221ms | 47.2721μs | 21.1541 KOps/s | 21.0205 KOps/s | $\color{#35bf28}+0.64\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 70.4410μs | 32.0114μs | 31.2388 KOps/s | 31.0353 KOps/s | $\color{#35bf28}+0.66\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 75.9110μs | 30.6674μs | 32.6079 KOps/s | 31.9490 KOps/s | $\color{#35bf28}+2.06\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 40.6400μs | 20.2107μs | 49.4788 KOps/s | 49.2138 KOps/s | $\color{#35bf28}+0.54\\%$ | | test_values[generalized_advantage_estimate-True-True] | 25.1394ms | 24.2981ms | 41.1554 Ops/s | 40.2141 Ops/s | $\color{#35bf28}+2.34\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 97.3219ms | 2.8393ms | 352.2044 Ops/s | 371.3538 Ops/s | $\textbf{\color{#d91a1a}-5.16\\%}$ | | test_values[td0_return_estimate-False-False] | 88.0220μs | 65.3314μs | 15.3066 KOps/s | 15.1974 KOps/s | $\color{#35bf28}+0.72\\%$ | | test_values[td1_return_estimate-False-False] | 59.6481ms | 55.2703ms | 18.0929 Ops/s | 18.1477 Ops/s | $\color{#d91a1a}-0.30\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.4693ms | 1.1060ms | 904.1997 Ops/s | 918.0831 Ops/s | $\color{#d91a1a}-1.51\\%$ | | test_values[td_lambda_return_estimate-True-False] | 94.7405ms | 89.5573ms | 11.1660 Ops/s | 11.5534 Ops/s | $\color{#d91a1a}-3.35\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.4745ms | 1.0827ms | 923.6064 Ops/s | 930.0468 Ops/s | $\color{#d91a1a}-0.69\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 24.7933ms | 24.1418ms | 41.4219 Ops/s | 40.9624 Ops/s | $\color{#35bf28}+1.12\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9274ms | 0.7072ms | 1.4140 KOps/s | 1.3806 KOps/s | $\color{#35bf28}+2.42\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.8169ms | 0.6614ms | 1.5119 KOps/s | 1.4980 KOps/s | $\color{#35bf28}+0.92\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.7011ms | 1.4541ms | 687.7115 Ops/s | 681.7935 Ops/s | $\color{#35bf28}+0.87\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.8605ms | 0.6745ms | 1.4825 KOps/s | 1.4691 KOps/s | $\color{#35bf28}+0.92\\%$ | | test_dqn_speed | 7.9976ms | 1.4509ms | 689.2254 Ops/s | 688.4917 Ops/s | $\color{#35bf28}+0.11\\%$ | | test_ddpg_speed | 3.2002ms | 2.9653ms | 337.2381 Ops/s | 338.3535 Ops/s | $\color{#d91a1a}-0.33\\%$ | | test_sac_speed | 9.0138ms | 8.4970ms | 117.6887 Ops/s | 118.2006 Ops/s | $\color{#d91a1a}-0.43\\%$ | | test_redq_speed | 12.8761ms | 11.1869ms | 89.3901 Ops/s | 91.2153 Ops/s | $\color{#d91a1a}-2.00\\%$ | | test_redq_deprec_speed | 12.5370ms | 11.7705ms | 84.9585 Ops/s | 86.8403 Ops/s | $\color{#d91a1a}-2.17\\%$ | | test_td3_speed | 8.5957ms | 8.4838ms | 117.8721 Ops/s | 116.9467 Ops/s | $\color{#35bf28}+0.79\\%$ | | test_cql_speed | 27.5610ms | 26.8292ms | 37.2729 Ops/s | 37.4922 Ops/s | $\color{#d91a1a}-0.58\\%$ | | test_a2c_speed | 6.1940ms | 5.8819ms | 170.0122 Ops/s | 174.4415 Ops/s | $\color{#d91a1a}-2.54\\%$ | | test_ppo_speed | 6.9374ms | 6.2512ms | 159.9687 Ops/s | 166.1854 Ops/s | $\color{#d91a1a}-3.74\\%$ | | test_reinforce_speed | 5.1820ms | 4.8024ms | 208.2289 Ops/s | 211.4878 Ops/s | $\color{#d91a1a}-1.54\\%$ | | test_iql_speed | 21.8923ms | 20.8485ms | 47.9650 Ops/s | 49.3395 Ops/s | $\color{#d91a1a}-2.79\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.0313ms | 5.8516ms | 170.8947 Ops/s | 172.0070 Ops/s | $\color{#d91a1a}-0.65\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.7451ms | 0.5266ms | 1.8990 KOps/s | 1.8786 KOps/s | $\color{#35bf28}+1.08\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7263ms | 0.5068ms | 1.9731 KOps/s | 1.9453 KOps/s | $\color{#35bf28}+1.43\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.0898ms | 5.7866ms | 172.8145 Ops/s | 174.5521 Ops/s | $\color{#d91a1a}-1.00\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.0344ms | 0.5165ms | 1.9360 KOps/s | 1.9089 KOps/s | $\color{#35bf28}+1.42\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7031ms | 0.5021ms | 1.9918 KOps/s | 1.9787 KOps/s | $\color{#35bf28}+0.66\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.1621ms | 1.9780ms | 505.5637 Ops/s | 491.1467 Ops/s | $\color{#35bf28}+2.94\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.0879ms | 1.8952ms | 527.6524 Ops/s | 513.6753 Ops/s | $\color{#35bf28}+2.72\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.0862ms | 5.9101ms | 169.2018 Ops/s | 169.1085 Ops/s | $\color{#35bf28}+0.06\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.1540s | 0.8388ms | 1.1922 KOps/s | 1.4820 KOps/s | $\textbf{\color{#d91a1a}-19.56\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8185ms | 0.6552ms | 1.5263 KOps/s | 1.5318 KOps/s | $\color{#d91a1a}-0.36\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 6.0465ms | 5.8341ms | 171.4050 Ops/s | 173.8461 Ops/s | $\color{#d91a1a}-1.40\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6862ms | 0.5240ms | 1.9083 KOps/s | 1.8910 KOps/s | $\color{#35bf28}+0.92\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6938ms | 0.5069ms | 1.9729 KOps/s | 1.9569 KOps/s | $\color{#35bf28}+0.82\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 6.1045ms | 5.7475ms | 173.9891 Ops/s | 173.9422 Ops/s | $\color{#35bf28}+0.03\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6759ms | 0.5185ms | 1.9287 KOps/s | 1.9276 KOps/s | $\color{#35bf28}+0.06\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 4.4179ms | 0.5082ms | 1.9676 KOps/s | 1.9801 KOps/s | $\color{#d91a1a}-0.63\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 6.2555ms | 5.9908ms | 166.9222 Ops/s | 169.9373 Ops/s | $\color{#d91a1a}-1.77\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.1975ms | 0.6775ms | 1.4760 KOps/s | 1.4651 KOps/s | $\color{#35bf28}+0.75\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8561ms | 0.6525ms | 1.5325 KOps/s | 1.5241 KOps/s | $\color{#35bf28}+0.55\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1577s | 8.2984ms | 120.5045 Ops/s | 123.6297 Ops/s | $\color{#d91a1a}-2.53\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.2857ms | 15.7911ms | 63.3269 Ops/s | 63.6579 Ops/s | $\color{#d91a1a}-0.52\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 6.4839ms | 1.3780ms | 725.7143 Ops/s | 797.1342 Ops/s | $\textbf{\color{#d91a1a}-8.96\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1339s | 7.8001ms | 128.2041 Ops/s | 127.9255 Ops/s | $\color{#35bf28}+0.22\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 18.7768ms | 15.8799ms | 62.9725 Ops/s | 64.1981 Ops/s | $\color{#d91a1a}-1.91\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.2390ms | 1.2588ms | 794.4214 Ops/s | 737.3893 Ops/s | $\textbf{\color{#35bf28}+7.73\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1433s | 10.7953ms | 92.6327 Ops/s | 124.5188 Ops/s | $\textbf{\color{#d91a1a}-25.61\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.2606ms | 15.8775ms | 62.9822 Ops/s | 63.3780 Ops/s | $\color{#d91a1a}-0.62\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.4199ms | 1.4178ms | 705.3027 Ops/s | 716.2566 Ops/s | $\color{#d91a1a}-1.53\\%$ |