search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.25k stars 297 forks source link

[Quality] Fix repr of MARL modules #2192

Closed vmoens closed 4 months ago

vmoens commented 4 months ago

Example:

from torchrl.modules import MultiAgentConvNet
n_agents = 2
cnn = MultiAgentConvNet(
     n_agents,
     centralised = True,
     share_params = True
)
print(cnn)

prints

MultiAgentConvNet(
    ConvNet(
      (0): LazyConv2d(0, 32, kernel_size=(5, 5), stride=(2, 2))
      (1): ELU(alpha=1.0)
      (2): Conv2d(32, 32, kernel_size=(5, 5), stride=(2, 2))
      (3): ELU(alpha=1.0)
      (4): Conv2d(32, 32, kernel_size=(5, 5), stride=(2, 2))
      (5): ELU(alpha=1.0)
      (6): SquashDims()
    ),
    n_agents=2,
    share_params=True,
    centralized=True,
    agent_dim=-4)

I also switched to American english for centralized

image

cc @TheRisenPhoenix

pytorch-bot[bot] commented 4 months ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2192

Note: Links to docs will display an error until the docs builds have been completed.

:x: 5 New Failures, 1 Unrelated Failure

As of commit 610bf077a41400e3c74f9b3d8f1cbdfc2445b992 with merge base 14056009d6cc318e9248310f079f49d562a890d2 (image):

NEW FAILURES - The following jobs have failed:

* [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2192#25657246937) ([gh](https://github.com/pytorch/rl/actions/runs/9320443209/job/25657246937)) `RuntimeError: Command docker exec -t 3e821a1cf3a9ab98f62e184cafb5e4855783ab96db8a05dbc9e98defb764425e /exec failed with exit code 139` * [Unit-tests on Linux / tests-cpu (3.11) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2192#25657247992) ([gh](https://github.com/pytorch/rl/actions/runs/9320443288/job/25657247992)) `RuntimeError: Command docker exec -t a746f2dd86dcd058a3658fad5413e8125f7567ec4cec82dba913177c03d15de3 /exec failed with exit code 1` * [Unit-tests on Linux / tests-cpu (3.8) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2192#25657248814) ([gh](https://github.com/pytorch/rl/actions/runs/9320443288/job/25657248814)) `RuntimeError: Command docker exec -t 534e87a82a89dd38b82f92024750387a041cc506d2a6bc57914ab751cb6ca23c /exec failed with exit code 1` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2192#25657250138) ([gh](https://github.com/pytorch/rl/actions/runs/9320443288/job/25657250138)) `RuntimeError: Command docker exec -t 09d01dcb108dc94f172738fcafd63f619db1b3976e5e3fa2ecb13f8fc4c8f670 /exec failed with exit code 1` * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2192#25657246004) ([gh](https://github.com/pytorch/rl/actions/runs/9320443226/job/25657246004)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2192#25657250409) ([gh](https://github.com/pytorch/rl/actions/runs/9320443288/job/25657250409)) ([trunk failure](https://hud.pytorch.org/pytorch/rl/commit/14056009d6cc318e9248310f079f49d562a890d2#25656032983)) `test/test_transforms.py::TestVecNorm::test_state_dict_vecnorm`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 4 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1244s | 60.3928ms | 16.5583 Ops/s | 17.5637 Ops/s | $\textbf{\color{#d91a1a}-5.72\\%}$ | | test_sync | 47.8417ms | 32.0117ms | 31.2385 Ops/s | 32.9002 Ops/s | $\textbf{\color{#d91a1a}-5.05\\%}$ | | test_async | 49.8794ms | 28.8832ms | 34.6222 Ops/s | 35.1298 Ops/s | $\color{#d91a1a}-1.44\\%$ | | test_simple | 0.4674s | 0.3978s | 2.5141 Ops/s | 2.6185 Ops/s | $\color{#d91a1a}-3.98\\%$ | | test_transformed | 0.6020s | 0.5515s | 1.8132 Ops/s | 1.8327 Ops/s | $\color{#d91a1a}-1.07\\%$ | | test_serial | 1.3224s | 1.2760s | 0.7837 Ops/s | 0.7887 Ops/s | $\color{#d91a1a}-0.63\\%$ | | test_parallel | 1.1266s | 1.0764s | 0.9290 Ops/s | 0.9378 Ops/s | $\color{#d91a1a}-0.93\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1644ms | 21.4211μs | 46.6830 KOps/s | 45.4807 KOps/s | $\color{#35bf28}+2.64\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 43.4420μs | 12.9458μs | 77.2450 KOps/s | 74.8967 KOps/s | $\color{#35bf28}+3.14\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 42.4190μs | 12.6730μs | 78.9077 KOps/s | 77.1416 KOps/s | $\color{#35bf28}+2.29\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 59.2210μs | 7.6725μs | 130.3351 KOps/s | 127.2484 KOps/s | $\color{#35bf28}+2.43\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 57.8980μs | 22.6519μs | 44.1464 KOps/s | 42.7824 KOps/s | $\color{#35bf28}+3.19\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 42.1790μs | 14.1123μs | 70.8601 KOps/s | 67.8865 KOps/s | $\color{#35bf28}+4.38\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 44.8640μs | 13.7745μs | 72.5978 KOps/s | 70.3443 KOps/s | $\color{#35bf28}+3.20\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 34.2440μs | 8.8777μs | 112.6419 KOps/s | 108.6441 KOps/s | $\color{#35bf28}+3.68\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 64.7020μs | 24.1075μs | 41.4809 KOps/s | 40.3307 KOps/s | $\color{#35bf28}+2.85\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 45.7160μs | 15.5834μs | 64.1709 KOps/s | 62.3116 KOps/s | $\color{#35bf28}+2.98\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 41.5380μs | 13.8598μs | 72.1509 KOps/s | 70.8485 KOps/s | $\color{#35bf28}+1.84\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 34.6150μs | 8.9048μs | 112.2990 KOps/s | 109.5553 KOps/s | $\color{#35bf28}+2.50\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 57.9690μs | 25.2032μs | 39.6775 KOps/s | 38.3556 KOps/s | $\color{#35bf28}+3.45\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 57.5180μs | 16.8666μs | 59.2888 KOps/s | 57.7216 KOps/s | $\color{#35bf28}+2.72\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 48.1400μs | 15.1308μs | 66.0902 KOps/s | 63.5575 KOps/s | $\color{#35bf28}+3.98\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 63.0380μs | 10.0182μs | 99.8185 KOps/s | 96.1425 KOps/s | $\color{#35bf28}+3.82\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 54.7130μs | 23.9125μs | 41.8192 KOps/s | 40.1636 KOps/s | $\color{#35bf28}+4.12\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 40.7370μs | 15.5144μs | 64.4563 KOps/s | 61.6539 KOps/s | $\color{#35bf28}+4.55\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 55.8950μs | 15.9874μs | 62.5494 KOps/s | 60.2641 KOps/s | $\color{#35bf28}+3.79\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 37.6110μs | 9.9609μs | 100.3924 KOps/s | 95.5682 KOps/s | $\textbf{\color{#35bf28}+5.05\\%}$ | | test_step_mdp_speed[False-True-False-True-True] | 69.0590μs | 25.1941μs | 39.6918 KOps/s | 38.4990 KOps/s | $\color{#35bf28}+3.10\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 66.3240μs | 16.6955μs | 59.8964 KOps/s | 57.5187 KOps/s | $\color{#35bf28}+4.13\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 43.9720μs | 17.3666μs | 57.5819 KOps/s | 56.6445 KOps/s | $\color{#35bf28}+1.65\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 35.2360μs | 11.2438μs | 88.9378 KOps/s | 86.6512 KOps/s | $\color{#35bf28}+2.64\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 59.5420μs | 26.3563μs | 37.9417 KOps/s | 36.1234 KOps/s | $\textbf{\color{#35bf28}+5.03\\%}$ | | test_step_mdp_speed[False-False-True-True-False] | 58.3300μs | 17.9307μs | 55.7703 KOps/s | 50.2701 KOps/s | $\textbf{\color{#35bf28}+10.94\\%}$ | | test_step_mdp_speed[False-False-True-False-True] | 49.8940μs | 17.2034μs | 58.1279 KOps/s | 56.0383 KOps/s | $\color{#35bf28}+3.73\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 53.1300μs | 11.1970μs | 89.3098 KOps/s | 85.5352 KOps/s | $\color{#35bf28}+4.41\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 44.8040μs | 27.7918μs | 35.9819 KOps/s | 34.8934 KOps/s | $\color{#35bf28}+3.12\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 50.2840μs | 19.0461μs | 52.5042 KOps/s | 50.4108 KOps/s | $\color{#35bf28}+4.15\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 48.0910μs | 18.0310μs | 55.4600 KOps/s | 53.8088 KOps/s | $\color{#35bf28}+3.07\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 51.5570μs | 12.2678μs | 81.5143 KOps/s | 79.7545 KOps/s | $\color{#35bf28}+2.21\\%$ | | test_values[generalized_advantage_estimate-True-True] | 12.4158ms | 9.6440ms | 103.6919 Ops/s | 104.7804 Ops/s | $\color{#d91a1a}-1.04\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 37.2186ms | 33.6382ms | 29.7281 Ops/s | 28.2032 Ops/s | $\textbf{\color{#35bf28}+5.41\\%}$ | | test_values[td0_return_estimate-False-False] | 0.2221ms | 0.1836ms | 5.4477 KOps/s | 6.0473 KOps/s | $\textbf{\color{#d91a1a}-9.92\\%}$ | | test_values[td1_return_estimate-False-False] | 27.2884ms | 24.2474ms | 41.2415 Ops/s | 41.9163 Ops/s | $\color{#d91a1a}-1.61\\%$ | | test_values[vec_td1_return_estimate-False-False] | 34.7176ms | 33.5964ms | 29.7651 Ops/s | 28.0568 Ops/s | $\textbf{\color{#35bf28}+6.09\\%}$ | | test_values[td_lambda_return_estimate-True-False] | 35.2595ms | 34.5237ms | 28.9656 Ops/s | 28.7427 Ops/s | $\color{#35bf28}+0.78\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 35.1627ms | 33.6536ms | 29.7145 Ops/s | 28.1535 Ops/s | $\textbf{\color{#35bf28}+5.54\\%}$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 13.6900ms | 8.4619ms | 118.1772 Ops/s | 119.4832 Ops/s | $\color{#d91a1a}-1.09\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.5744ms | 1.9404ms | 515.3644 Ops/s | 519.8215 Ops/s | $\color{#d91a1a}-0.86\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.6736ms | 0.3613ms | 2.7678 KOps/s | 2.7863 KOps/s | $\color{#d91a1a}-0.66\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 46.3736ms | 44.8558ms | 22.2937 Ops/s | 21.7814 Ops/s | $\color{#35bf28}+2.35\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.9248ms | 3.0625ms | 326.5358 Ops/s | 327.8747 Ops/s | $\color{#d91a1a}-0.41\\%$ | | test_dqn_speed | 1.5265ms | 1.3883ms | 720.2993 Ops/s | 718.2762 Ops/s | $\color{#35bf28}+0.28\\%$ | | test_ddpg_speed | 4.6363ms | 2.9534ms | 338.5962 Ops/s | 339.9358 Ops/s | $\color{#d91a1a}-0.39\\%$ | | test_sac_speed | 9.9183ms | 8.8270ms | 113.2885 Ops/s | 114.5877 Ops/s | $\color{#d91a1a}-1.13\\%$ | | test_redq_speed | 14.6545ms | 13.7749ms | 72.5957 Ops/s | 74.3781 Ops/s | $\color{#d91a1a}-2.40\\%$ | | test_redq_deprec_speed | 15.3805ms | 13.7694ms | 72.6250 Ops/s | 73.7555 Ops/s | $\color{#d91a1a}-1.53\\%$ | | test_td3_speed | 9.9742ms | 8.6537ms | 115.5573 Ops/s | 116.1914 Ops/s | $\color{#d91a1a}-0.55\\%$ | | test_cql_speed | 39.3079ms | 37.0502ms | 26.9904 Ops/s | 27.2588 Ops/s | $\color{#d91a1a}-0.98\\%$ | | test_a2c_speed | 8.8823ms | 7.5776ms | 131.9677 Ops/s | 132.8370 Ops/s | $\color{#d91a1a}-0.65\\%$ | | test_ppo_speed | 9.1667ms | 7.7874ms | 128.4118 Ops/s | 126.7203 Ops/s | $\color{#35bf28}+1.33\\%$ | | test_reinforce_speed | 8.9363ms | 6.8892ms | 145.1539 Ops/s | 145.2288 Ops/s | $\color{#d91a1a}-0.05\\%$ | | test_iql_speed | 34.1327ms | 32.9995ms | 30.3035 Ops/s | 30.1717 Ops/s | $\color{#35bf28}+0.44\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.8650ms | 3.6068ms | 277.2561 Ops/s | 279.4444 Ops/s | $\color{#d91a1a}-0.78\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.9970ms | 0.5121ms | 1.9528 KOps/s | 1.9816 KOps/s | $\color{#d91a1a}-1.45\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7144ms | 0.4801ms | 2.0828 KOps/s | 2.0875 KOps/s | $\color{#d91a1a}-0.23\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.7994ms | 3.5371ms | 282.7192 Ops/s | 276.3124 Ops/s | $\color{#35bf28}+2.32\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.0304ms | 0.4895ms | 2.0429 KOps/s | 1.9925 KOps/s | $\color{#35bf28}+2.53\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 99.6846ms | 0.5521ms | 1.8111 KOps/s | 2.0624 KOps/s | $\textbf{\color{#d91a1a}-12.18\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 1.4874ms | 1.2543ms | 797.2814 Ops/s | 791.1037 Ops/s | $\color{#35bf28}+0.78\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 1.4486ms | 1.1840ms | 844.5599 Ops/s | 837.2002 Ops/s | $\color{#35bf28}+0.88\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.4694ms | 3.6507ms | 273.9218 Ops/s | 268.1619 Ops/s | $\color{#35bf28}+2.15\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.1054ms | 0.6235ms | 1.6040 KOps/s | 1.6041 KOps/s | $\color{#d91a1a}-0.01\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8260ms | 0.6013ms | 1.6630 KOps/s | 1.6754 KOps/s | $\color{#d91a1a}-0.74\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.4859ms | 3.5514ms | 281.5762 Ops/s | 284.4818 Ops/s | $\color{#d91a1a}-1.02\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7340ms | 0.5083ms | 1.9672 KOps/s | 1.6031 KOps/s | $\textbf{\color{#35bf28}+22.71\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.9990ms | 0.4890ms | 2.0451 KOps/s | 2.0809 KOps/s | $\color{#d91a1a}-1.72\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.6628ms | 3.7588ms | 266.0393 Ops/s | 281.3102 Ops/s | $\textbf{\color{#d91a1a}-5.43\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7729ms | 0.5014ms | 1.9944 KOps/s | 2.0032 KOps/s | $\color{#d91a1a}-0.44\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7441ms | 0.4732ms | 2.1134 KOps/s | 2.0737 KOps/s | $\color{#35bf28}+1.92\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.5416ms | 3.8521ms | 259.5956 Ops/s | 272.9537 Ops/s | $\color{#d91a1a}-4.89\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.2981ms | 0.6434ms | 1.5543 KOps/s | 1.6101 KOps/s | $\color{#d91a1a}-3.47\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8618ms | 0.6216ms | 1.6086 KOps/s | 1.6661 KOps/s | $\color{#d91a1a}-3.45\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1162s | 5.9866ms | 167.0385 Ops/s | 178.9675 Ops/s | $\textbf{\color{#d91a1a}-6.67\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 15.3032ms | 12.7899ms | 78.1867 Ops/s | 78.1862 Ops/s | $+0.00\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 4.5269ms | 1.2005ms | 832.9745 Ops/s | 931.6684 Ops/s | $\textbf{\color{#d91a1a}-10.59\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 95.5180ms | 7.3139ms | 136.7267 Ops/s | 131.7521 Ops/s | $\color{#35bf28}+3.78\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 15.2187ms | 12.7128ms | 78.6607 Ops/s | 77.8199 Ops/s | $\color{#35bf28}+1.08\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.5313ms | 1.0853ms | 921.3984 Ops/s | 939.0510 Ops/s | $\color{#d91a1a}-1.88\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 93.3394ms | 5.8373ms | 171.3130 Ops/s | 169.5668 Ops/s | $\color{#35bf28}+1.03\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 16.8994ms | 13.0646ms | 76.5429 Ops/s | 76.0376 Ops/s | $\color{#35bf28}+0.66\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 3.9579ms | 1.4566ms | 686.5122 Ops/s | 696.3197 Ops/s | $\color{#d91a1a}-1.41\\%$ |
github-actions[bot] commented 4 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1175s | 0.1152s | 8.6770 Ops/s | 8.5396 Ops/s | $\color{#35bf28}+1.61\\%$ | | test_sync | 0.1051s | 0.1036s | 9.6495 Ops/s | 9.5017 Ops/s | $\color{#35bf28}+1.56\\%$ | | test_async | 0.1990s | 80.3610ms | 12.4438 Ops/s | 10.2381 Ops/s | $\textbf{\color{#35bf28}+21.54\\%}$ | | test_single_pixels | 0.1259s | 0.1256s | 7.9588 Ops/s | 7.8993 Ops/s | $\color{#35bf28}+0.75\\%$ | | test_sync_pixels | 85.6899ms | 82.7715ms | 12.0815 Ops/s | 12.3602 Ops/s | $\color{#d91a1a}-2.26\\%$ | | test_async_pixels | 0.1501s | 64.6184ms | 15.4755 Ops/s | 14.3869 Ops/s | $\textbf{\color{#35bf28}+7.57\\%}$ | | test_simple | 0.7895s | 0.7888s | 1.2677 Ops/s | 1.2530 Ops/s | $\color{#35bf28}+1.17\\%$ | | test_transformed | 1.0453s | 1.0399s | 0.9616 Ops/s | 0.9435 Ops/s | $\color{#35bf28}+1.91\\%$ | | test_serial | 2.4896s | 2.4398s | 0.4099 Ops/s | 0.4013 Ops/s | $\color{#35bf28}+2.15\\%$ | | test_parallel | 2.4359s | 2.3550s | 0.4246 Ops/s | 0.4234 Ops/s | $\color{#35bf28}+0.28\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1082ms | 33.8097μs | 29.5773 KOps/s | 30.4046 KOps/s | $\color{#d91a1a}-2.72\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 0.2148ms | 19.8285μs | 50.4325 KOps/s | 50.7572 KOps/s | $\color{#d91a1a}-0.64\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 0.2256ms | 19.0243μs | 52.5642 KOps/s | 52.4169 KOps/s | $\color{#35bf28}+0.28\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 28.5400μs | 11.0366μs | 90.6074 KOps/s | 88.2548 KOps/s | $\color{#35bf28}+2.67\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 61.5520μs | 34.7897μs | 28.7441 KOps/s | 28.6699 KOps/s | $\color{#35bf28}+0.26\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 52.9610μs | 21.8924μs | 45.6779 KOps/s | 46.6798 KOps/s | $\color{#d91a1a}-2.15\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 51.2710μs | 20.8296μs | 48.0085 KOps/s | 48.4927 KOps/s | $\color{#d91a1a}-1.00\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 0.3060ms | 13.1472μs | 76.0617 KOps/s | 75.6633 KOps/s | $\color{#35bf28}+0.53\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 0.2330ms | 36.8925μs | 27.1058 KOps/s | 27.2274 KOps/s | $\color{#d91a1a}-0.45\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 0.2231ms | 23.4727μs | 42.6026 KOps/s | 42.6249 KOps/s | $\color{#d91a1a}-0.05\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 66.0120μs | 21.0144μs | 47.5864 KOps/s | 48.3800 KOps/s | $\color{#d91a1a}-1.64\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 0.2235ms | 13.0253μs | 76.7735 KOps/s | 75.9550 KOps/s | $\color{#35bf28}+1.08\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 0.2695ms | 38.7069μs | 25.8352 KOps/s | 25.9127 KOps/s | $\color{#d91a1a}-0.30\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 53.3510μs | 24.4880μs | 40.8363 KOps/s | 39.8576 KOps/s | $\color{#35bf28}+2.46\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 0.2120ms | 22.4288μs | 44.5855 KOps/s | 44.9366 KOps/s | $\color{#d91a1a}-0.78\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 0.2045ms | 14.7306μs | 67.8858 KOps/s | 67.3701 KOps/s | $\color{#35bf28}+0.77\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 0.3298ms | 36.6484μs | 27.2863 KOps/s | 26.8423 KOps/s | $\color{#35bf28}+1.65\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 48.1810μs | 23.1017μs | 43.2869 KOps/s | 42.3681 KOps/s | $\color{#35bf28}+2.17\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 0.2166ms | 24.6495μs | 40.5688 KOps/s | 39.7229 KOps/s | $\color{#35bf28}+2.13\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 0.2124ms | 15.0470μs | 66.4585 KOps/s | 66.7310 KOps/s | $\color{#d91a1a}-0.41\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 0.2368ms | 38.7822μs | 25.7850 KOps/s | 25.3960 KOps/s | $\color{#35bf28}+1.53\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 50.1110μs | 24.8449μs | 40.2497 KOps/s | 39.6634 KOps/s | $\color{#35bf28}+1.48\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 48.2010μs | 25.7360μs | 38.8561 KOps/s | 37.2798 KOps/s | $\color{#35bf28}+4.23\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 0.2390ms | 17.0653μs | 58.5984 KOps/s | 59.7471 KOps/s | $\color{#d91a1a}-1.92\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 0.2403ms | 41.4178μs | 24.1442 KOps/s | 24.5996 KOps/s | $\color{#d91a1a}-1.85\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 50.3310μs | 27.5200μs | 36.3373 KOps/s | 36.6115 KOps/s | $\color{#d91a1a}-0.75\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 0.2622ms | 26.2396μs | 38.1103 KOps/s | 36.6618 KOps/s | $\color{#35bf28}+3.95\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 0.2141ms | 16.7807μs | 59.5923 KOps/s | 58.8054 KOps/s | $\color{#35bf28}+1.34\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 66.2310μs | 42.0793μs | 23.7647 KOps/s | 23.1388 KOps/s | $\color{#35bf28}+2.70\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 0.2391ms | 28.7833μs | 34.7424 KOps/s | 33.8646 KOps/s | $\color{#35bf28}+2.59\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 61.4010μs | 28.5779μs | 34.9921 KOps/s | 35.0299 KOps/s | $\color{#d91a1a}-0.11\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 0.2101ms | 18.1543μs | 55.0834 KOps/s | 53.1121 KOps/s | $\color{#35bf28}+3.71\\%$ | | test_values[generalized_advantage_estimate-True-True] | 24.9864ms | 24.3927ms | 40.9959 Ops/s | 41.6591 Ops/s | $\color{#d91a1a}-1.59\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 83.0404ms | 3.2172ms | 310.8272 Ops/s | 314.9615 Ops/s | $\color{#d91a1a}-1.31\\%$ | | test_values[td0_return_estimate-False-False] | 90.4320μs | 61.4139μs | 16.2830 KOps/s | 15.6794 KOps/s | $\color{#35bf28}+3.85\\%$ | | test_values[td1_return_estimate-False-False] | 53.7919ms | 52.3259ms | 19.1110 Ops/s | 19.4648 Ops/s | $\color{#d91a1a}-1.82\\%$ | | test_values[vec_td1_return_estimate-False-False] | 2.0208ms | 1.7406ms | 574.5024 Ops/s | 568.5587 Ops/s | $\color{#35bf28}+1.05\\%$ | | test_values[td_lambda_return_estimate-True-False] | 85.8344ms | 81.8206ms | 12.2219 Ops/s | 12.2024 Ops/s | $\color{#35bf28}+0.16\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 2.0109ms | 1.7472ms | 572.3311 Ops/s | 570.8438 Ops/s | $\color{#35bf28}+0.26\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 24.0019ms | 23.6986ms | 42.1965 Ops/s | 44.0321 Ops/s | $\color{#d91a1a}-4.17\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.8899ms | 0.6687ms | 1.4955 KOps/s | 1.4683 KOps/s | $\color{#35bf28}+1.85\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7109ms | 0.6458ms | 1.5486 KOps/s | 1.5584 KOps/s | $\color{#d91a1a}-0.63\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5127ms | 1.4299ms | 699.3621 Ops/s | 694.3061 Ops/s | $\color{#35bf28}+0.73\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.9186ms | 0.6603ms | 1.5144 KOps/s | 1.5183 KOps/s | $\color{#d91a1a}-0.26\\%$ | | test_dqn_speed | 1.6333ms | 1.3617ms | 734.3771 Ops/s | 637.1351 Ops/s | $\textbf{\color{#35bf28}+15.26\\%}$ | | test_ddpg_speed | 3.2012ms | 2.8386ms | 352.2884 Ops/s | 337.8483 Ops/s | $\color{#35bf28}+4.27\\%$ | | test_sac_speed | 8.3435ms | 8.0373ms | 124.4197 Ops/s | 121.5057 Ops/s | $\color{#35bf28}+2.40\\%$ | | test_redq_speed | 11.9830ms | 10.3228ms | 96.8728 Ops/s | 95.7568 Ops/s | $\color{#35bf28}+1.17\\%$ | | test_redq_deprec_speed | 11.6722ms | 11.1533ms | 89.6599 Ops/s | 87.4594 Ops/s | $\color{#35bf28}+2.52\\%$ | | test_td3_speed | 8.3805ms | 8.0186ms | 124.7108 Ops/s | 123.1281 Ops/s | $\color{#35bf28}+1.29\\%$ | | test_cql_speed | 25.6926ms | 24.8830ms | 40.1881 Ops/s | 39.6103 Ops/s | $\color{#35bf28}+1.46\\%$ | | test_a2c_speed | 5.6707ms | 5.3835ms | 185.7516 Ops/s | 181.6778 Ops/s | $\color{#35bf28}+2.24\\%$ | | test_ppo_speed | 5.9621ms | 5.6688ms | 176.4040 Ops/s | 170.0432 Ops/s | $\color{#35bf28}+3.74\\%$ | | test_reinforce_speed | 5.2312ms | 4.4246ms | 226.0097 Ops/s | 222.1502 Ops/s | $\color{#35bf28}+1.74\\%$ | | test_iql_speed | 19.5533ms | 18.9337ms | 52.8159 Ops/s | 52.7265 Ops/s | $\color{#35bf28}+0.17\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.8394ms | 4.5337ms | 220.5727 Ops/s | 215.8166 Ops/s | $\color{#35bf28}+2.20\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.3619ms | 0.5839ms | 1.7127 KOps/s | 1.7083 KOps/s | $\color{#35bf28}+0.26\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.8190ms | 0.5594ms | 1.7877 KOps/s | 1.7763 KOps/s | $\color{#35bf28}+0.64\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.8717ms | 4.5132ms | 221.5703 Ops/s | 218.9978 Ops/s | $\color{#35bf28}+1.17\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.4755ms | 0.5743ms | 1.7413 KOps/s | 1.7312 KOps/s | $\color{#35bf28}+0.58\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7756ms | 0.5501ms | 1.8178 KOps/s | 1.7883 KOps/s | $\color{#35bf28}+1.65\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 4.5326ms | 1.6777ms | 596.0517 Ops/s | 679.5827 Ops/s | $\textbf{\color{#d91a1a}-12.29\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 1.5767ms | 1.3812ms | 723.9839 Ops/s | 707.8918 Ops/s | $\color{#35bf28}+2.27\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.9759ms | 4.6704ms | 214.1154 Ops/s | 211.6172 Ops/s | $\color{#35bf28}+1.18\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8235ms | 0.7020ms | 1.4245 KOps/s | 1.4119 KOps/s | $\color{#35bf28}+0.89\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.3178ms | 0.6802ms | 1.4702 KOps/s | 1.4590 KOps/s | $\color{#35bf28}+0.77\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.7663ms | 4.5313ms | 220.6859 Ops/s | 218.0293 Ops/s | $\color{#35bf28}+1.22\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.9201ms | 0.5819ms | 1.7185 KOps/s | 1.7081 KOps/s | $\color{#35bf28}+0.61\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6697ms | 0.5569ms | 1.7956 KOps/s | 1.7641 KOps/s | $\color{#35bf28}+1.79\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.8821ms | 4.5919ms | 217.7770 Ops/s | 217.8511 Ops/s | $\color{#d91a1a}-0.03\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.3565ms | 0.5764ms | 1.7350 KOps/s | 1.7288 KOps/s | $\color{#35bf28}+0.36\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7024ms | 0.5531ms | 1.8080 KOps/s | 1.8057 KOps/s | $\color{#35bf28}+0.13\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 4.8320ms | 4.6461ms | 215.2326 Ops/s | 213.0300 Ops/s | $\color{#35bf28}+1.03\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.9935ms | 0.7022ms | 1.4242 KOps/s | 1.4126 KOps/s | $\color{#35bf28}+0.82\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8081ms | 0.6775ms | 1.4760 KOps/s | 1.4478 KOps/s | $\color{#35bf28}+1.95\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1154s | 9.0686ms | 110.2711 Ops/s | 109.4822 Ops/s | $\color{#35bf28}+0.72\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 17.5301ms | 15.2215ms | 65.6964 Ops/s | 64.8930 Ops/s | $\color{#35bf28}+1.24\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.4193ms | 1.3231ms | 755.7738 Ops/s | 763.1863 Ops/s | $\color{#d91a1a}-0.97\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1068s | 8.8530ms | 112.9562 Ops/s | 145.6697 Ops/s | $\textbf{\color{#d91a1a}-22.46\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 17.5052ms | 15.1135ms | 66.1660 Ops/s | 57.2143 Ops/s | $\textbf{\color{#35bf28}+15.65\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.3826ms | 1.2929ms | 773.4742 Ops/s | 740.9853 Ops/s | $\color{#35bf28}+4.38\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1053s | 7.1683ms | 139.5032 Ops/s | 138.8996 Ops/s | $\color{#35bf28}+0.43\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.3131ms | 15.5614ms | 64.2614 Ops/s | 63.8297 Ops/s | $\color{#35bf28}+0.68\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.8251ms | 1.6519ms | 605.3498 Ops/s | 595.3302 Ops/s | $\color{#35bf28}+1.68\\%$ |
TheRisenPhoenix commented 4 months ago

Awesome! 😀