search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.01k stars 269 forks source link

[BugFix] Fix collectors with non tensors #2232

Closed vmoens closed 3 weeks ago

vmoens commented 3 weeks ago

Based on https://github.com/pytorch/tensordict/pull/817

Code to test:

from torchrl.envs import GymEnv, Transform, check_env_specs
from torchrl.data import NonTensorSpec, TensorSpec
from torchrl.collectors import SyncDataCollector
from tensordict import TensorDictBase, TensorDict, NonTensorData

class AddNontTensorData(Transform):
    def _call(self, tensordict: TensorDictBase) -> TensorDictBase:
        tensordict["nt"] = "a string!"
        return tensordict

    def _reset(
        self, tensordict: TensorDictBase, tensordict_reset: TensorDictBase
    ) -> TensorDictBase:
        return tensordict_reset.set("nt", NonTensorData("reset!"))

    def transform_observation_spec(self, observation_spec: TensorSpec) -> TensorSpec:
        observation_spec["nt"] = NonTensorSpec(shape=())
        return observation_spec

env = GymEnv("CartPole-v1").append_transform(AddNontTensorData())
print(env.rollout(3))
check_env_specs(env)

collector = SyncDataCollector(env, frames_per_batch=10)
for data in collector:
    print(data)
    break
pytorch-bot[bot] commented 3 weeks ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2232

Note: Links to docs will display an error until the docs builds have been completed.

:x: 2 New Failures, 13 Unrelated Failures

As of commit 93ee8947dc71bcf41b53a5e8db23fe5d032a577c with merge base 45ab9de967e3633f73f6905a869c1c4fa78e7457 (image):

NEW FAILURES - The following jobs have failed:

* [Generate documentation / build-docs (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2232#26429592579) ([gh](https://github.com/pytorch/rl/actions/runs/9584908455/job/26429592579)) * [Unit-tests on Windows / unittests-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/rl/2232#26429591702) ([gh](https://github.com/pytorch/rl/actions/runs/9584908453/job/26429591702)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

* [Examples Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2232#26429591929) ([gh](https://github.com/pytorch/rl/actions/runs/9584908457/job/26429591929)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Habitat Tests on Linux / tests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2232#26429592039) ([gh](https://github.com/pytorch/rl/actions/runs/9584908454/job/26429592039)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2232#26429619497) ([gh](https://github.com/pytorch/rl/actions/runs/9584908449/job/26429619497)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2232#26429619785) ([gh](https://github.com/pytorch/rl/actions/runs/9584908449/job/26429619785)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2232#26429592351) ([gh](https://github.com/pytorch/rl/actions/runs/9584908448/job/26429592351)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.10) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2232#26429612687) ([gh](https://github.com/pytorch/rl/actions/runs/9584908442/job/26429612687)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.11) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2232#26429613298) ([gh](https://github.com/pytorch/rl/actions/runs/9584908442/job/26429613298)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.8) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2232#26429613760) ([gh](https://github.com/pytorch/rl/actions/runs/9584908442/job/26429613760)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-cpu (3.9) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2232#26429614094) ([gh](https://github.com/pytorch/rl/actions/runs/9584908442/job/26429614094)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-gpu (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2232#26429614818) ([gh](https://github.com/pytorch/rl/actions/runs/9584908442/job/26429614818)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2232#26429615197) ([gh](https://github.com/pytorch/rl/actions/runs/9584908442/job/26429615197)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2232#26429615494) ([gh](https://github.com/pytorch/rl/actions/runs/9584908442/job/26429615494)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job](https://hud.pytorch.org/pr/pytorch/rl/2232#26429615950) ([gh](https://github.com/pytorch/rl/actions/runs/9584908442/job/26429615950)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions[bot] commented 3 weeks ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}16$. Worsened: $\large\color{#d91a1a}1$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_single | 0.1197s | 59.9147ms | 16.6904 Ops/s | 17.1821 Ops/s | $\color{#d91a1a}-2.86\\%$ | | test_sync | 34.6752ms | 31.2297ms | 32.0208 Ops/s | 29.2211 Ops/s | $\textbf{\color{#35bf28}+9.58\\%}$ | | test_async | 57.6006ms | 29.4117ms | 34.0001 Ops/s | 33.8806 Ops/s | $\color{#35bf28}+0.35\\%$ | | test_simple | 0.3812s | 0.3805s | 2.6279 Ops/s | 2.5637 Ops/s | $\color{#35bf28}+2.50\\%$ | | test_transformed | 0.5301s | 0.5284s | 1.8924 Ops/s | 1.8389 Ops/s | $\color{#35bf28}+2.91\\%$ | | test_serial | 1.3199s | 1.2620s | 0.7924 Ops/s | 0.7820 Ops/s | $\color{#35bf28}+1.33\\%$ | | test_parallel | 1.1038s | 1.0587s | 0.9446 Ops/s | 0.9023 Ops/s | $\color{#35bf28}+4.68\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.1810ms | 21.2144μs | 47.1378 KOps/s | 46.3472 KOps/s | $\color{#35bf28}+1.71\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 44.9820μs | 12.8937μs | 77.5576 KOps/s | 74.9329 KOps/s | $\color{#35bf28}+3.50\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 31.7300μs | 12.5404μs | 79.7422 KOps/s | 77.8596 KOps/s | $\color{#35bf28}+2.42\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 26.9310μs | 7.5838μs | 131.8596 KOps/s | 127.6285 KOps/s | $\color{#35bf28}+3.32\\%$ | | test_step_mdp_speed[True-True-False-True-True] | 49.9840μs | 22.4324μs | 44.5783 KOps/s | 42.8819 KOps/s | $\color{#35bf28}+3.96\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 34.9960μs | 14.2282μs | 70.2830 KOps/s | 69.0741 KOps/s | $\color{#35bf28}+1.75\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 45.6360μs | 13.7840μs | 72.5478 KOps/s | 71.1295 KOps/s | $\color{#35bf28}+1.99\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 61.8370μs | 8.8576μs | 112.8972 KOps/s | 111.1495 KOps/s | $\color{#35bf28}+1.57\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 0.1485ms | 24.2527μs | 41.2324 KOps/s | 40.8254 KOps/s | $\color{#35bf28}+1.00\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 39.6050μs | 15.4364μs | 64.7818 KOps/s | 63.2101 KOps/s | $\color{#35bf28}+2.49\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 64.9420μs | 13.6863μs | 73.0657 KOps/s | 71.7598 KOps/s | $\color{#35bf28}+1.82\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 29.5060μs | 8.7831μs | 113.8551 KOps/s | 110.4712 KOps/s | $\color{#35bf28}+3.06\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 51.2060μs | 24.8032μs | 40.3173 KOps/s | 39.1515 KOps/s | $\color{#35bf28}+2.98\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 42.3800μs | 16.5716μs | 60.3444 KOps/s | 58.8433 KOps/s | $\color{#35bf28}+2.55\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 49.9240μs | 14.7409μs | 67.8383 KOps/s | 65.6330 KOps/s | $\color{#35bf28}+3.36\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 29.0140μs | 9.8601μs | 101.4190 KOps/s | 97.5129 KOps/s | $\color{#35bf28}+4.01\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 48.5720μs | 23.6828μs | 42.2247 KOps/s | 40.8876 KOps/s | $\color{#35bf28}+3.27\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 51.1460μs | 15.3954μs | 64.9547 KOps/s | 63.1917 KOps/s | $\color{#35bf28}+2.79\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 42.6200μs | 15.8565μs | 63.0657 KOps/s | 60.6879 KOps/s | $\color{#35bf28}+3.92\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 42.7110μs | 9.8777μs | 101.2386 KOps/s | 96.7759 KOps/s | $\color{#35bf28}+4.61\\%$ | | test_step_mdp_speed[False-True-False-True-True] | 63.2090μs | 24.7560μs | 40.3942 KOps/s | 38.9342 KOps/s | $\color{#35bf28}+3.75\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 48.7920μs | 16.4478μs | 60.7983 KOps/s | 58.3334 KOps/s | $\color{#35bf28}+4.23\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 42.5300μs | 16.9282μs | 59.0729 KOps/s | 56.5972 KOps/s | $\color{#35bf28}+4.37\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 35.4770μs | 11.1115μs | 89.9965 KOps/s | 86.7695 KOps/s | $\color{#35bf28}+3.72\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 61.1950μs | 25.9229μs | 38.5759 KOps/s | 36.4807 KOps/s | $\textbf{\color{#35bf28}+5.74\\%}$ | | test_step_mdp_speed[False-False-True-True-False] | 42.9410μs | 17.7467μs | 56.3486 KOps/s | 53.5613 KOps/s | $\textbf{\color{#35bf28}+5.20\\%}$ | | test_step_mdp_speed[False-False-True-False-True] | 41.2270μs | 16.8635μs | 59.2997 KOps/s | 55.7363 KOps/s | $\textbf{\color{#35bf28}+6.39\\%}$ | | test_step_mdp_speed[False-False-True-False-False] | 40.2250μs | 11.1137μs | 89.9793 KOps/s | 85.6723 KOps/s | $\textbf{\color{#35bf28}+5.03\\%}$ | | test_step_mdp_speed[False-False-False-True-True] | 40.8770μs | 27.6879μs | 36.1169 KOps/s | 34.4935 KOps/s | $\color{#35bf28}+4.71\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 48.7210μs | 18.8989μs | 52.9132 KOps/s | 50.5287 KOps/s | $\color{#35bf28}+4.72\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 69.1910μs | 18.0220μs | 55.4876 KOps/s | 52.9456 KOps/s | $\color{#35bf28}+4.80\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 45.7870μs | 12.2768μs | 81.4542 KOps/s | 78.5968 KOps/s | $\color{#35bf28}+3.64\\%$ | | test_values[generalized_advantage_estimate-True-True] | 9.6236ms | 9.3866ms | 106.5347 Ops/s | 104.6781 Ops/s | $\color{#35bf28}+1.77\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 36.4431ms | 35.2056ms | 28.4046 Ops/s | 28.3631 Ops/s | $\color{#35bf28}+0.15\\%$ | | test_values[td0_return_estimate-False-False] | 0.1990ms | 0.1777ms | 5.6268 KOps/s | 5.6890 KOps/s | $\color{#d91a1a}-1.09\\%$ | | test_values[td1_return_estimate-False-False] | 24.3021ms | 23.8104ms | 41.9985 Ops/s | 39.4520 Ops/s | $\textbf{\color{#35bf28}+6.45\\%}$ | | test_values[vec_td1_return_estimate-False-False] | 48.0842ms | 35.5334ms | 28.1425 Ops/s | 28.0282 Ops/s | $\color{#35bf28}+0.41\\%$ | | test_values[td_lambda_return_estimate-True-False] | 37.4899ms | 34.5470ms | 28.9461 Ops/s | 28.1503 Ops/s | $\color{#35bf28}+2.83\\%$ | | test_values[vec_td_lambda_return_estimate-True-False] | 36.9595ms | 35.4010ms | 28.2478 Ops/s | 28.0848 Ops/s | $\color{#35bf28}+0.58\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 11.0091ms | 8.2845ms | 120.7075 Ops/s | 116.6147 Ops/s | $\color{#35bf28}+3.51\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.0918ms | 1.8866ms | 530.0530 Ops/s | 504.3696 Ops/s | $\textbf{\color{#35bf28}+5.09\\%}$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.6381ms | 0.3597ms | 2.7799 KOps/s | 2.7587 KOps/s | $\color{#35bf28}+0.77\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 46.6221ms | 44.5474ms | 22.4480 Ops/s | 22.7859 Ops/s | $\color{#d91a1a}-1.48\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.6909ms | 3.0428ms | 328.6477 Ops/s | 326.3876 Ops/s | $\color{#35bf28}+0.69\\%$ | | test_dqn_speed | 6.1135ms | 1.3722ms | 728.7340 Ops/s | 738.8447 Ops/s | $\color{#d91a1a}-1.37\\%$ | | test_ddpg_speed | 4.3684ms | 2.9096ms | 343.6876 Ops/s | 349.3017 Ops/s | $\color{#d91a1a}-1.61\\%$ | | test_sac_speed | 9.0863ms | 8.5108ms | 117.4971 Ops/s | 116.9543 Ops/s | $\color{#35bf28}+0.46\\%$ | | test_redq_speed | 17.5801ms | 13.2719ms | 75.3471 Ops/s | 74.0277 Ops/s | $\color{#35bf28}+1.78\\%$ | | test_redq_deprec_speed | 93.0242ms | 14.5549ms | 68.7056 Ops/s | 71.6346 Ops/s | $\color{#d91a1a}-4.09\\%$ | | test_td3_speed | 8.6189ms | 8.4218ms | 118.7388 Ops/s | 117.4807 Ops/s | $\color{#35bf28}+1.07\\%$ | | test_cql_speed | 38.0164ms | 36.6791ms | 27.2635 Ops/s | 26.9322 Ops/s | $\color{#35bf28}+1.23\\%$ | | test_a2c_speed | 8.7068ms | 7.4825ms | 133.6452 Ops/s | 130.7672 Ops/s | $\color{#35bf28}+2.20\\%$ | | test_ppo_speed | 9.1705ms | 7.7287ms | 129.3881 Ops/s | 125.5260 Ops/s | $\color{#35bf28}+3.08\\%$ | | test_reinforce_speed | 7.4473ms | 6.6457ms | 150.4723 Ops/s | 148.5387 Ops/s | $\color{#35bf28}+1.30\\%$ | | test_iql_speed | 38.7926ms | 33.3692ms | 29.9678 Ops/s | 30.1265 Ops/s | $\color{#d91a1a}-0.53\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.6952ms | 3.4633ms | 288.7427 Ops/s | 273.9643 Ops/s | $\textbf{\color{#35bf28}+5.39\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6032ms | 0.4969ms | 2.0126 KOps/s | 1.9668 KOps/s | $\color{#35bf28}+2.33\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7711ms | 0.4770ms | 2.0965 KOps/s | 1.8237 KOps/s | $\textbf{\color{#35bf28}+14.95\\%}$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.8531ms | 3.4175ms | 292.6157 Ops/s | 276.8254 Ops/s | $\textbf{\color{#35bf28}+5.70\\%}$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6450ms | 0.4903ms | 2.0396 KOps/s | 1.9805 KOps/s | $\color{#35bf28}+2.98\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 3.4137ms | 0.4682ms | 2.1361 KOps/s | 2.0586 KOps/s | $\color{#35bf28}+3.76\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.4323ms | 1.7342ms | 576.6411 Ops/s | 573.0608 Ops/s | $\color{#35bf28}+0.62\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 2.2514ms | 1.6417ms | 609.1365 Ops/s | 599.8438 Ops/s | $\color{#35bf28}+1.55\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.8969ms | 3.6126ms | 276.8126 Ops/s | 255.8624 Ops/s | $\textbf{\color{#35bf28}+8.19\\%}$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.0091ms | 0.6158ms | 1.6240 KOps/s | 1.5507 KOps/s | $\color{#35bf28}+4.73\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8839ms | 0.5854ms | 1.7081 KOps/s | 1.6291 KOps/s | $\color{#35bf28}+4.85\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.6931ms | 3.4329ms | 291.2979 Ops/s | 267.2862 Ops/s | $\textbf{\color{#35bf28}+8.98\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.9540ms | 0.4984ms | 2.0065 KOps/s | 1.8996 KOps/s | $\textbf{\color{#35bf28}+5.63\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.6858ms | 0.4760ms | 2.1008 KOps/s | 2.0049 KOps/s | $\color{#35bf28}+4.78\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.4014ms | 3.4439ms | 290.3690 Ops/s | 265.3226 Ops/s | $\textbf{\color{#35bf28}+9.44\\%}$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.5845ms | 0.4922ms | 2.0318 KOps/s | 1.9445 KOps/s | $\color{#35bf28}+4.49\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 3.5387ms | 0.4710ms | 2.1230 KOps/s | 2.0410 KOps/s | $\color{#35bf28}+4.01\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.1645ms | 3.5857ms | 278.8871 Ops/s | 261.1065 Ops/s | $\textbf{\color{#35bf28}+6.81\\%}$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.1401ms | 0.6189ms | 1.6159 KOps/s | 1.5874 KOps/s | $\color{#35bf28}+1.79\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7198ms | 0.5872ms | 1.7030 KOps/s | 1.6591 KOps/s | $\color{#35bf28}+2.64\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1095s | 7.8667ms | 127.1180 Ops/s | 117.7979 Ops/s | $\textbf{\color{#35bf28}+7.91\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 14.3807ms | 12.2965ms | 81.3240 Ops/s | 80.1550 Ops/s | $\color{#35bf28}+1.46\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 1.1731ms | 1.0345ms | 966.6604 Ops/s | 948.9799 Ops/s | $\color{#35bf28}+1.86\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1024s | 5.6818ms | 176.0004 Ops/s | 168.1489 Ops/s | $\color{#35bf28}+4.67\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 14.7626ms | 12.3160ms | 81.1952 Ops/s | 79.7102 Ops/s | $\color{#35bf28}+1.86\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 4.3777ms | 1.0919ms | 915.7964 Ops/s | 949.0716 Ops/s | $\color{#d91a1a}-3.51\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1093s | 5.9348ms | 168.4989 Ops/s | 166.5289 Ops/s | $\color{#35bf28}+1.18\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 14.9479ms | 12.4555ms | 80.2859 Ops/s | 78.9406 Ops/s | $\color{#35bf28}+1.70\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 0.1054s | 3.2904ms | 303.9104 Ops/s | 773.7237 Ops/s | $\textbf{\color{#d91a1a}-60.72\\%}$ |
github-actions[bot] commented 3 weeks ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ----------------------------------------------------------------------------------------- | --------- | --------- | -------------- | ------------------ | ----------------------------------- | | test_single | 0.1218s | 0.1187s | 8.4229 Ops/s | 8.3900 Ops/s | $\color{#35bf28}+0.39\\%$ | | test_sync | 0.1050s | 0.1044s | 9.5758 Ops/s | 9.5697 Ops/s | $\color{#35bf28}+0.06\\%$ | | test_async | 0.2001s | 78.8886ms | 12.6761 Ops/s | 10.5391 Ops/s | $\textbf{\color{#35bf28}+20.28\\%}$ | | test_single_pixels | 0.1290s | 0.1288s | 7.7614 Ops/s | 7.6403 Ops/s | $\color{#35bf28}+1.58\\%$ | | test_sync_pixels | 85.2986ms | 82.3905ms | 12.1373 Ops/s | 12.0448 Ops/s | $\color{#35bf28}+0.77\\%$ | | test_async_pixels | 0.1616s | 70.4018ms | 14.2042 Ops/s | 14.8632 Ops/s | $\color{#d91a1a}-4.43\\%$ | | test_simple | 0.8346s | 0.8230s | 1.2151 Ops/s | 1.1900 Ops/s | $\color{#35bf28}+2.11\\%$ | | test_transformed | 1.1072s | 1.1044s | 0.9055 Ops/s | 0.9243 Ops/s | $\color{#d91a1a}-2.03\\%$ | | test_serial | 2.6652s | 2.5900s | 0.3861 Ops/s | 0.3964 Ops/s | $\color{#d91a1a}-2.60\\%$ | | test_parallel | 2.4009s | 2.3516s | 0.4252 Ops/s | 0.4236 Ops/s | $\color{#35bf28}+0.39\\%$ | | test_step_mdp_speed[True-True-True-True-True] | 0.3381ms | 34.2032μs | 29.2370 KOps/s | 28.8159 KOps/s | $\color{#35bf28}+1.46\\%$ | | test_step_mdp_speed[True-True-True-True-False] | 41.8500μs | 20.4286μs | 48.9510 KOps/s | 46.7979 KOps/s | $\color{#35bf28}+4.60\\%$ | | test_step_mdp_speed[True-True-True-False-True] | 0.2187ms | 19.5117μs | 51.2513 KOps/s | 48.9775 KOps/s | $\color{#35bf28}+4.64\\%$ | | test_step_mdp_speed[True-True-True-False-False] | 35.0110μs | 11.5939μs | 86.2525 KOps/s | 81.1660 KOps/s | $\textbf{\color{#35bf28}+6.27\\%}$ | | test_step_mdp_speed[True-True-False-True-True] | 0.2296ms | 36.2653μs | 27.5745 KOps/s | 27.5980 KOps/s | $\color{#d91a1a}-0.08\\%$ | | test_step_mdp_speed[True-True-False-True-False] | 0.2342ms | 22.4250μs | 44.5931 KOps/s | 44.0284 KOps/s | $\color{#35bf28}+1.28\\%$ | | test_step_mdp_speed[True-True-False-False-True] | 0.2423ms | 21.5155μs | 46.4781 KOps/s | 45.4706 KOps/s | $\color{#35bf28}+2.22\\%$ | | test_step_mdp_speed[True-True-False-False-False] | 32.7200μs | 13.4783μs | 74.1935 KOps/s | 70.9571 KOps/s | $\color{#35bf28}+4.56\\%$ | | test_step_mdp_speed[True-False-True-True-True] | 0.2298ms | 37.9015μs | 26.3842 KOps/s | 25.6785 KOps/s | $\color{#35bf28}+2.75\\%$ | | test_step_mdp_speed[True-False-True-True-False] | 0.2236ms | 24.5146μs | 40.7920 KOps/s | 40.7158 KOps/s | $\color{#35bf28}+0.19\\%$ | | test_step_mdp_speed[True-False-True-False-True] | 50.5010μs | 21.4028μs | 46.7228 KOps/s | 45.5478 KOps/s | $\color{#35bf28}+2.58\\%$ | | test_step_mdp_speed[True-False-True-False-False] | 0.2029ms | 13.4874μs | 74.1431 KOps/s | 71.2936 KOps/s | $\color{#35bf28}+4.00\\%$ | | test_step_mdp_speed[True-False-False-True-True] | 0.2279ms | 39.7711μs | 25.1439 KOps/s | 25.0726 KOps/s | $\color{#35bf28}+0.28\\%$ | | test_step_mdp_speed[True-False-False-True-False] | 0.2241ms | 26.1144μs | 38.2931 KOps/s | 38.2618 KOps/s | $\color{#35bf28}+0.08\\%$ | | test_step_mdp_speed[True-False-False-False-True] | 44.3200μs | 23.0414μs | 43.4002 KOps/s | 41.9007 KOps/s | $\color{#35bf28}+3.58\\%$ | | test_step_mdp_speed[True-False-False-False-False] | 0.2183ms | 15.4078μs | 64.9022 KOps/s | 63.0852 KOps/s | $\color{#35bf28}+2.88\\%$ | | test_step_mdp_speed[False-True-True-True-True] | 0.2392ms | 37.7625μs | 26.4813 KOps/s | 26.2525 KOps/s | $\color{#35bf28}+0.87\\%$ | | test_step_mdp_speed[False-True-True-True-False] | 0.2138ms | 23.9013μs | 41.8387 KOps/s | 40.5504 KOps/s | $\color{#35bf28}+3.18\\%$ | | test_step_mdp_speed[False-True-True-False-True] | 45.3610μs | 25.3603μs | 39.4318 KOps/s | 38.3816 KOps/s | $\color{#35bf28}+2.74\\%$ | | test_step_mdp_speed[False-True-True-False-False] | 0.3130ms | 15.4756μs | 64.6179 KOps/s | 61.2072 KOps/s | $\textbf{\color{#35bf28}+5.57\\%}$ | | test_step_mdp_speed[False-True-False-True-True] | 0.2275ms | 39.7371μs | 25.1654 KOps/s | 25.4413 KOps/s | $\color{#d91a1a}-1.08\\%$ | | test_step_mdp_speed[False-True-False-True-False] | 0.2116ms | 25.8948μs | 38.6178 KOps/s | 38.1593 KOps/s | $\color{#35bf28}+1.20\\%$ | | test_step_mdp_speed[False-True-False-False-True] | 47.5510μs | 27.0290μs | 36.9973 KOps/s | 35.7006 KOps/s | $\color{#35bf28}+3.63\\%$ | | test_step_mdp_speed[False-True-False-False-False] | 0.2324ms | 17.1949μs | 58.1569 KOps/s | 56.8976 KOps/s | $\color{#35bf28}+2.21\\%$ | | test_step_mdp_speed[False-False-True-True-True] | 0.2285ms | 41.3631μs | 24.1761 KOps/s | 24.0361 KOps/s | $\color{#35bf28}+0.58\\%$ | | test_step_mdp_speed[False-False-True-True-False] | 0.2515ms | 28.3735μs | 35.2441 KOps/s | 35.1503 KOps/s | $\color{#35bf28}+0.27\\%$ | | test_step_mdp_speed[False-False-True-False-True] | 56.3700μs | 27.2492μs | 36.6983 KOps/s | 35.8404 KOps/s | $\color{#35bf28}+2.39\\%$ | | test_step_mdp_speed[False-False-True-False-False] | 0.2040ms | 17.2944μs | 57.8222 KOps/s | 56.5468 KOps/s | $\color{#35bf28}+2.26\\%$ | | test_step_mdp_speed[False-False-False-True-True] | 66.0910μs | 44.1486μs | 22.6508 KOps/s | 23.1760 KOps/s | $\color{#d91a1a}-2.27\\%$ | | test_step_mdp_speed[False-False-False-True-False] | 0.2356ms | 30.0558μs | 33.2715 KOps/s | 32.8983 KOps/s | $\color{#35bf28}+1.13\\%$ | | test_step_mdp_speed[False-False-False-False-True] | 0.2219ms | 28.3912μs | 35.2222 KOps/s | 33.9835 KOps/s | $\color{#35bf28}+3.65\\%$ | | test_step_mdp_speed[False-False-False-False-False] | 37.8610μs | 18.8918μs | 52.9329 KOps/s | 51.5065 KOps/s | $\color{#35bf28}+2.77\\%$ | | test_values[generalized_advantage_estimate-True-True] | 26.7202ms | 25.6999ms | 38.9107 Ops/s | 39.8816 Ops/s | $\color{#d91a1a}-2.43\\%$ | | test_values[vec_generalized_advantage_estimate-True-True] | 88.3654ms | 2.6700ms | 374.5362 Ops/s | 371.3613 Ops/s | $\color{#35bf28}+0.85\\%$ | | test_values[td0_return_estimate-False-False] | 94.2210μs | 68.3830μs | 14.6235 KOps/s | 14.9480 KOps/s | $\color{#d91a1a}-2.17\\%$ | | test_values[td1_return_estimate-False-False] | 59.5483ms | 57.6678ms | 17.3407 Ops/s | 17.8954 Ops/s | $\color{#d91a1a}-3.10\\%$ | | test_values[vec_td1_return_estimate-False-False] | 1.2942ms | 1.0844ms | 922.1290 Ops/s | 916.4126 Ops/s | $\color{#35bf28}+0.62\\%$ | | test_values[td_lambda_return_estimate-True-False] | 94.2791ms | 93.4101ms | 10.7055 Ops/s | 11.2988 Ops/s | $\textbf{\color{#d91a1a}-5.25\\%}$ | | test_values[vec_td_lambda_return_estimate-True-False] | 1.2654ms | 1.0827ms | 923.6478 Ops/s | 921.6671 Ops/s | $\color{#35bf28}+0.21\\%$ | | test_gae_speed[generalized_advantage_estimate-False-1-512] | 26.3334ms | 25.4097ms | 39.3551 Ops/s | 40.6598 Ops/s | $\color{#d91a1a}-3.21\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.9769ms | 0.7308ms | 1.3684 KOps/s | 1.3725 KOps/s | $\color{#d91a1a}-0.30\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7790ms | 0.6975ms | 1.4337 KOps/s | 1.4784 KOps/s | $\color{#d91a1a}-3.03\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.6444ms | 1.4806ms | 675.3799 Ops/s | 676.6091 Ops/s | $\color{#d91a1a}-0.18\\%$ | | test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.7624ms | 0.7058ms | 1.4169 KOps/s | 1.4547 KOps/s | $\color{#d91a1a}-2.60\\%$ | | test_dqn_speed | 79.0521ms | 1.6746ms | 597.1682 Ops/s | 674.7902 Ops/s | $\textbf{\color{#d91a1a}-11.50\\%}$ | | test_ddpg_speed | 3.2739ms | 3.0156ms | 331.6055 Ops/s | 330.1403 Ops/s | $\color{#35bf28}+0.44\\%$ | | test_sac_speed | 9.3777ms | 8.6241ms | 115.9546 Ops/s | 117.2239 Ops/s | $\color{#d91a1a}-1.08\\%$ | | test_redq_speed | 11.5650ms | 10.7312ms | 93.1862 Ops/s | 95.0754 Ops/s | $\color{#d91a1a}-1.99\\%$ | | test_redq_deprec_speed | 12.4624ms | 11.8675ms | 84.2641 Ops/s | 86.5079 Ops/s | $\color{#d91a1a}-2.59\\%$ | | test_td3_speed | 17.8538ms | 8.5927ms | 116.3772 Ops/s | 116.6601 Ops/s | $\color{#d91a1a}-0.24\\%$ | | test_cql_speed | 27.7132ms | 26.0367ms | 38.4073 Ops/s | 38.0314 Ops/s | $\color{#35bf28}+0.99\\%$ | | test_a2c_speed | 5.8486ms | 5.6421ms | 177.2395 Ops/s | 172.6937 Ops/s | $\color{#35bf28}+2.63\\%$ | | test_ppo_speed | 6.5886ms | 5.9839ms | 167.1144 Ops/s | 164.7370 Ops/s | $\color{#35bf28}+1.44\\%$ | | test_reinforce_speed | 4.9831ms | 4.6301ms | 215.9785 Ops/s | 211.5937 Ops/s | $\color{#35bf28}+2.07\\%$ | | test_iql_speed | 20.0129ms | 19.4027ms | 51.5391 Ops/s | 50.6160 Ops/s | $\color{#35bf28}+1.82\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 5.0283ms | 4.7189ms | 211.9122 Ops/s | 213.4861 Ops/s | $\color{#d91a1a}-0.74\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7989ms | 0.5920ms | 1.6891 KOps/s | 1.6912 KOps/s | $\color{#d91a1a}-0.12\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 4.6153ms | 0.5702ms | 1.7537 KOps/s | 1.7548 KOps/s | $\color{#d91a1a}-0.06\\%$ | | test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 5.0073ms | 4.6962ms | 212.9377 Ops/s | 214.8037 Ops/s | $\color{#d91a1a}-0.87\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8943ms | 0.5857ms | 1.7074 KOps/s | 1.6971 KOps/s | $\color{#35bf28}+0.61\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 4.5820ms | 0.5676ms | 1.7617 KOps/s | 1.7755 KOps/s | $\color{#d91a1a}-0.78\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.3050ms | 2.1149ms | 472.8274 Ops/s | 468.8320 Ops/s | $\color{#35bf28}+0.85\\%$ | | test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 5.9784ms | 2.0157ms | 496.1172 Ops/s | 495.1814 Ops/s | $\color{#35bf28}+0.19\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0458ms | 4.8538ms | 206.0262 Ops/s | 209.2571 Ops/s | $\color{#d91a1a}-1.54\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.5480ms | 0.7182ms | 1.3923 KOps/s | 1.3913 KOps/s | $\color{#35bf28}+0.08\\%$ | | test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8073ms | 0.6918ms | 1.4455 KOps/s | 1.4316 KOps/s | $\color{#35bf28}+0.97\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 4.9238ms | 4.7146ms | 212.1083 Ops/s | 213.8317 Ops/s | $\color{#d91a1a}-0.81\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.1899ms | 0.5931ms | 1.6861 KOps/s | 1.6846 KOps/s | $\color{#35bf28}+0.09\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.8781ms | 0.5710ms | 1.7513 KOps/s | 1.7383 KOps/s | $\color{#35bf28}+0.75\\%$ | | test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 4.8761ms | 4.6920ms | 213.1290 Ops/s | 214.2363 Ops/s | $\color{#d91a1a}-0.52\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.8632ms | 0.5864ms | 1.7054 KOps/s | 1.6970 KOps/s | $\color{#35bf28}+0.49\\%$ | | test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.6808ms | 0.5620ms | 1.7792 KOps/s | 1.7787 KOps/s | $\color{#35bf28}+0.03\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 5.0576ms | 4.8650ms | 205.5513 Ops/s | 208.5507 Ops/s | $\color{#d91a1a}-1.44\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.3888ms | 0.7175ms | 1.3937 KOps/s | 1.3849 KOps/s | $\color{#35bf28}+0.64\\%$ | | test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8725ms | 0.6941ms | 1.4407 KOps/s | 1.4383 KOps/s | $\color{#35bf28}+0.16\\%$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1314s | 7.5391ms | 132.6419 Ops/s | 130.1348 Ops/s | $\color{#35bf28}+1.93\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 18.4571ms | 16.1059ms | 62.0891 Ops/s | 60.7139 Ops/s | $\color{#35bf28}+2.27\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 7.3767ms | 1.4805ms | 675.4520 Ops/s | 782.3438 Ops/s | $\textbf{\color{#d91a1a}-13.66\\%}$ | | test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1244s | 7.4180ms | 134.8070 Ops/s | 133.9132 Ops/s | $\color{#35bf28}+0.67\\%$ | | test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 18.6336ms | 15.9724ms | 62.6081 Ops/s | 53.1020 Ops/s | $\textbf{\color{#35bf28}+17.90\\%}$ | | test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 6.7735ms | 1.4277ms | 700.4256 Ops/s | 779.7386 Ops/s | $\textbf{\color{#d91a1a}-10.17\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1265s | 10.0361ms | 99.6402 Ops/s | 131.0914 Ops/s | $\textbf{\color{#d91a1a}-23.99\\%}$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.7535ms | 16.3139ms | 61.2975 Ops/s | 60.0942 Ops/s | $\color{#35bf28}+2.00\\%$ | | test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.5080ms | 1.4916ms | 670.4144 Ops/s | 637.2536 Ops/s | $\textbf{\color{#35bf28}+5.20\\%}$ |