search

pytorch / tensordict

TensorDict is a pytorch dedicated tensor container.
MIT License
832 stars 74 forks source link

[Feature] Multithreaded apply #844

Closed vmoens closed 4 months ago

vmoens commented 4 months ago

multithreaded apply should only be used internally as there is a non-negligeable set of operations that can lead to deadlocks. As of now, it is not made to speed up the tensordict operations (reduce TD overhead) but speed up the execution of any function passed to apply that can benefit from being executed asynchronously over the leaves.

Some benchmarks:

import torch
from tensordict import TensorDict
from torch.utils.benchmark import Timer
# torch.set_num_threads(16)
import functools

d = {}
sub = d
for _ in range(20):
    for i in range(50):
        sub[str(i)] = torch.rand(2048, 2048)
    sub["nested"] = {}
    sub = sub["nested"]
td = TensorDict(d, batch_size=[])

assert (td._multithread_apply_nest(lambda x: x, num_threads=16) == td).all()

func = lambda x: x.pin_memory()
print(Timer("td._fast_apply(func)", globals=globals()).adaptive_autorange())
print(Timer("td._multithread_apply_nest(func, checked=True, num_threads=4)", globals=globals()).adaptive_autorange())
print(Timer("td._multithread_apply_nest(func, checked=True, num_threads=8)", globals=globals()).adaptive_autorange())
print(Timer("td._multithread_apply_nest(func, checked=True, num_threads=16)", globals=globals()).adaptive_autorange())

Results:

td._fast_apply(func)
  Median: 1.74 s
  IQR:    0.02 s (1.73 to 1.75)
  4 measurements, 1 runs per measurement, 1 thread
<torch.utils.benchmark.utils.common.Measurement object at 0x7fc456a92530>
td._multithread_apply_nest(func, checked=True, num_threads=4)
  Median: 498.21 ms
  IQR:    80.43 ms (455.67 to 536.10)
  20 measurements, 1 runs per measurement, 1 thread
  WARNING: Interquartile range is 16.1% of the median measurement.
           This could indicate system fluctuation.
<torch.utils.benchmark.utils.common.Measurement object at 0x7fc3537e44c0>
td._multithread_apply_nest(func, checked=True, num_threads=8)
  Median: 303.86 ms
  IQR:    24.10 ms (295.45 to 319.54)
  4 measurements, 1 runs per measurement, 1 thread
<torch.utils.benchmark.utils.common.Measurement object at 0x7fc35377da80>
td._multithread_apply_nest(func, checked=True, num_threads=16)
  Median: 220.13 ms
  IQR:    13.43 ms (214.42 to 227.85)
  4 measurements, 1 runs per measurement, 1 thread
print(Timer("td._fast_apply(func).to('cuda')", globals=globals()).adaptive_autorange())
print(Timer("td._multithread_apply_nest(func, checked=True, num_threads=4).to('cuda')", globals=globals()).adaptive_autorange())
print(Timer("td._multithread_apply_nest(func, checked=True, num_threads=8).to('cuda')", globals=globals()).adaptive_autorange())
print(Timer("td._multithread_apply_nest(func, checked=True, num_threads=16).to('cuda')", globals=globals()).adaptive_autorange())

Results

<torch.utils.benchmark.utils.common.Measurement object at 0x7fc35377fc10>
td._fast_apply(func).to('cuda')
  Median: 1.73 s
  IQR:    0.02 s (1.72 to 1.73)
  4 measurements, 1 runs per measurement, 1 thread
<torch.utils.benchmark.utils.common.Measurement object at 0x7fc3541dbe50>
td._multithread_apply_nest(func, checked=True, num_threads=4).to('cuda')
  Median: 864.55 ms
  IQR:    80.37 ms (817.05 to 897.42)
  9 measurements, 1 runs per measurement, 1 thread
<torch.utils.benchmark.utils.common.Measurement object at 0x7fc35413cc10>
td._multithread_apply_nest(func, checked=True, num_threads=8).to('cuda')
  Median: 680.82 ms
  IQR:    19.13 ms (679.74 to 698.87)
  5 measurements, 1 runs per measurement, 1 thread
<torch.utils.benchmark.utils.common.Measurement object at 0x7fc3541a4400>
td._multithread_apply_nest(func, checked=True, num_threads=16).to('cuda')
  Median: 608.28 ms
  IQR:    12.52 ms (603.43 to 615.95)
  4 measurements, 1 runs per measurement, 1 thread
github-actions[bot] commented 4 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 144. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}14$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | ------------------------------------------ | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_plain_set_nested | 37.0500μs | 17.7191μs | 56.4364 KOps/s | 55.1864 KOps/s | $\color{#35bf28}+2.27\\%$ | | test_plain_set_stack_nested | 36.8390μs | 18.1107μs | 55.2159 KOps/s | 55.3576 KOps/s | $\color{#d91a1a}-0.26\\%$ | | test_plain_set_nested_inplace | 70.1220μs | 20.1915μs | 49.5258 KOps/s | 49.0112 KOps/s | $\color{#35bf28}+1.05\\%$ | | test_plain_set_stack_nested_inplace | 54.4820μs | 20.0514μs | 49.8719 KOps/s | 49.6224 KOps/s | $\color{#35bf28}+0.50\\%$ | | test_items | 40.4160μs | 2.5967μs | 385.1073 KOps/s | 365.6312 KOps/s | $\textbf{\color{#35bf28}+5.33\\%}$ | | test_items_nested | 0.4884ms | 0.2798ms | 3.5744 KOps/s | 3.5935 KOps/s | $\color{#d91a1a}-0.53\\%$ | | test_items_nested_locked | 0.4248ms | 0.2740ms | 3.6491 KOps/s | 3.5977 KOps/s | $\color{#35bf28}+1.43\\%$ | | test_items_nested_leaf | 0.1404ms | 79.2649μs | 12.6159 KOps/s | 12.5759 KOps/s | $\color{#35bf28}+0.32\\%$ | | test_items_stack_nested | 0.4372ms | 0.2783ms | 3.5932 KOps/s | 3.6224 KOps/s | $\color{#d91a1a}-0.81\\%$ | | test_items_stack_nested_leaf | 0.1439ms | 80.5344μs | 12.4171 KOps/s | 12.4462 KOps/s | $\color{#d91a1a}-0.23\\%$ | | test_items_stack_nested_locked | 0.9532ms | 0.2792ms | 3.5815 KOps/s | 3.5932 KOps/s | $\color{#d91a1a}-0.33\\%$ | | test_keys | 40.6650μs | 3.9288μs | 254.5316 KOps/s | 264.6343 KOps/s | $\color{#d91a1a}-3.82\\%$ | | test_keys_nested | 0.2301ms | 0.1379ms | 7.2529 KOps/s | 7.1181 KOps/s | $\color{#35bf28}+1.89\\%$ | | test_keys_nested_locked | 0.7721ms | 0.1437ms | 6.9580 KOps/s | 6.8511 KOps/s | $\color{#35bf28}+1.56\\%$ | | test_keys_nested_leaf | 0.2085ms | 0.1179ms | 8.4794 KOps/s | 8.3531 KOps/s | $\color{#35bf28}+1.51\\%$ | | test_keys_stack_nested | 0.2211ms | 0.1377ms | 7.2645 KOps/s | 7.1051 KOps/s | $\color{#35bf28}+2.24\\%$ | | test_keys_stack_nested_leaf | 0.1988ms | 0.1149ms | 8.7017 KOps/s | 8.3448 KOps/s | $\color{#35bf28}+4.28\\%$ | | test_keys_stack_nested_locked | 0.2411ms | 0.1435ms | 6.9709 KOps/s | 6.9113 KOps/s | $\color{#35bf28}+0.86\\%$ | | test_values | 10.3917μs | 1.1364μs | 879.9969 KOps/s | 889.0502 KOps/s | $\color{#d91a1a}-1.02\\%$ | | test_values_nested | 96.4200μs | 50.6357μs | 19.7489 KOps/s | 19.8973 KOps/s | $\color{#d91a1a}-0.75\\%$ | | test_values_nested_locked | 93.5460μs | 51.0393μs | 19.5927 KOps/s | 19.7279 KOps/s | $\color{#d91a1a}-0.69\\%$ | | test_values_nested_leaf | 0.1885ms | 46.2946μs | 21.6008 KOps/s | 22.0039 KOps/s | $\color{#d91a1a}-1.83\\%$ | | test_values_stack_nested | 0.1136ms | 51.5038μs | 19.4160 KOps/s | 19.6375 KOps/s | $\color{#d91a1a}-1.13\\%$ | | test_values_stack_nested_leaf | 85.8990μs | 46.6786μs | 21.4231 KOps/s | 21.8930 KOps/s | $\color{#d91a1a}-2.15\\%$ | | test_values_stack_nested_locked | 0.1035ms | 51.4880μs | 19.4220 KOps/s | 19.5684 KOps/s | $\color{#d91a1a}-0.75\\%$ | | test_membership | 14.7580μs | 1.3297μs | 752.0369 KOps/s | 751.6651 KOps/s | $\color{#35bf28}+0.05\\%$ | | test_membership_nested | 24.6560μs | 3.3827μs | 295.6261 KOps/s | 289.6401 KOps/s | $\color{#35bf28}+2.07\\%$ | | test_membership_nested_leaf | 26.8500μs | 3.4481μs | 290.0166 KOps/s | 288.6774 KOps/s | $\color{#35bf28}+0.46\\%$ | | test_membership_stacked_nested | 20.4990μs | 3.3967μs | 294.4000 KOps/s | 289.9824 KOps/s | $\color{#35bf28}+1.52\\%$ | | test_membership_stacked_nested_leaf | 42.4200μs | 3.4252μs | 291.9505 KOps/s | 291.7255 KOps/s | $\color{#35bf28}+0.08\\%$ | | test_membership_nested_last | 40.2750μs | 4.1869μs | 238.8423 KOps/s | 237.1389 KOps/s | $\color{#35bf28}+0.72\\%$ | | test_membership_nested_leaf_last | 45.7370μs | 4.2259μs | 236.6347 KOps/s | 233.5101 KOps/s | $\color{#35bf28}+1.34\\%$ | | test_membership_stacked_nested_last | 26.2090μs | 4.2038μs | 237.8817 KOps/s | 207.2660 KOps/s | $\textbf{\color{#35bf28}+14.77\\%}$ | | test_membership_stacked_nested_leaf_last | 24.5360μs | 4.2458μs | 235.5267 KOps/s | 208.6794 KOps/s | $\textbf{\color{#35bf28}+12.87\\%}$ | | test_nested_getleaf | 47.3790μs | 10.7771μs | 92.7896 KOps/s | 92.4838 KOps/s | $\color{#35bf28}+0.33\\%$ | | test_nested_get | 48.2200μs | 10.2472μs | 97.5878 KOps/s | 98.0418 KOps/s | $\color{#d91a1a}-0.46\\%$ | | test_stacked_getleaf | 49.0420μs | 10.6549μs | 93.8539 KOps/s | 92.5514 KOps/s | $\color{#35bf28}+1.41\\%$ | | test_stacked_get | 25.0170μs | 10.1401μs | 98.6184 KOps/s | 99.0015 KOps/s | $\color{#d91a1a}-0.39\\%$ | | test_nested_getitemleaf | 55.5140μs | 11.2879μs | 88.5901 KOps/s | 89.1676 KOps/s | $\color{#d91a1a}-0.65\\%$ | | test_nested_getitem | 47.4190μs | 10.5167μs | 95.0866 KOps/s | 96.3869 KOps/s | $\color{#d91a1a}-1.35\\%$ | | test_stacked_getitemleaf | 33.5720μs | 11.2299μs | 89.0477 KOps/s | 87.5216 KOps/s | $\color{#35bf28}+1.74\\%$ | | test_stacked_getitem | 55.5840μs | 10.4011μs | 96.1435 KOps/s | 96.8597 KOps/s | $\color{#d91a1a}-0.74\\%$ | | test_lock_nested | 0.8328ms | 0.3463ms | 2.8877 KOps/s | 2.9662 KOps/s | $\color{#d91a1a}-2.65\\%$ | | test_lock_stack_nested | 0.4325ms | 0.3180ms | 3.1449 KOps/s | 3.2800 KOps/s | $\color{#d91a1a}-4.12\\%$ | | test_unlock_nested | 0.9168ms | 0.3484ms | 2.8701 KOps/s | 2.9480 KOps/s | $\color{#d91a1a}-2.64\\%$ | | test_unlock_stack_nested | 0.4058ms | 0.3269ms | 3.0593 KOps/s | 3.1971 KOps/s | $\color{#d91a1a}-4.31\\%$ | | test_flatten_speed | 0.4768ms | 98.1872μs | 10.1846 KOps/s | 10.1604 KOps/s | $\color{#35bf28}+0.24\\%$ | | test_unflatten_speed | 0.6345ms | 0.4193ms | 2.3851 KOps/s | 2.4210 KOps/s | $\color{#d91a1a}-1.48\\%$ | | test_common_ops | 3.9949ms | 0.7581ms | 1.3190 KOps/s | 1.3461 KOps/s | $\color{#d91a1a}-2.01\\%$ | | test_creation | 22.1810μs | 2.0870μs | 479.1670 KOps/s | 533.4718 KOps/s | $\textbf{\color{#d91a1a}-10.18\\%}$ | | test_creation_empty | 49.6420μs | 12.3045μs | 81.2714 KOps/s | 83.6296 KOps/s | $\color{#d91a1a}-2.82\\%$ | | test_creation_nested_1 | 58.2990μs | 15.2179μs | 65.7123 KOps/s | 67.3842 KOps/s | $\color{#d91a1a}-2.48\\%$ | | test_creation_nested_2 | 45.3650μs | 19.0593μs | 52.4678 KOps/s | 54.4797 KOps/s | $\color{#d91a1a}-3.69\\%$ | | test_clone | 99.7970μs | 13.3562μs | 74.8714 KOps/s | 77.2574 KOps/s | $\color{#d91a1a}-3.09\\%$ | | test_getitem[int] | 28.1120μs | 11.5905μs | 86.2779 KOps/s | 88.9149 KOps/s | $\color{#d91a1a}-2.97\\%$ | | test_getitem[slice_int] | 70.4320μs | 23.0931μs | 43.3030 KOps/s | 44.7265 KOps/s | $\color{#d91a1a}-3.18\\%$ | | test_getitem[range] | 82.1740μs | 60.5174μs | 16.5242 KOps/s | 16.9277 KOps/s | $\color{#d91a1a}-2.38\\%$ | | test_getitem[tuple] | 49.6030μs | 19.5967μs | 51.0290 KOps/s | 53.2841 KOps/s | $\color{#d91a1a}-4.23\\%$ | | test_getitem[list] | 93.9660μs | 42.2306μs | 23.6795 KOps/s | 24.8169 KOps/s | $\color{#d91a1a}-4.58\\%$ | | test_setitem_dim[int] | 87.0230μs | 36.7014μs | 27.2469 KOps/s | 27.4820 KOps/s | $\color{#d91a1a}-0.86\\%$ | | test_setitem_dim[slice_int] | 0.1452ms | 64.5938μs | 15.4814 KOps/s | 15.5628 KOps/s | $\color{#d91a1a}-0.52\\%$ | | test_setitem_dim[range] | 0.1370ms | 86.2803μs | 11.5901 KOps/s | 11.7283 KOps/s | $\color{#d91a1a}-1.18\\%$ | | test_setitem_dim[tuple] | 99.3260μs | 53.4697μs | 18.7022 KOps/s | 18.7588 KOps/s | $\color{#d91a1a}-0.30\\%$ | | test_setitem | 62.4970μs | 21.5657μs | 46.3699 KOps/s | 47.3594 KOps/s | $\color{#d91a1a}-2.09\\%$ | | test_set | 61.2650μs | 20.7954μs | 48.0875 KOps/s | 48.4757 KOps/s | $\color{#d91a1a}-0.80\\%$ | | test_set_shared | 1.1602ms | 0.1420ms | 7.0416 KOps/s | 7.0592 KOps/s | $\color{#d91a1a}-0.25\\%$ | | test_update | 0.1593ms | 24.4019μs | 40.9804 KOps/s | 41.3385 KOps/s | $\color{#d91a1a}-0.87\\%$ | | test_update_nested | 0.1391ms | 34.4268μs | 29.0471 KOps/s | 30.5816 KOps/s | $\textbf{\color{#d91a1a}-5.02\\%}$ | | test_update__nested | 59.4920μs | 25.5773μs | 39.0972 KOps/s | 39.7787 KOps/s | $\color{#d91a1a}-1.71\\%$ | | test_set_nested | 74.3390μs | 23.4095μs | 42.7177 KOps/s | 44.1949 KOps/s | $\color{#d91a1a}-3.34\\%$ | | test_set_nested_new | 66.4040μs | 28.1707μs | 35.4979 KOps/s | 37.2015 KOps/s | $\color{#d91a1a}-4.58\\%$ | | test_select | 93.7260μs | 43.7846μs | 22.8391 KOps/s | 24.1747 KOps/s | $\textbf{\color{#d91a1a}-5.52\\%}$ | | test_select_nested | 0.1226ms | 64.0738μs | 15.6070 KOps/s | 17.3957 KOps/s | $\textbf{\color{#d91a1a}-10.28\\%}$ | | test_exclude_nested | 0.3174ms | 0.1323ms | 7.5564 KOps/s | 8.4561 KOps/s | $\textbf{\color{#d91a1a}-10.64\\%}$ | | test_empty[True] | 0.7170ms | 0.4002ms | 2.4987 KOps/s | 2.5233 KOps/s | $\color{#d91a1a}-0.98\\%$ | | test_empty[False] | 11.3813μs | 1.3852μs | 721.9214 KOps/s | 981.5195 KOps/s | $\textbf{\color{#d91a1a}-26.45\\%}$ | | test_unbind_speed | 0.5706ms | 0.2647ms | 3.7785 KOps/s | 4.0368 KOps/s | $\textbf{\color{#d91a1a}-6.40\\%}$ | | test_unbind_speed_stack0 | 0.4426ms | 0.2574ms | 3.8849 KOps/s | 4.0290 KOps/s | $\color{#d91a1a}-3.58\\%$ | | test_unbind_speed_stack1 | 69.8018ms | 0.7548ms | 1.3249 KOps/s | 1.4070 KOps/s | $\textbf{\color{#d91a1a}-5.83\\%}$ | | test_split | 67.4559ms | 1.6413ms | 609.2780 Ops/s | 637.4384 Ops/s | $\color{#d91a1a}-4.42\\%$ | | test_chunk | 70.6369ms | 1.6470ms | 607.1697 Ops/s | 638.0973 Ops/s | $\color{#d91a1a}-4.85\\%$ | | test_creation[device0] | 0.1798ms | 83.2402μs | 12.0134 KOps/s | 11.9424 KOps/s | $\color{#35bf28}+0.59\\%$ | | test_creation_from_tensor | 0.1801ms | 83.9563μs | 11.9110 KOps/s | 11.8650 KOps/s | $\color{#35bf28}+0.39\\%$ | | test_add_one[memmap_tensor0] | 0.1025ms | 5.5490μs | 180.2134 KOps/s | 184.6172 KOps/s | $\color{#d91a1a}-2.39\\%$ | | test_contiguous[memmap_tensor0] | 6.8930μs | 0.6340μs | 1.5773 MOps/s | 1.5876 MOps/s | $\color{#d91a1a}-0.65\\%$ | | test_stack[memmap_tensor0] | 24.5960μs | 3.5906μs | 278.5080 KOps/s | 288.0925 KOps/s | $\color{#d91a1a}-3.33\\%$ | | test_memmaptd_index | 1.0479ms | 0.2634ms | 3.7970 KOps/s | 3.9607 KOps/s | $\color{#d91a1a}-4.13\\%$ | | test_memmaptd_index_astensor | 0.7118ms | 0.3387ms | 2.9528 KOps/s | 3.0571 KOps/s | $\color{#d91a1a}-3.41\\%$ | | test_memmaptd_index_op | 1.1512ms | 0.6598ms | 1.5157 KOps/s | 1.5691 KOps/s | $\color{#d91a1a}-3.40\\%$ | | test_serialize_model | 0.1006s | 97.0829ms | 10.3005 Ops/s | 9.8411 Ops/s | $\color{#35bf28}+4.67\\%$ | | test_serialize_model_pickle | 0.4576s | 0.3799s | 2.6323 Ops/s | 2.6140 Ops/s | $\color{#35bf28}+0.70\\%$ | | test_serialize_weights | 0.1101s | 95.8740ms | 10.4304 Ops/s | 9.4222 Ops/s | $\textbf{\color{#35bf28}+10.70\\%}$ | | test_serialize_weights_returnearly | 0.3712s | 0.1520s | 6.5801 Ops/s | 8.2839 Ops/s | $\textbf{\color{#d91a1a}-20.57\\%}$ | | test_serialize_weights_pickle | 1.1040s | 0.5636s | 1.7743 Ops/s | 1.5444 Ops/s | $\textbf{\color{#35bf28}+14.88\\%}$ | | test_serialize_weights_filesystem | 99.1827ms | 93.1650ms | 10.7336 Ops/s | 10.8547 Ops/s | $\color{#d91a1a}-1.12\\%$ | | test_serialize_model_filesystem | 0.1608s | 99.7301ms | 10.0271 Ops/s | 9.8886 Ops/s | $\color{#35bf28}+1.40\\%$ | | test_reshape_pytree | 65.8330μs | 26.1900μs | 38.1826 KOps/s | 37.9730 KOps/s | $\color{#35bf28}+0.55\\%$ | | test_reshape_td | 0.1035ms | 34.3557μs | 29.1073 KOps/s | 29.3528 KOps/s | $\color{#d91a1a}-0.84\\%$ | | test_view_pytree | 73.6180μs | 26.0953μs | 38.3210 KOps/s | 38.7244 KOps/s | $\color{#d91a1a}-1.04\\%$ | | test_view_td | 0.1163ms | 39.8420μs | 25.0991 KOps/s | 25.5649 KOps/s | $\color{#d91a1a}-1.82\\%$ | | test_unbind_pytree | 72.6760μs | 29.7011μs | 33.6687 KOps/s | 33.4518 KOps/s | $\color{#35bf28}+0.65\\%$ | | test_unbind_td | 0.4135ms | 38.9819μs | 25.6529 KOps/s | 26.9167 KOps/s | $\color{#d91a1a}-4.70\\%$ | | test_split_pytree | 62.5570μs | 29.9925μs | 33.3417 KOps/s | 33.3542 KOps/s | $\color{#d91a1a}-0.04\\%$ | | test_split_td | 0.1163ms | 41.5568μs | 24.0635 KOps/s | 25.1661 KOps/s | $\color{#d91a1a}-4.38\\%$ | | test_add_pytree | 70.9330μs | 35.8589μs | 27.8871 KOps/s | 28.6492 KOps/s | $\color{#d91a1a}-2.66\\%$ | | test_add_td | 0.1287ms | 59.9165μs | 16.6899 KOps/s | 17.6133 KOps/s | $\textbf{\color{#d91a1a}-5.24\\%}$ | | test_distributed | 0.1980ms | 0.1040ms | 9.6122 KOps/s | 9.7732 KOps/s | $\color{#d91a1a}-1.65\\%$ | | test_tdmodule | 92.6840μs | 18.8603μs | 53.0215 KOps/s | 52.7429 KOps/s | $\color{#35bf28}+0.53\\%$ | | test_tdmodule_dispatch | 60.8240μs | 37.2241μs | 26.8643 KOps/s | 26.4102 KOps/s | $\color{#35bf28}+1.72\\%$ | | test_tdseq | 43.4720μs | 21.3264μs | 46.8902 KOps/s | 45.1721 KOps/s | $\color{#35bf28}+3.80\\%$ | | test_tdseq_dispatch | 78.7580μs | 42.2173μs | 23.6869 KOps/s | 22.6798 KOps/s | $\color{#35bf28}+4.44\\%$ | | test_instantiation_functorch | 2.4544ms | 1.3236ms | 755.5024 Ops/s | 747.2395 Ops/s | $\color{#35bf28}+1.11\\%$ | | test_instantiation_td | 1.4641ms | 1.0154ms | 984.8749 Ops/s | 986.1014 Ops/s | $\color{#d91a1a}-0.12\\%$ | | test_exec_functorch | 0.2421ms | 0.1622ms | 6.1645 KOps/s | 6.0608 KOps/s | $\color{#35bf28}+1.71\\%$ | | test_exec_functional_call | 0.2937ms | 0.1483ms | 6.7415 KOps/s | 6.5474 KOps/s | $\color{#35bf28}+2.96\\%$ | | test_exec_td | 0.3831ms | 0.1463ms | 6.8366 KOps/s | 6.6174 KOps/s | $\color{#35bf28}+3.31\\%$ | | test_exec_td_decorator | 0.4039ms | 0.2247ms | 4.4501 KOps/s | 4.4885 KOps/s | $\color{#d91a1a}-0.85\\%$ | | test_vmap_mlp_speed[True-True] | 0.9366ms | 0.4979ms | 2.0084 KOps/s | 2.0439 KOps/s | $\color{#d91a1a}-1.74\\%$ | | test_vmap_mlp_speed[True-False] | 0.6575ms | 0.4859ms | 2.0580 KOps/s | 2.0358 KOps/s | $\color{#35bf28}+1.09\\%$ | | test_vmap_mlp_speed[False-True] | 0.6552ms | 0.3943ms | 2.5361 KOps/s | 2.5298 KOps/s | $\color{#35bf28}+0.25\\%$ | | test_vmap_mlp_speed[False-False] | 0.5768ms | 0.3941ms | 2.5377 KOps/s | 2.5325 KOps/s | $\color{#35bf28}+0.21\\%$ | | test_vmap_mlp_speed_decorator[True-True] | 69.6705ms | 0.6092ms | 1.6416 KOps/s | 1.7771 KOps/s | $\textbf{\color{#d91a1a}-7.63\\%}$ | | test_vmap_mlp_speed_decorator[True-False] | 0.7604ms | 0.5629ms | 1.7766 KOps/s | 1.7853 KOps/s | $\color{#d91a1a}-0.49\\%$ | | test_vmap_mlp_speed_decorator[False-True] | 0.6552ms | 0.4622ms | 2.1636 KOps/s | 2.1906 KOps/s | $\color{#d91a1a}-1.23\\%$ | | test_vmap_mlp_speed_decorator[False-False] | 0.6318ms | 0.4606ms | 2.1711 KOps/s | 2.1789 KOps/s | $\color{#d91a1a}-0.36\\%$ | | test_to_module_speed[True] | 2.6420ms | 1.6996ms | 588.3612 Ops/s | 603.4506 Ops/s | $\color{#d91a1a}-2.50\\%$ | | test_to_module_speed[False] | 1.7974ms | 1.6690ms | 599.1608 Ops/s | 612.4751 Ops/s | $\color{#d91a1a}-2.17\\%$ | | test_tc_init | 97.3630μs | 32.9216μs | 30.3752 KOps/s | 30.8601 KOps/s | $\color{#d91a1a}-1.57\\%$ | | test_tc_init_nested | 0.1593ms | 67.5353μs | 14.8071 KOps/s | 15.3724 KOps/s | $\color{#d91a1a}-3.68\\%$ | | test_tc_first_layer_tensor | 5.8309μs | 0.7272μs | 1.3752 MOps/s | 1.4761 MOps/s | $\textbf{\color{#d91a1a}-6.84\\%}$ | | test_tc_first_layer_nontensor | 2.7842μs | 0.7009μs | 1.4268 MOps/s | 1.4742 MOps/s | $\color{#d91a1a}-3.21\\%$ | | test_tc_second_layer_tensor | 17.5930μs | 1.9145μs | 522.3426 KOps/s | 532.7125 KOps/s | $\color{#d91a1a}-1.95\\%$ | | test_tc_second_layer_nontensor | 35.8870μs | 1.7218μs | 580.7842 KOps/s | 611.1330 KOps/s | $\color{#d91a1a}-4.97\\%$ | | test_unbind | 80.7391ms | 7.4625ms | 134.0031 Ops/s | 119.4969 Ops/s | $\textbf{\color{#35bf28}+12.14\\%}$ | | test_full_like | 15.3345ms | 11.5387ms | 86.6649 Ops/s | 91.3730 Ops/s | $\textbf{\color{#d91a1a}-5.15\\%}$ | | test_zeros_like | 12.3471ms | 5.8556ms | 170.7772 Ops/s | 164.0362 Ops/s | $\color{#35bf28}+4.11\\%$ | | test_ones_like | 13.2015ms | 6.7475ms | 148.2021 Ops/s | 157.8128 Ops/s | $\textbf{\color{#d91a1a}-6.09\\%}$ | | test_clone | 12.4891ms | 8.3923ms | 119.1574 Ops/s | 125.0376 Ops/s | $\color{#d91a1a}-4.70\\%$ | | test_squeeze | 60.3230μs | 14.6039μs | 68.4750 KOps/s | 67.7478 KOps/s | $\color{#35bf28}+1.07\\%$ | | test_unsqueeze | 0.1296ms | 61.6399μs | 16.2232 KOps/s | 16.2390 KOps/s | $\color{#d91a1a}-0.10\\%$ | | test_split | 0.2051ms | 0.1155ms | 8.6558 KOps/s | 8.8481 KOps/s | $\color{#d91a1a}-2.17\\%$ | | test_permute | 0.2300ms | 0.1269ms | 7.8793 KOps/s | 7.7383 KOps/s | $\color{#35bf28}+1.82\\%$ | | test_stack | 28.7808ms | 22.9356ms | 43.6003 Ops/s | 43.7669 Ops/s | $\color{#d91a1a}-0.38\\%$ | | test_cat | 28.1501ms | 22.4865ms | 44.4711 Ops/s | 43.7830 Ops/s | $\color{#35bf28}+1.57\\%$ |
github-actions[bot] commented 4 months ago

$\color{#D29922}\textsf{\Large\⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 152. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}26$.

Expand to view detailed results | Name | Max | Mean | Ops | Ops on Repo `HEAD` | Change | | -------------------------------------------------- | --------- | --------- | --------------- | ------------------ | ----------------------------------- | | test_plain_set_nested | 26.5720μs | 13.4166μs | 74.5345 KOps/s | 80.4429 KOps/s | $\textbf{\color{#d91a1a}-7.34\\%}$ | | test_plain_set_stack_nested | 32.2320μs | 13.4542μs | 74.3260 KOps/s | 79.9119 KOps/s | $\textbf{\color{#d91a1a}-6.99\\%}$ | | test_plain_set_nested_inplace | 32.4520μs | 14.7887μs | 67.6191 KOps/s | 72.6974 KOps/s | $\textbf{\color{#d91a1a}-6.99\\%}$ | | test_plain_set_stack_nested_inplace | 40.5220μs | 14.8283μs | 67.4387 KOps/s | 72.0636 KOps/s | $\textbf{\color{#d91a1a}-6.42\\%}$ | | test_items | 22.7320μs | 4.6280μs | 216.0774 KOps/s | 210.8713 KOps/s | $\color{#35bf28}+2.47\\%$ | | test_items_nested | 0.3935ms | 0.3441ms | 2.9065 KOps/s | 2.9616 KOps/s | $\color{#d91a1a}-1.86\\%$ | | test_items_nested_locked | 0.3930ms | 0.3438ms | 2.9089 KOps/s | 2.9451 KOps/s | $\color{#d91a1a}-1.23\\%$ | | test_items_nested_leaf | 0.1025ms | 82.3480μs | 12.1436 KOps/s | 12.0873 KOps/s | $\color{#35bf28}+0.47\\%$ | | test_items_stack_nested | 0.3784ms | 0.3447ms | 2.9014 KOps/s | 2.9407 KOps/s | $\color{#d91a1a}-1.34\\%$ | | test_items_stack_nested_leaf | 0.1026ms | 82.4450μs | 12.1293 KOps/s | 12.1529 KOps/s | $\color{#d91a1a}-0.19\\%$ | | test_items_stack_nested_locked | 0.3986ms | 0.3422ms | 2.9219 KOps/s | 2.9629 KOps/s | $\color{#d91a1a}-1.38\\%$ | | test_keys | 21.6020μs | 4.3519μs | 229.7856 KOps/s | 230.5526 KOps/s | $\color{#d91a1a}-0.33\\%$ | | test_keys_nested | 90.0550μs | 68.7922μs | 14.5365 KOps/s | 14.3823 KOps/s | $\color{#35bf28}+1.07\\%$ | | test_keys_nested_locked | 0.7646ms | 74.1366μs | 13.4886 KOps/s | 13.4442 KOps/s | $\color{#35bf28}+0.33\\%$ | | test_keys_nested_leaf | 74.6730μs | 57.5621μs | 17.3725 KOps/s | 16.6564 KOps/s | $\color{#35bf28}+4.30\\%$ | | test_keys_stack_nested | 94.1950μs | 67.1590μs | 14.8900 KOps/s | 14.7468 KOps/s | $\color{#35bf28}+0.97\\%$ | | test_keys_stack_nested_leaf | 81.3140μs | 58.1525μs | 17.1962 KOps/s | 16.8703 KOps/s | $\color{#35bf28}+1.93\\%$ | | test_keys_stack_nested_locked | 94.5450μs | 72.4118μs | 13.8099 KOps/s | 13.6558 KOps/s | $\color{#35bf28}+1.13\\%$ | | test_values | 8.3940μs | 1.8094μs | 552.6594 KOps/s | 552.1759 KOps/s | $\color{#35bf28}+0.09\\%$ | | test_values_nested | 51.8540μs | 35.3292μs | 28.3052 KOps/s | 28.5587 KOps/s | $\color{#d91a1a}-0.89\\%$ | | test_values_nested_locked | 54.0130μs | 36.8555μs | 27.1330 KOps/s | 26.8521 KOps/s | $\color{#35bf28}+1.05\\%$ | | test_values_nested_leaf | 47.2630μs | 31.2835μs | 31.9658 KOps/s | 32.3356 KOps/s | $\color{#d91a1a}-1.14\\%$ | | test_values_stack_nested | 59.0630μs | 36.2645μs | 27.5751 KOps/s | 27.8215 KOps/s | $\color{#d91a1a}-0.89\\%$ | | test_values_stack_nested_leaf | 48.8420μs | 32.2796μs | 30.9793 KOps/s | 31.0090 KOps/s | $\color{#d91a1a}-0.10\\%$ | | test_values_stack_nested_locked | 62.8030μs | 37.6960μs | 26.5280 KOps/s | 26.4673 KOps/s | $\color{#35bf28}+0.23\\%$ | | test_membership | 1.4361μs | 0.6966μs | 1.4356 MOps/s | 1.4630 MOps/s | $\color{#d91a1a}-1.87\\%$ | | test_membership_nested | 19.1910μs | 2.5219μs | 396.5250 KOps/s | 388.1111 KOps/s | $\color{#35bf28}+2.17\\%$ | | test_membership_nested_leaf | 26.0310μs | 2.5207μs | 396.7100 KOps/s | 389.2667 KOps/s | $\color{#35bf28}+1.91\\%$ | | test_membership_stacked_nested | 22.8310μs | 2.5498μs | 392.1846 KOps/s | 391.4007 KOps/s | $\color{#35bf28}+0.20\\%$ | | test_membership_stacked_nested_leaf | 20.8610μs | 2.5340μs | 394.6293 KOps/s | 389.9994 KOps/s | $\color{#35bf28}+1.19\\%$ | | test_membership_nested_last | 19.6710μs | 3.0931μs | 323.3030 KOps/s | 321.0168 KOps/s | $\color{#35bf28}+0.71\\%$ | | test_membership_nested_leaf_last | 19.4810μs | 3.0952μs | 323.0835 KOps/s | 320.2864 KOps/s | $\color{#35bf28}+0.87\\%$ | | test_membership_stacked_nested_last | 29.9230μs | 9.7481μs | 102.5845 KOps/s | 102.7055 KOps/s | $\color{#d91a1a}-0.12\\%$ | | test_membership_stacked_nested_leaf_last | 27.2910μs | 9.6855μs | 103.2467 KOps/s | 102.9133 KOps/s | $\color{#35bf28}+0.32\\%$ | | test_nested_getleaf | 27.5820μs | 8.3432μs | 119.8581 KOps/s | 120.1756 KOps/s | $\color{#d91a1a}-0.26\\%$ | | test_nested_get | 30.1710μs | 7.8372μs | 127.5959 KOps/s | 127.4533 KOps/s | $\color{#35bf28}+0.11\\%$ | | test_stacked_getleaf | 27.9010μs | 8.4094μs | 118.9148 KOps/s | 119.2715 KOps/s | $\color{#d91a1a}-0.30\\%$ | | test_stacked_get | 23.3910μs | 7.8491μs | 127.4023 KOps/s | 127.5622 KOps/s | $\color{#d91a1a}-0.13\\%$ | | test_nested_getitemleaf | 23.5510μs | 8.4975μs | 117.6822 KOps/s | 117.3127 KOps/s | $\color{#35bf28}+0.31\\%$ | | test_nested_getitem | 32.3410μs | 8.0268μs | 124.5822 KOps/s | 124.3107 KOps/s | $\color{#35bf28}+0.22\\%$ | | test_stacked_getitemleaf | 46.4230μs | 8.5534μs | 116.9125 KOps/s | 117.2755 KOps/s | $\color{#d91a1a}-0.31\\%$ | | test_stacked_getitem | 29.7110μs | 8.0024μs | 124.9620 KOps/s | 125.1345 KOps/s | $\color{#d91a1a}-0.14\\%$ | | test_lock_nested | 58.8374ms | 0.4049ms | 2.4696 KOps/s | 2.5253 KOps/s | $\color{#d91a1a}-2.20\\%$ | | test_lock_stack_nested | 0.3278ms | 0.2926ms | 3.4172 KOps/s | 3.4210 KOps/s | $\color{#d91a1a}-0.11\\%$ | | test_unlock_nested | 60.8772ms | 0.4053ms | 2.4672 KOps/s | 2.4871 KOps/s | $\color{#d91a1a}-0.80\\%$ | | test_unlock_stack_nested | 0.3382ms | 0.3012ms | 3.3203 KOps/s | 3.3228 KOps/s | $\color{#d91a1a}-0.08\\%$ | | test_flatten_speed | 0.4149ms | 0.1024ms | 9.7645 KOps/s | 9.8215 KOps/s | $\color{#d91a1a}-0.58\\%$ | | test_unflatten_speed | 0.3257ms | 0.2895ms | 3.4539 KOps/s | 3.4276 KOps/s | $\color{#35bf28}+0.77\\%$ | | test_common_ops | 1.0641ms | 0.6103ms | 1.6387 KOps/s | 1.7092 KOps/s | $\color{#d91a1a}-4.13\\%$ | | test_creation | 16.4710μs | 1.5914μs | 628.3625 KOps/s | 634.3603 KOps/s | $\color{#d91a1a}-0.95\\%$ | | test_creation_empty | 32.0830μs | 9.7121μs | 102.9639 KOps/s | 126.1587 KOps/s | $\textbf{\color{#d91a1a}-18.39\\%}$ | | test_creation_nested_1 | 27.2220μs | 11.4231μs | 87.5423 KOps/s | 103.0824 KOps/s | $\textbf{\color{#d91a1a}-15.08\\%}$ | | test_creation_nested_2 | 31.9320μs | 13.6557μs | 73.2296 KOps/s | 83.2681 KOps/s | $\textbf{\color{#d91a1a}-12.06\\%}$ | | test_clone | 80.2640μs | 12.0096μs | 83.2669 KOps/s | 85.8705 KOps/s | $\color{#d91a1a}-3.03\\%$ | | test_getitem[int] | 28.4020μs | 11.1468μs | 89.7121 KOps/s | 93.9929 KOps/s | $\color{#d91a1a}-4.55\\%$ | | test_getitem[slice_int] | 42.3430μs | 21.7439μs | 45.9898 KOps/s | 47.7057 KOps/s | $\color{#d91a1a}-3.60\\%$ | | test_getitem[range] | 66.8840μs | 49.5588μs | 20.1781 KOps/s | 20.6405 KOps/s | $\color{#d91a1a}-2.24\\%$ | | test_getitem[tuple] | 50.6630μs | 19.7258μs | 50.6951 KOps/s | 53.0516 KOps/s | $\color{#d91a1a}-4.44\\%$ | | test_getitem[list] | 0.1149ms | 35.2635μs | 28.3580 KOps/s | 28.8005 KOps/s | $\color{#d91a1a}-1.54\\%$ | | test_setitem_dim[int] | 48.2440μs | 31.8186μs | 31.4281 KOps/s | 33.7765 KOps/s | $\textbf{\color{#d91a1a}-6.95\\%}$ | | test_setitem_dim[slice_int] | 88.3950μs | 52.6979μs | 18.9761 KOps/s | 19.5030 KOps/s | $\color{#d91a1a}-2.70\\%$ | | test_setitem_dim[range] | 90.2450μs | 71.6656μs | 13.9537 KOps/s | 14.0013 KOps/s | $\color{#d91a1a}-0.34\\%$ | | test_setitem_dim[tuple] | 63.4330μs | 46.7840μs | 21.3748 KOps/s | 22.8617 KOps/s | $\textbf{\color{#d91a1a}-6.50\\%}$ | | test_setitem | 62.6330μs | 17.5018μs | 57.1371 KOps/s | 63.0555 KOps/s | $\textbf{\color{#d91a1a}-9.39\\%}$ | | test_set | 46.3220μs | 16.8507μs | 59.3447 KOps/s | 65.6463 KOps/s | $\textbf{\color{#d91a1a}-9.60\\%}$ | | test_set_shared | 1.7001ms | 0.1007ms | 9.9338 KOps/s | 10.0966 KOps/s | $\color{#d91a1a}-1.61\\%$ | | test_update | 90.5250μs | 20.2586μs | 49.3618 KOps/s | 55.5341 KOps/s | $\textbf{\color{#d91a1a}-11.11\\%}$ | | test_update_nested | 56.4040μs | 25.1234μs | 39.8035 KOps/s | 43.1151 KOps/s | $\textbf{\color{#d91a1a}-7.68\\%}$ | | test_update__nested | 72.5340μs | 23.3001μs | 42.9182 KOps/s | 44.3894 KOps/s | $\color{#d91a1a}-3.31\\%$ | | test_set_nested | 66.3930μs | 18.2243μs | 54.8718 KOps/s | 60.3446 KOps/s | $\textbf{\color{#d91a1a}-9.07\\%}$ | | test_set_nested_new | 58.2840μs | 20.1587μs | 49.6063 KOps/s | 51.1706 KOps/s | $\color{#d91a1a}-3.06\\%$ | | test_select | 77.8540μs | 32.0718μs | 31.1800 KOps/s | 32.3632 KOps/s | $\color{#d91a1a}-3.66\\%$ | | test_select_nested | 0.6389ms | 51.5589μs | 19.3953 KOps/s | 19.3434 KOps/s | $\color{#35bf28}+0.27\\%$ | | test_exclude_nested | 0.1334ms | 0.1065ms | 9.3922 KOps/s | 9.3019 KOps/s | $\color{#35bf28}+0.97\\%$ | | test_empty[True] | 0.3949ms | 0.3400ms | 2.9414 KOps/s | 2.9015 KOps/s | $\color{#35bf28}+1.37\\%$ | | test_empty[False] | 2.2312μs | 0.7949μs | 1.2580 MOps/s | 1.2336 MOps/s | $\color{#35bf28}+1.98\\%$ | | test_to | 87.9840μs | 58.7709μs | 17.0152 KOps/s | 17.1734 KOps/s | $\color{#d91a1a}-0.92\\%$ | | test_to_nonblocking | 57.2440μs | 35.5634μs | 28.1188 KOps/s | 28.8620 KOps/s | $\color{#d91a1a}-2.58\\%$ | | test_unbind_speed | 0.2986ms | 0.2610ms | 3.8308 KOps/s | 3.9084 KOps/s | $\color{#d91a1a}-1.98\\%$ | | test_unbind_speed_stack0 | 0.3084ms | 0.2566ms | 3.8977 KOps/s | 3.9507 KOps/s | $\color{#d91a1a}-1.34\\%$ | | test_unbind_speed_stack1 | 75.6114ms | 0.7735ms | 1.2928 KOps/s | 1.2956 KOps/s | $\color{#d91a1a}-0.22\\%$ | | test_split | 75.8999ms | 1.7351ms | 576.3497 Ops/s | 586.8128 Ops/s | $\color{#d91a1a}-1.78\\%$ | | test_chunk | 1.6408ms | 1.6034ms | 623.6815 Ops/s | 587.4298 Ops/s | $\textbf{\color{#35bf28}+6.17\\%}$ | | test_creation[device0] | 0.1170ms | 59.5120μs | 16.8033 KOps/s | 17.1356 KOps/s | $\color{#d91a1a}-1.94\\%$ | | test_creation_from_tensor | 0.1319ms | 55.4646μs | 18.0295 KOps/s | 18.6018 KOps/s | $\color{#d91a1a}-3.08\\%$ | | test_add_one[memmap_tensor0] | 0.1016ms | 7.8342μs | 127.6460 KOps/s | 140.6707 KOps/s | $\textbf{\color{#d91a1a}-9.26\\%}$ | | test_contiguous[memmap_tensor0] | 18.4610μs | 0.6769μs | 1.4774 MOps/s | 1.5088 MOps/s | $\color{#d91a1a}-2.08\\%$ | | test_stack[memmap_tensor0] | 30.2320μs | 5.4841μs | 182.3444 KOps/s | 205.4223 KOps/s | $\textbf{\color{#d91a1a}-11.23\\%}$ | | test_memmaptd_index | 1.0100ms | 0.2906ms | 3.4415 KOps/s | 3.5875 KOps/s | $\color{#d91a1a}-4.07\\%$ | | test_memmaptd_index_astensor | 0.6159ms | 0.3525ms | 2.8371 KOps/s | 2.9518 KOps/s | $\color{#d91a1a}-3.88\\%$ | | test_memmaptd_index_op | 77.5016ms | 0.7426ms | 1.3466 KOps/s | 1.5850 KOps/s | $\textbf{\color{#d91a1a}-15.04\\%}$ | | test_serialize_model | 0.1001s | 94.4489ms | 10.5877 Ops/s | 10.2767 Ops/s | $\color{#35bf28}+3.03\\%$ | | test_serialize_model_pickle | 1.3656s | 1.2378s | 0.8079 Ops/s | 0.8072 Ops/s | $\color{#35bf28}+0.08\\%$ | | test_serialize_weights | 0.1770s | 0.1017s | 9.8341 Ops/s | 9.4791 Ops/s | $\color{#35bf28}+3.74\\%$ | | test_serialize_weights_returnearly | 0.2459s | 85.0040ms | 11.7642 Ops/s | 11.8320 Ops/s | $\color{#d91a1a}-0.57\\%$ | | test_serialize_weights_pickle | 1.3585s | 1.2497s | 0.8002 Ops/s | 0.8011 Ops/s | $\color{#d91a1a}-0.12\\%$ | | test_reshape_pytree | 53.7920μs | 26.6282μs | 37.5541 KOps/s | 38.2191 KOps/s | $\color{#d91a1a}-1.74\\%$ | | test_reshape_td | 54.1530μs | 32.3731μs | 30.8899 KOps/s | 31.9101 KOps/s | $\color{#d91a1a}-3.20\\%$ | | test_view_pytree | 57.3240μs | 26.5377μs | 37.6822 KOps/s | 38.9113 KOps/s | $\color{#d91a1a}-3.16\\%$ | | test_view_td | 80.4650μs | 36.9166μs | 27.0881 KOps/s | 28.1769 KOps/s | $\color{#d91a1a}-3.86\\%$ | | test_unbind_pytree | 0.1261ms | 33.0359μs | 30.2701 KOps/s | 31.4091 KOps/s | $\color{#d91a1a}-3.63\\%$ | | test_unbind_td | 0.5092ms | 40.1551μs | 24.9034 KOps/s | 24.5870 KOps/s | $\color{#35bf28}+1.29\\%$ | | test_split_pytree | 59.6440μs | 35.5672μs | 28.1158 KOps/s | 28.3675 KOps/s | $\color{#d91a1a}-0.89\\%$ | | test_split_td | 0.1051ms | 40.5790μs | 24.6433 KOps/s | 24.8230 KOps/s | $\color{#d91a1a}-0.72\\%$ | | test_add_pytree | 0.1770ms | 41.0600μs | 24.3546 KOps/s | 26.1931 KOps/s | $\textbf{\color{#d91a1a}-7.02\\%}$ | | test_add_td | 0.1882ms | 56.5304μs | 17.6896 KOps/s | 19.2624 KOps/s | $\textbf{\color{#d91a1a}-8.17\\%}$ | | test_distributed | 0.2080ms | 68.5590μs | 14.5860 KOps/s | 14.6641 KOps/s | $\color{#d91a1a}-0.53\\%$ | | test_tdmodule | 78.4940μs | 16.4588μs | 60.7579 KOps/s | 69.9127 KOps/s | $\textbf{\color{#d91a1a}-13.09\\%}$ | | test_tdmodule_dispatch | 56.9640μs | 32.0139μs | 31.2364 KOps/s | 35.1450 KOps/s | $\textbf{\color{#d91a1a}-11.12\\%}$ | | test_tdseq | 33.2520μs | 17.0425μs | 58.6767 KOps/s | 62.2390 KOps/s | $\textbf{\color{#d91a1a}-5.72\\%}$ | | test_tdseq_dispatch | 57.5530μs | 33.2101μs | 30.1113 KOps/s | 32.1696 KOps/s | $\textbf{\color{#d91a1a}-6.40\\%}$ | | test_instantiation_functorch | 1.6621ms | 1.4304ms | 699.0810 Ops/s | 693.4594 Ops/s | $\color{#35bf28}+0.81\\%$ | | test_instantiation_td | 1.4972ms | 0.9945ms | 1.0055 KOps/s | 1.0072 KOps/s | $\color{#d91a1a}-0.17\\%$ | | test_exec_functorch | 0.1798ms | 0.1511ms | 6.6186 KOps/s | 6.8261 KOps/s | $\color{#d91a1a}-3.04\\%$ | | test_exec_functional_call | 0.2147ms | 0.1390ms | 7.1930 KOps/s | 7.0634 KOps/s | $\color{#35bf28}+1.83\\%$ | | test_exec_td | 0.1809ms | 0.1395ms | 7.1677 KOps/s | 7.1876 KOps/s | $\color{#d91a1a}-0.28\\%$ | | test_exec_td_decorator | 0.6046ms | 0.2092ms | 4.7791 KOps/s | 4.6994 KOps/s | $\color{#35bf28}+1.69\\%$ | | test_vmap_mlp_speed[True-True] | 0.6397ms | 0.5780ms | 1.7302 KOps/s | 1.7183 KOps/s | $\color{#35bf28}+0.69\\%$ | | test_vmap_mlp_speed[True-False] | 0.6404ms | 0.5780ms | 1.7302 KOps/s | 1.7180 KOps/s | $\color{#35bf28}+0.71\\%$ | | test_vmap_mlp_speed[False-True] | 0.5803ms | 0.5301ms | 1.8864 KOps/s | 1.9508 KOps/s | $\color{#d91a1a}-3.30\\%$ | | test_vmap_mlp_speed[False-False] | 0.5486ms | 0.5064ms | 1.9748 KOps/s | 1.9492 KOps/s | $\color{#35bf28}+1.31\\%$ | | test_vmap_mlp_speed_decorator[True-True] | 0.7535ms | 0.6610ms | 1.5129 KOps/s | 1.4950 KOps/s | $\color{#35bf28}+1.19\\%$ | | test_vmap_mlp_speed_decorator[True-False] | 0.9072ms | 0.6638ms | 1.5065 KOps/s | 1.5630 KOps/s | $\color{#d91a1a}-3.61\\%$ | | test_vmap_mlp_speed_decorator[False-True] | 0.8791ms | 0.6145ms | 1.6274 KOps/s | 1.7635 KOps/s | $\textbf{\color{#d91a1a}-7.72\\%}$ | | test_vmap_mlp_speed_decorator[False-False] | 0.7940ms | 0.5939ms | 1.6839 KOps/s | 1.7594 KOps/s | $\color{#d91a1a}-4.29\\%$ | | test_vmap_transformer_speed[True-True] | 8.4225ms | 7.8995ms | 126.5909 Ops/s | 129.9813 Ops/s | $\color{#d91a1a}-2.61\\%$ | | test_vmap_transformer_speed[True-False] | 8.2669ms | 7.8518ms | 127.3592 Ops/s | 128.8924 Ops/s | $\color{#d91a1a}-1.19\\%$ | | test_vmap_transformer_speed[False-True] | 8.4634ms | 7.8975ms | 126.6228 Ops/s | 124.4469 Ops/s | $\color{#35bf28}+1.75\\%$ | | test_vmap_transformer_speed[False-False] | 9.1285ms | 7.7852ms | 128.4482 Ops/s | 127.7585 Ops/s | $\color{#35bf28}+0.54\\%$ | | test_vmap_transformer_speed_decorator[True-True] | 19.4617ms | 19.0329ms | 52.5407 Ops/s | 52.0567 Ops/s | $\color{#35bf28}+0.93\\%$ | | test_vmap_transformer_speed_decorator[True-False] | 19.6938ms | 19.0352ms | 52.5341 Ops/s | 52.1938 Ops/s | $\color{#35bf28}+0.65\\%$ | | test_vmap_transformer_speed_decorator[False-True] | 18.5207ms | 18.4211ms | 54.2854 Ops/s | 52.3416 Ops/s | $\color{#35bf28}+3.71\\%$ | | test_vmap_transformer_speed_decorator[False-False] | 18.5882ms | 18.4555ms | 54.1844 Ops/s | 52.4789 Ops/s | $\color{#35bf28}+3.25\\%$ | | test_to_module_speed[True] | 2.0828ms | 1.4809ms | 675.2745 Ops/s | 664.8356 Ops/s | $\color{#35bf28}+1.57\\%$ | | test_to_module_speed[False] | 1.9994ms | 1.4504ms | 689.4606 Ops/s | 668.0140 Ops/s | $\color{#35bf28}+3.21\\%$ | | test_tc_init | 49.0320μs | 27.8654μs | 35.8868 KOps/s | 42.5009 KOps/s | $\textbf{\color{#d91a1a}-15.56\\%}$ | | test_tc_init_nested | 88.7050μs | 56.4219μs | 17.7236 KOps/s | 22.1021 KOps/s | $\textbf{\color{#d91a1a}-19.81\\%}$ | | test_tc_first_layer_tensor | 1.1686μs | 0.3568μs | 2.8023 MOps/s | 2.7997 MOps/s | $\color{#35bf28}+0.09\\%$ | | test_tc_first_layer_nontensor | 1.5417μs | 0.3817μs | 2.6200 MOps/s | 2.5956 MOps/s | $\color{#35bf28}+0.94\\%$ | | test_tc_second_layer_tensor | 10.4244μs | 0.9560μs | 1.0461 MOps/s | 1.0070 MOps/s | $\color{#35bf28}+3.88\\%$ | | test_tc_second_layer_nontensor | 0.2300ms | 0.8212μs | 1.2177 MOps/s | 1.1932 MOps/s | $\color{#35bf28}+2.06\\%$ | | test_unbind | 0.1098s | 6.7669ms | 147.7790 Ops/s | 125.1353 Ops/s | $\textbf{\color{#35bf28}+18.10\\%}$ | | test_full_like | 11.3362ms | 11.1110ms | 90.0006 Ops/s | 75.4891 Ops/s | $\textbf{\color{#35bf28}+19.22\\%}$ | | test_zeros_like | 8.3889ms | 7.8702ms | 127.0608 Ops/s | 128.0259 Ops/s | $\color{#d91a1a}-0.75\\%$ | | test_ones_like | 8.3310ms | 7.9216ms | 126.2374 Ops/s | 128.2212 Ops/s | $\color{#d91a1a}-1.55\\%$ | | test_clone | 9.3273ms | 9.1900ms | 108.8133 Ops/s | 109.1400 Ops/s | $\color{#d91a1a}-0.30\\%$ | | test_squeeze | 57.2630μs | 11.0656μs | 90.3704 KOps/s | 90.6923 KOps/s | $\color{#d91a1a}-0.35\\%$ | | test_unsqueeze | 96.2450μs | 53.4934μs | 18.6939 KOps/s | 19.1990 KOps/s | $\color{#d91a1a}-2.63\\%$ | | test_split | 0.1555ms | 99.8983μs | 10.0102 KOps/s | 10.2505 KOps/s | $\color{#d91a1a}-2.34\\%$ | | test_permute | 0.1720ms | 0.1120ms | 8.9275 KOps/s | 9.0012 KOps/s | $\color{#d91a1a}-0.82\\%$ | | test_stack | 26.7387ms | 26.5810ms | 37.6209 Ops/s | 37.6615 Ops/s | $\color{#d91a1a}-0.11\\%$ | | test_cat | 26.8822ms | 26.5639ms | 37.6451 Ops/s | 37.8253 Ops/s | $\color{#d91a1a}-0.48\\%$ |