search

pytorch-labs / tritonbench

Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.
BSD 3-Clause "New" or "Revised" License
21 stars 3 forks source link

Add embedding,rms_norm,rope #15

Closed FindHao closed 4 weeks ago

FindHao commented 1 month ago

Migrated from https://github.com/pytorch/benchmark/pull/2517 Add embedding,rms_norm,rope from liger kernel

Test Plan:

% python run.py --op embedding,rms_norm,rope  --num-inputs 1
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:02<00:00,  2.02s/it]
  x_val    torch_embedding-latency    liger_embedding-latency    inductor_embedding-latency
-------  -------------------------  -------------------------  ----------------------------
      0                   0.102272                   0.034432                      0.033856
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.34it/s]
  x_val    llama_rms-latency    liger_rms-latency    inductor_rms-latency
-------  -------------------  -------------------  ----------------------
      0             0.043584             0.013504                0.018048
  0%|                                                                                                                                                          | 0/1 [00:00<?, ?it/s]`LlamaRotaryEmbedding` can now be fully parameterized by passing the model config through the `config` argument. All other arguments will be removed in v4.46
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:05<00:00,  5.96s/it]
  x_val    apply_rotary_pos_emb-latency    liger_rotary_pos_emb-latency    inductor_rotary_pos_emb_full_op-latency
-------  ------------------------------  ------------------------------  -----------------------------------------
      0                        0.291264                         0.05024                                   0.055104
FindHao commented 1 month ago

Need to fix liger collection later.

facebook-github-bot commented 1 month ago

@FindHao has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 1 month ago

@FindHao has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

FindHao commented 4 weeks ago

bot failed to close this OSS PR. The code has been merged.