search

intel / torch-xpu-ops

Apache License 2.0
23 stars 15 forks source link

Huggingface models accuracy not meet target on MTL #753

Open mengfei25 opened 1 month ago

mengfei25 commented 1 month ago

🐛 Describe the bug

models

xpu eval LayoutLMForSequenceClassification
E0811 04:36:32.669000 134825 torch/_dynamo/utils.py:1555] RMSE (res-fp64): 0.00373, (ref-fp64): 0.00084 and shape=torch.Size([1, 2]). res.dtype: torch.float16, multiplier: 3.000000, tol: 0.001000 E0811 04:36:32.669000 134825 torch/_dynamo/utils.py:1447] Accuracy failed for key name logits fail_accuracy xpu eval DebertaForQuestionAnswering
E0811 03:59:14.697000 113914 torch/_dynamo/utils.py:1555] RMSE (res-fp64): 0.00889, (ref-fp64): 0.00107 and shape=torch.Size([]). res.dtype: torch.float16, multiplier: 3.000000, tol: 0.001000 fail_accuracy xpu eval DebertaV2ForQuestionAnswering
E0811 04:00:28.630000 114111 torch/_dynamo/utils.py:1555] RMSE (res-fp64): 0.00952, (ref-fp64): 0.00170 and shape=torch.Size([]). res.dtype: torch.float16, multiplier: 3.000000, tol: 0.001000 fail_accuracy

Versions

pytorch: 7d5462bfd6171467c0a6c23633732484cf85861e torch-xpu-ops: https://github.com/intel/torch-xpu-ops/commit/24a4557bcb140b9dffebc57e9d47e1849c4e9140

chuanqi129 commented 3 weeks ago

Do we need to double check it and decide whether this issue need to target for PT2.5