pytorch / executorch

On-device AI across mobile, embedded and edge for PyTorch
https://pytorch.org/executorch/
Other
1.31k stars 206 forks source link

fix constant tagging in mps backend #3503

Closed cccclai closed 1 week ago

cccclai commented 2 weeks ago

Summary: Test with https://github.com/pytorch/executorch/pull/3399 and this command passes

python -m examples.models.llama2.export_llama -kv --mps

Without this diff, it will error out

in _verify_exported_program_signature
    raise SpecViolationError(
torch._export.verifier.SpecViolationError: Buffer output getitem_1 does not point to a buffer that exists.
Dict of buffers that are mutated, in order: {'getitem_1': 'layers_0_attention_SDPA_kv_cache_k_cache', 'getitem': 'layers_0_attention_SDPA_kv_cache_v_cache', 'getitem_3': 'layers_1_attention_SDPA_kv_cache_k_cache', 'getitem_2': 'layers_1_attention_SDPA_kv_cache_v_cache', 'getitem_5': 'layers_2_attention_SDPA_kv_cache_k_cache', 'getitem_4': 'layers_2_attention_SDPA_kv_cache_v_cache', 'getitem_7': 'layers_3_attention_SDPA_kv_cache_k_cache', 'getitem_6': 'layers_3_attention_SDPA_kv_cache_v_cache', 'getitem_9': 'layers_4_attention_SDPA_kv_cache_k_cache', 'getitem_8': 'layers_4_attention_SDPA_kv_cache_v_cache'}
Buffer nodes available: []

The root cause is that by is_parameter, it tags all data including mutable buffers.

Differential Revision: D56941763

pytorch-bot[bot] commented 2 weeks ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/3503

Note: Links to docs will display an error until the docs builds have been completed.

:x: 1 New Failure

As of commit 25eae446d14f5d54693d4f590ec88dcad8d3672a with merge base a116d89b83e65875137351ca50ea880bf97c982a (image):

NEW FAILURE - The following job has failed:

* [Apple / upload-frameworks-ios](https://hud.pytorch.org/pr/pytorch/executorch/3503#24663085041) ([gh](https://github.com/pytorch/executorch/actions/runs/8979455051/job/24663085041)) `Credentials could not be loaded, please check your action inputs: Could not load credentials from any providers`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot commented 2 weeks ago

This pull request was exported from Phabricator. Differential Revision: D56941763

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D56941763

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D56941763

facebook-github-bot commented 1 week ago

This pull request has been merged in pytorch/executorch@50e9ee92f632fc9242b3d7387ed2a954b0e34f91.