Phi3 conversion to tflite

[a8nova]

Description of the bug:

I wanted to convert phi3 to tflite, I started modifying phi2.py to apply the differences. In the Attention class of phi3, there is a combined projection layer self.qkv_proj = nn.Linear(self.hidden_size, op_size, bias=False).

I have defined the TENSOR_NAMES for phi3 just like in phi2.py, I am having an issue with ModelLoader class in loader.py where it has:

  @dataclass
  class TensorNames:
    attn_query_proj: str
    attn_key_proj: str
    attn_value_proj: str
    attn_output_proj: str
    ...

I can't seem to pass the combined self.qkv_proj to the loader class..

Thanks!

Actual vs expected behavior:

No response

Any other information you'd like to share?

No response

[haozha111]

Thanks for reporting the issue! Yes as you mentioned, Phi-3 has an updated attention mechanism which is different from what the library provides as of today. Adding @hheydary to look into the issue (who is original author for the phi-2 example).

[hheydary]

Greetings,

We just added a new field to the loader class attn_fused_qkv_proj. Please use that to load model parameters.

[a8nova]

Thank you for the quick fix @haozha111 @hheydary