search

google-ai-edge / mediapipe-samples

Apache License 2.0
1.63k stars 418 forks source link

phi-2,Falcon are not working #458

Open krishna1870 opened 1 month ago

krishna1870 commented 1 month ago

I tried to build phi-2 and falcon on samsung A-55 5g 8gb ram. if i use mediapipe genai 15 and 16 versions model is taking infinite time to give response while i tried with mediapipe 13,14 versions it is giving junk output.

talumbau commented 1 week ago

Can you provide some more info on your steps? There are a couple of issues that have been hitting users that we hope to better address soon:

  1. For non-Google models (like Phi 2 and Falcon), when you go to make the BundleConfig object as discussed here you have to supply a tokenizer model in the SentencePiece ("SP") format, which might not be available. So the extra additional step would be to build a new SP tokenizer model from the given tokenizer for Phi2 (or Falcon or whatever model you choose). If the created SP tokenizer model is not correct, the output will not be correct.
  2. If you have previously converted these models through ai_edge_torch can you try again with the latest main in the repo? Also, please do a fresh install of ai_edge_torch. We needed to pin our TF dependency due to changes in MP. So please use MP 0.10.16 or later.

Thanks! I will leave this issue open so you can respond.