search

airockchip / rknn-llm

Other
344 stars 29 forks source link

Guide to adapting new models? #95

Open kinchahoy opened 1 week ago

kinchahoy commented 1 week ago

Folks,

Could you provide a small example or walkthrough on how to adapt a new model type? I'm thinking in particular of multi-modal model types like Florence2ForConditionalGeneration or Phi3VForCausalLM

Thank you!

happyme531 commented 3 days ago

I implemented Florence2: https://huggingface.co/happyme531/Florence-2-base-ft-ONNX-RKNN2/tree/main For Phi3V, it is really hard to do