search

keras-team / keras-nlp

Modular Natural Language Processing workflows with Keras
Apache License 2.0
730 stars 215 forks source link

Add remainig Phi-3 models #1645

Open awsaf49 opened 1 month ago

awsaf49 commented 1 month ago

The Phi3-mini model has recently been added to Keras-NLP (https://github.com/keras-team/keras-nlp/pull/1597). However, more models such as Phi-small, Phi3-medium, and Phi3-vision have been released. It would be great to have them included here.

I would love to work on this, but @abuelnasr0 has already done great work on Phi3-mini. So, if he can work on it, he should do it.

cc: @mattdangerw

Reference: HuggingFace

GOHqZ7iasAM0O9O

kernel-loophole commented 4 weeks ago

hi there @awsaf49 i can help to add new models .

awsaf49 commented 4 weeks ago

That's great. Sadly, I am unable to contribute on this one due to some ongoing workload. It would be great if keras-team could assign it to you.

kernel-loophole commented 4 weeks ago

okay sure

abuelnasr0 commented 3 weeks ago

@awsaf49 Thanks for reporting! I just read this issue when @kernel-loophole commented.

I looked at the new models and I found that phi3-meduim can be converted easily using the conversion script, but unfortunately I don't have compute resources to run the script, so If @kernel-loophole or @mattdangerw can run the script, it will be great.

But phi3-small uses a modified model and uses different tokenizer so it will need a new modeling, tokenizer, and conversion script. If @kernel-loophole can start working on it, it will be a great addition to the library.

Phi3-vision will be a new challenge for me actually, I didn't write a VLM model yet, so I would like to work in it, if @kernel-loophole haven't planned to implement phi3-vision, I can start working on it or we can collaborate if you like.

kernel-loophole commented 3 weeks ago

@abuelnasr0 what are the spec to run the conversion script .and yes i am looking forward to discuss more.happy to implement phi3-small