search

elixir-nx / bumblebee

Pre-trained Neural Network models in Axon (+ 🤗 Models integration)
Apache License 2.0
1.26k stars 90 forks source link

Add Phi-3 model #374

Closed bowyern closed 1 month ago

bowyern commented 1 month ago

My attempt at adding phi3-4k to Bumblebee. The outputs aren't right, and in talking with Jonatan it looks like there's a problem between the embedder and where the FFN is applied in the decoder layers.

jonatanklosko commented 1 month ago

Thanks!