search

kimborgen / falcon-llm

Apache License 2.0
1 stars 0 forks source link

Split Alibi and Rotary models #8

Closed kimborgen closed 1 year ago

kimborgen commented 1 year ago

As determined in #2, split the model into two different models. One that uses Rotary embeddings and one using Alibi functionality. This will increase readability and make it easier to understand and extend the models.

kimborgen commented 1 year ago

Extracted rotary code, alibi is defered and put into not_working folder...