search

ReaLLMASIC / nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.
MIT License
20 stars 16 forks source link

Quests: Rotary Positional Embedding #116

Open gkielian opened 4 months ago

gkielian commented 4 months ago
  1. Test the ROPE implementations, both current software and hardware implementations against publicly available implementations
  2. Look into forward and backward pass -- straight-through implementations
gkielian commented 4 months ago

@alibillalhammoud

gkielian commented 4 months ago

https://github.com/ReaLLMASIC/nanoGPT/pull/115