explosion / curated-transformers

🤖 A PyTorch library of curated Transformer models and their composable components
MIT License
864 stars 34 forks source link

Move the old Falcon architecuture to the extras/addons pacakage #355

Open shadeMe opened 1 year ago

shadeMe commented 1 year ago

We currently support two different architectures for the Falcon family of models. This adds a fair bit of complexity to the codebase since one of the architectures doesn't lend itself to the reuse of the existing components we've already implemented. So, it would be a good idea to move this arch to the extras/addons package.