microsoft / archai

Accelerate your Neural Architecture Search (NAS) through fast, reproducible and modular research.

https://microsoft.github.io/archai

MIT License

467 stars 92 forks source link

New Tf++ search space #197

Closed piero2c closed 1 year ago

piero2c commented 1 year ago

Pull Request Template

Description

Update TF++ search space with

[x] Flash-attention optimization
[X] FlashFFT optimization for Sgconv ops
[X] Refactored code
[X] Option to switch between different backbones (Codegen and GPT2)
[X] Sgconv3 new primitive

Final checks:

[ ] Follows guidelines of project
[ ] Self-review of code
[ ] Commented code in hard-to-understand areas
[ ] Corresponding changes to the documentation
[ ] Changes generate no new warnings
[ ] Dependent changes have already been merged
[ ] Checked code correction any misspellings