PygmalionAI / aphrodite-engine

PygmalionAI's large-scale inference engine
https://pygmalion.chat
GNU Affero General Public License v3.0
660 stars 80 forks source link

feat: model execution refactor #367

Closed AlpinDale closed 2 months ago

AlpinDale commented 2 months ago

To keep in line with upstream, model execution has been split into modular classes now. Should make support for other devices, e.g. CPU, much easier. Also, engine code is simplified now.