Accelerate offloading fixes - Githubissues

one-some / KoboldAI-united

GNU Affero General Public License v3.0

3 stars 2 forks source link

Accelerate offloading fixes #20

Closed henk717 closed 1 year ago

henk717 commented 1 year ago

Fixes CPU usage
Fixes --cpu because of a new method to hide the GPU's that allows us to remove all other checks in the future
Fixes the console bar spam

Pending:

ROCm doesn't work since it uses an older Torch version (New one was not yet stable on ROCm)
TPU does not work since it tries to do GPU stuff in the patchers file (Old Torch stuff but also accelerate which does not support the TPU)
Loading in UI2 does not report progress or remaining time