search

Haidra-Org / horde-worker-reGen

The default client software to create images for the AI-Horde
https://aihorde.net/
GNU Affero General Public License v3.0
93 stars 42 forks source link

Dockerfile for ROCm #311

Closed HPPinata closed 1 month ago

HPPinata commented 1 month ago

This is a first Draft of a usable Dockerfile for AMD ROCm cards.

Based on my testing it currently only works with the raw-png branch, probably due to the newer dependencies (9.0.7 on AMD is also broken with micromamba).

The flash_attn handling is also not that pretty, but I'm not sure how to improve it. Feedback is especially welcome here.

I'll also have to do some performance testing to see whether there's any negative impact, and if newer ROCm versions (6.1.2) or even potentially mismatched versions (6.0 on 6.2 or 6.1 on 6.2) improve things.

There might also STILL be some memory leaks during model load, but using Docker and venv compard to micromamba makes bisecting easier.

HPPinata commented 1 month ago

Superseded by: https://github.com/Haidra-Org/horde-worker-reGen/pull/325