File Changes
DockerFile - Added onediff and now repo works on torch 2.3.0 with other libs on
pipelines/text-to-image.py
routes/text-to-image.py
bench.py - make it optimized for adding multiple optimization
How to run this
Download all the model as mention in readme.md under runner dir
Build the docker image
docker build -t ai-runner-benchmark .
Run the bench.py
docker run --gpus all -v ./models:/models ai-runner-benchmark:latest python bench.py --pipeline text-to-image --model_id SG161222/RealVisXL_V4.0_Lightning --runs 3 --batch_size 1 --num_inference_steps 10 --optimization onediff
As for What i did i added an onediff optimization support under which i made some changes in the following files Issue Posted Upon
https://github.com/livepeer/bounties/issues/37
Previous Pr https://github.com/livepeer/ai-worker/pull/149
File Changes DockerFile - Added onediff and now repo works on torch 2.3.0 with other libs on pipelines/text-to-image.py routes/text-to-image.py bench.py - make it optimized for adding multiple optimization How to run this
Download all the model as mention in readme.md under runner dir Build the docker image docker build -t ai-runner-benchmark . Run the bench.py docker run --gpus all -v ./models:/models ai-runner-benchmark:latest python bench.py --pipeline text-to-image --model_id SG161222/RealVisXL_V4.0_Lightning --runs 3 --batch_size 1 --num_inference_steps 10 --optimization onediff