Closed YousufSSyed closed 10 months ago
I was wondering the same thing so I knocked up a quick Diffusers script for a 512x512 image running on a 8gb MacMini M1
on CPU float32 - 10/10 [00:46<00:00, 4.70s/it] for MPS using float16* - 10/10 [00:18<00:00, 1.84s/it]
Note it starts slow and gets quicker so less steps would be slower per iteration (but faster over all) it seems to peak at around 1.99s/iter on the CPU
*float32 MPS requires the use of swap space on my MacMini so SSD access slowed it down but for completeness I got 10/10 [02:30<00:00, 15.10s/it] On a 16Gb M1/2 it would be significantly faster.
@Vargol Nice insights, I don't have a Mac machine to test these things.
@YousufSSyed,@Vargol Yes, I have created a test branch for Mac support. Could you please try this?
git clone https://github.com/rupeshs/fastsdcpu/
cd fastsdcpu
git checkout add-mac-cpu-support
chmod +x install-mac.sh
./install-mac.sh
To start : ./start.sh
Feel free to share your findings
install it on my MBP M2+ running around 2.82s/it on a 512x768 generated image , if you need any specific test let me know , was able to generate 10 images at the same time at around 20s/it
@Autantpourmoi Thank you for tests,so the above commands worked right? I think you checked desktop GUI? how much time it takes to generate a single 512x 512 image 4 steps?
@Autantpourmoi Could you please share one screenshot of FastSD CPU running on Mac. I will add it in the readme.
everything seems to work right , yes desktop gui . 512x512 / 4 steps / 10.49 seconds
@rupeshs is that ok ?
@Autantpourmoi yes, thank you so much for the Mac test support
@rupeshs no problem , feel free if you need any other things. one thing , using OpeVINO ended with error ,actually, i don't know what is it for OSError: /Users/lucvaligny/.cache/huggingface/hub/models--deinferno--LCM_Dreamshaper_v7-openvino/snapshots/62289fd6f0dcdde26d7e86cba0e50815611a5c6d does not appear to have a file named preprocessor_config.json. Checkout 'https://huggingface.co//Users/lucvaligny/.cache/huggingface/hub/models--deinferno--LCM_Dreamshaper_v7-openvino/snapshots/62289fd6f0dcdde26d7e86cba0e50815611a5c6d/main' for available files.
@Autantpourmoi can you retry OpenVINO? @deinferno
should i reinstall first or git clone ? the 2 small arrow on the side of the image aren't working
@Autantpourmoi No,simply check use_openvino setting and generate
same result File "/Users/lucvaligny/fastsdcpu/src/frontend/gui/image_generator_worker.py", line 29, in run result = self.fn(*self.args, self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucvaligny/fastsdcpu/src/frontend/gui/app_window.py", line 387, in generate_image images = self.context.generate_text_to_image( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucvaligny/fastsdcpu/src/context.py", line 27, in generate_text_to_image self.lcm_text_to_image.init( File "/Users/lucvaligny/fastsdcpu/src/backend/lcm_text_to_image.py", line 56, in init self.pipeline = OVLatentConsistencyModelPipeline.from_pretrained( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucvaligny/fastsdcpu/env/lib/python3.11/site-packages/optimum/modeling_base.py", line 372, in from_pretrained return from_pretrained_method( ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucvaligny/fastsdcpu/env/lib/python3.11/site-packages/optimum/intel/openvino/modeling_diffusion.py", line 252, in _from_pretrained kwargs[name] = load_method(new_model_save_dir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucvaligny/fastsdcpu/env/lib/python3.11/site-packages/transformers/image_processing_utils.py", line 202, in from_pretrained image_processor_dict, kwargs = cls.get_image_processor_dict(pretrained_model_name_or_path, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucvaligny/fastsdcpu/env/lib/python3.11/site-packages/transformers/image_processing_utils.py", line 329, in get_image_processor_dict resolved_image_processor_file = cached_file( ^^^^^^^^^^^^ File "/Users/lucvaligny/fastsdcpu/env/lib/python3.11/site-packages/transformers/utils/hub.py", line 400, in cached_file raise EnvironmentError( OSError: /Users/lucvaligny/.cache/huggingface/hub/models--deinferno--LCM_Dreamshaper_v7-openvino/snapshots/62289fd6f0dcdde26d7e86cba0e50815611a5c6d does not appear to have a file named preprocessor_config.json. Checkout 'https://huggingface.co//Users/lucvaligny/.cache/huggingface/hub/models--deinferno--LCM_Dreamshaper_v7-openvino/snapshots/62289fd6f0dcdde26d7e86cba0e50815611a5c6d/main' for available files.
Ok it seems like OpenVINO is hard on Mac
what is it for ?
OpenVINO increases image generation speed by 2x it is only for CPU.
Uncheck the use_openvino setting.
@Autantpourmoi Could you please run this command
export DEVICE=mps
start using start.sh
, it will access the GPUuncheck openvino setting
Could you please run this command
export DEVICE=mps
start using start.sh
, it will access the GPU
Are you seeing any speed improvement?
are you also dev for comfy ui this lcm ?
No, I created FastSD CPU I have other tools like DiffusionMagic https://github.com/rupeshs/diffusionmagic and neuralstyler.com
with export DEVICE=mps 512x512/ 4 steps / 1.57 seconds
So speed increased now it is using GPU on the Mac
i should keep it like that , will try generate 10 images with this settings/ lol don't have time to clik 10 times that first images already there 512x512 / 25 steps / 6.52 sec
My testing is over.
@Autantpourmoi Thanks for your great support.
@rupeshs no pb
@Autantpourmoi how much system RAM do you have?
64Go
@YousufSSyed Add mac support in the release 1.0.0 beta 8 https://github.com/rupeshs/fastsdcpu/releases/tag/v1.0.0-beta.8
I've heard it runs fast on Intel Core CPUs, I wonder how fast it would be on M1 and M2 Macs.