OpenVino Mac?

[cchance27]

Just wondering why openvino on mac wasn't implemented did it not result in a similar speedup?

[rupeshs]

@cchance27 I don't have a Mac machine to test, if you can check it, please let me know. https://github.com/rupeshs/fastsdcpu#fastsd-cpu-on-mac

[cchance27]

I commented out the gui exclusion for "darwin" that blocked osx from using openvino and got the following when i tried to use openvino

It appears your using optimum-intel maybe thats intel restricted for converting the unet?

Compiling the vae_decoder to CPU ... Compiling the unet to CPU ... Traceback (most recent call last): File "/Users/user/Downloads/fastsdcpu/src/frontend/gui/image_generator_worker.py", line 29, in run result = self.fn(*self.args, self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/Downloads/fastsdcpu/src/frontend/gui/app_window.py", line 534, in generate_image images = self.context.generate_text_to_image( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/Downloads/fastsdcpu/src/context.py", line 29, in generate_text_to_image self.lcm_text_to_image.init( File "/Users/user/Downloads/fastsdcpu/src/backend/lcm_text_to_image.py", line 163, in init self.pipeline = OVStableDiffusionPipeline.from_pretrained( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/christopherchance/Downloads/fastsdcpu/env/lib/python3.11/site-packages/optimum/modeling_base.py", line 372, in from_pretrained return from_pretrained_method( ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/Downloads/fastsdcpu/env/lib/python3.11/site-packages/optimum/intel/openvino/modeling_diffusion.py", line 274, in _from_pretrained return cls(unet=unet, config=config, model_save_dir=model_save_dir, components, **kwargs)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/Downloads/fastsdcpu/env/lib/python3.11/site-packages/optimum/intel/openvino/modeling_diffusion.py", line 125, in init self.compile() File "/Users/user/Downloads/fastsdcpu/env/lib/python3.11/site-packages/optimum/intel/openvino/modeling_diffusion.py", line 499, in compile self.unet._compile() File "/Users/user/Downloads/fastsdcpu/env/lib/python3.11/site-packages/optimum/intel/openvino/modeling_diffusion.py", line 542, in _compile self.request = core.compile_model(self.model, self.device, self.ov_config)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/Downloads/fastsdcpu/env/lib/python3.11/site-packages/openvino/runtime/ie_api.py", line 543, in compile_model super().compile_model(model, device_name, {} if config is None else config),

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: Exception from src/inference/src/core.cpp:116: [ GENERAL_ERROR ] could not create a primitive descriptor for an inner product forward propagation primitive

On Tue, 14 Nov 2023 at 22:38, Rupesh Sreeraman @.***> wrote:

@cchance27 https://github.com/cchance27 I don't have a Mac machine to test, if you can check it, please let me know. https://github.com/rupeshs/fastsdcpu#fastsd-cpu-on-mac

— Reply to this email directly, view it on GitHub https://github.com/rupeshs/fastsdcpu/issues/65#issuecomment-1811758814, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADJA5V323TLCJ7DHQWLNRCLYEQ2KVAVCNFSM6AAAAAA7LUKT6SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJRG42TQOBRGQ . You are receiving this because you were mentioned.Message ID: @.***>

[rupeshs]

@cchance27 Are you using M1?

[jamienk]

FYI on my 2015 Mac with 2.5 GHz Quad-Core Intel Core i7, this works and enables openvino

[rupeshs]

FYI on my 2015 Mac with 2.5 GHz Quad-Core Intel Core i7, this works and enables openvino

Thanks for trying FastSD CPU