Mac M2 getting error:: size mismatch for proj_in.weight

[seeqor]

MacBook Pro M2 96gig

created conda env exclusive to SM yesterday (May 1) installed most recent (May 1) Pytorch nightly installed most recent (May 1) ComfyUI installed most recent (May 1) ComfyUI Manager installed most recent (May 1) Steerable Motion downloaded workflows refereced on Steerable Motion's github installed most recent (May 1) missing custom_nodes

added four images (all same size/ratio in each test) (used various image sizes, various ratios, various 1.5 checkpoints, various workflows)

verified path to all models Queued Prompt Preview Image node recieved images without issue

I've been geting the following error in a variety of installs since the first release of Steerable Motion. Been busy with other projects, now ready to make this work on my Mac.

Batch Creative Interpolation node gets purple outline red letter error dialog pops up Terminal displays following error:

got prompt model_type EPS Using split attention in VAE Using split attention in VAE clip missing: ['clip_l.logit_scale', 'clip_l.transformer.text_projection.weight'] /Users/seeqor/miniconda3/envs/comfy_SM/ComfyUI/custom_nodes/ComfyUI_FizzNodes/BatchFuncs.py:169: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value ' ' has dtype incompatible with float64, please explicitly cast to a compatible dtype first. cur_prompt_series[f] = str(current_prompt) /Users/seeqor/miniconda3/envs/comfy_SM/ComfyUI/custom_nodes/ComfyUI_FizzNodes/BatchFuncs.py:170: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value ' ' has dtype incompatible with float64, please explicitly cast to a compatible dtype first. nxt_prompt_series[f] = str(next_prompt) Requested to load SD1ClipModel Loading 1 new model Requested to load CLIPVisionModelProjection Loading 1 new model !!! Exception during processing!!! Error(s) in loading state_dict for ResamplerImport: size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]). Traceback (most recent call last): File "/Users/seeqor/miniconda3/envs/comfy_SM/ComfyUI/execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/seeqor/miniconda3/envs/comfy_SM/ComfyUI/execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/seeqor/miniconda3/envs/comfy_SM/ComfyUI/execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(*slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/seeqor/miniconda3/envs/comfy_SM/ComfyUI/custom_nodes/steerable-motion/SteerableMotion.py", line 568, in combined_function model, = ipadapter_application.apply_ipadapter(model=model, ipadapter=ipadapter, image=torch.cat(bin.imageBatch, dim=0), weight=[x base_ipa_advanced_settings["ipa_weight"] for x in bin.weight_schedule], weight_type=base_ipa_advanced_settings["ipa_weight_type"], start_at=base_ipa_advanced_settings["ipa_starts_at"], end_at=base_ipa_advanced_settings["ipa_ends_at"], clip_vision=clip_vision,image_negative=negative_noise,embeds_scaling=base_ipa_advanced_settings["ipa_embeds_scaling"], image_schedule=bin.image_schedule) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/seeqor/miniconda3/envs/comfy_SM/ComfyUI/custom_nodes/steerable-motion/imports/ComfyUI_IPAdapter_plus/IPAdapterPlus.py", line 686, in apply_ipadapter return (ipadapter_execute(model.clone(), ipadapter_model, clip_vision, **ipa_args), ) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/seeqor/miniconda3/envs/comfy_SM/ComfyUI/custom_nodes/steerable-motion/imports/ComfyUI_IPAdapter_plus/IPAdapterPlus.py", line 334, in ipadapter_execute ipa = IPAdapterImport( ^^^^^^^^^^^^^^^^ File "/Users/seeqor/miniconda3/envs/comfy_SM/ComfyUI/custom_nodes/steerable-motion/imports/ComfyUI_IPAdapter_plus/IPAdapterPlus.py", line 69, in init self.image_proj_model.load_state_dict(ipadapter_model["image_proj"]) File "/Users/seeqor/miniconda3/envs/comfy_SM/lib/python3.11/site-packages/torch/nn/modules/module.py", line 2191, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for ResamplerImport: size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]).

Prompt executed in 5.40 seconds

[peteromallet]

It looks like you have have the wrong Clip Vision model selection - see the instructions on the bottom left

[seeqor]

That was it. My bad. I'm usually the one to remind folks to read the documentation. Generating video without issue. Thanks!