EvilBT
commented
1 month ago It seems you are encountering an error related to missing model files. Please make sure you have followed step 3 in the readme.md file, which involves manually downloading the Joy-Caption-alpha-two model files.
Specifically, you need to:
Navigate to the Joy-Caption-alpha-two repository on Hugging Face: https://huggingface.co/spaces/fancyfeast/joy-caption-alpha-two/tree/main
Download the contents of the cgrkzexw-599808 folder.
Copy the entire downloaded folder into your ComfyUI models/Joy_caption_two directory.
It's crucial to download and place these files correctly for the node to function properly.
Please carefully review the readme file, especially the sections about downloading and placing the necessary models. If you continue to have issues after following these steps, feel free to ask for further assistance.
chentongwilliam
System Info
Platform: Linux OS: Ubuntu 22 Python version: 3.10.14 (main, May 6 2024, 19:42:50) [GCC 11.2.0] Total VRAM 24202 MB, total RAM 64093 MB pytorch version: 2.5.0 for cuda 12.4 Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
Package Info
huggingface_hub==0.23.4 transformers==4.46.0 (same issue by 4.44.0) numpy==1.26.4 sentencepiece==0.2.0 pillow>=10.4.0 bitsandbytes==0.44.1 peft==0.13.2
Model Info
siglip model downloaded from: https://huggingface.co/google/siglip-so400m-patch14-384/tree/main
Traceback (most recent call last):
File "/home/chentong/Documents/DeepLearning/AIGC/ComfyUI/execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/home/chentong/Documents/DeepLearning/AIGC/ComfyUI/execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/home/chentong/Documents/DeepLearning/AIGC/ComfyUI/execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "/home/chentong/Documents/DeepLearning/AIGC/ComfyUI/execution.py", line 158, in process_inputs results.append(getattr(obj, func)(**inputs)) File "/home/chentong/Documents/DeepLearning/AIGC/ComfyUI/custom_nodes/ComfyUI_SLK_joy_caption_two/joy_caption_two_node.py", line 383, in generate text_model = joy_two_pipeline.llm.load_llm_model() File "/home/chentong/Documents/DeepLearning/AIGC/ComfyUI/custom_nodes/ComfyUI_SLK_joy_caption_two/joy_caption_two_node.py", line 177, in load_llm_model text_model = AutoModelForCausalLM.from_pretrained(text_model_path, File "/home/chentong/miniconda3/envs/comfyenv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 567, in from_pretrained raise ValueError( ValueError: Unrecognized configuration class <class 'transformers.models.siglip.configuration_siglip.SiglipConfig'> for this kind of AutoModel: AutoModelForCausalLM. Model type should be one of BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BloomConfig, CamembertConfig, LlamaConfig, CodeGenConfig, CohereConfig, CpmAntConfig, CTRLConfig, Data2VecTextConfig, DbrxConfig, ElectraConfig, ErnieConfig, FalconConfig, FalconMambaConfig, FuyuConfig, GemmaConfig, Gemma2Config, GitConfig, GlmConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTNeoXJapaneseConfig, GPTJConfig, GraniteConfig, GraniteMoeConfig, JambaConfig, JetMoeConfig, LlamaConfig, MambaConfig, Mamba2Config, MarianConfig, MBartConfig, MegaConfig, MegatronBertConfig, MistralConfig, MixtralConfig, MllamaConfig, MoshiConfig, MptConfig, MusicgenConfig, MusicgenMelodyConfig, MvpConfig, NemotronConfig, OlmoConfig, OlmoeConfig, OpenLlamaConfig, OpenAIGPTConfig, OPTConfig, PegasusConfig, PersimmonConfig, PhiConfig, Phi3Config, PhimoeConfig, PLBartConfig, ProphetNetConfig, QDQBertConfig, Qwen2Config, Qwen2MoeConfig, RecurrentGemmaConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, RwkvConfig, Speech2Text2Config, StableLmConfig, Starcoder2Config, TransfoXLConfig, TrOCRConfig, WhisperConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, XmodConfig, ZambaConfig.