ComfyUI_VLM_nodes is very slow to launch

[julien-blanchon]

On my configuration ComfyUI_VLM_nodes take almost 1.7 seconds to launch. This is very long, this make it unpractible for many serverless usage. For comparaison here is the others node launch time on my configuration:

Import times for custom nodes:
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/canvas_tab
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI-Logic
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_Noise
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/masquerade-nodes-comfyui
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_experiments
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_essentials
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_FizzNodes
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI-Custom-Scripts
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/Derfuu_ComfyUI_ModdedNodes
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/comfyui-workspace-manager
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/comfyui-deploy
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI-LLaVA-Captioner
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_Comfyroll_CustomNodes
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/comfyui-reactor-node
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/comfyui-art-venture
   0.1 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI-Manager
   0.1 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI-Impact-Pack
   0.1 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite
   0.3 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/clipseg.py
   0.4 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/comfyui_controlnet_aux
   0.5 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_InstantID
   1.7 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_VLM_nodes

ComfyUI-LLaVA-Captioner take 0.0 sec

[gokayfem]

thanks for the feedback, i will work on this

[gokayfem]

okay it should be a lot faster, now. i am going to make further optimizations soon.

[gokayfem]

now it takes 0.2 seconds

[julien-blanchon]

Yes !

   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/canvas_tab
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI-Logic
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/masquerade-nodes-comfyui
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_experiments
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_essentials
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_FizzNodes
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_Noise
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI-Custom-Scripts
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/Derfuu_ComfyUI_ModdedNodes
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/comfyui-workspace-manager
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/comfyui-deploy
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI-LLaVA-Captioner
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_Comfyroll_CustomNodes
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/comfyui-reactor-node
   0.0 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/comfyui-art-venture
   0.1 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI-Manager
   0.1 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI-Impact-Pack
   0.1 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite
   0.3 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/clipseg.py
   0.3 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/comfyui_controlnet_aux
   0.4 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_VLM_nodes
   0.5 seconds: /media/julien-blanchon/DNA1/ComfyUI/custom_nodes/ComfyUI_InstantID

Btw It's not idea to store models in the custom_nodes directory because in many custom we need to have a separation between models storage and custom_nodes storage. For example we might want to mount the models directory using a remote volume and keep the custom_nodes in a light docker image. It's not a big deal

[julien-blanchon]

I'm closing the issue, thanks you very much for this good project

[gokayfem]

thanks for feedback about folders, im planning to move all folders to llavacheckpoints

[gokayfem]

now all of the model files in their respective folders in models/LLavacheckpoints