Refactored model fetching and updating logic, creating a generic createModelFetchExtension function.
Registered model fetching extensions separately for LLM and VLM nodes.
Updated node UI to support multiline text output display.
5. Other Optimizations
Added new VLM node to NODE_CLASS_MAPPINGS and NODE_DISPLAY_NAME_MAPPINGS.
Retained "No LLM Enhancement" and "No VLM Enhancement" options for improved compatibility and flexibility.
6. Test Workflow
Provided a test workflow file LVM node test workflow.json to demonstrate and validate the functionality of the new VLM node.
This update enables the BizyAir node library to handle more complex multimodal AI tasks. Users can now seamlessly integrate text and image inputs in ComfyUI, achieving richer AI-assisted creation and analysis workflows. The test workflow file provides a practical usage example, helping users quickly understand and apply the new features.
Add SiliconCloud VLM API Node, Refactor Model Fetching Logic, Enhance BizyAir Node Library's Multimodal AI Capabilities
This PR introduces support for SiliconFlow's latest VLM (Vision Language Model) API while optimizing existing LLM functionality. Key updates include:
1. New SiliconCloudVLMAPI Node
SiliconCloudVLMAPI
class inllm.py
, supporting image input and text prompts.encode_comfy_image
function to process input images, supporting multiple image inputs.2. Refactored Model Fetching Logic
fetch_all_models
function inllm.py
to retrieve all available models.get_silicon_cloud_llm_models_endpoint
andget_silicon_cloud_vlm_models_endpoint
.3. Updated utils.py
get_vlm_response
function to handle VLM API request logic.4. Frontend JavaScript Updates (js/siliconcloud_llm_api.js)
createModelFetchExtension
function.5. Other Optimizations
NODE_CLASS_MAPPINGS
andNODE_DISPLAY_NAME_MAPPINGS
.6. Test Workflow
This update enables the BizyAir node library to handle more complex multimodal AI tasks. Users can now seamlessly integrate text and image inputs in ComfyUI, achieving richer AI-assisted creation and analysis workflows. The test workflow file provides a practical usage example, helping users quickly understand and apply the new features.
新增SiliconCloud VLM API节点,重构模型获取逻辑,增强BizyAir节点库的多模态AI能力
本次PR主要引入了对SiliconFlow最新VLM(视觉语言模型)API的支持,同时优化了现有的LLM功能。主要更新内容如下:
1. 新增SiliconCloudVLMAPI节点
llm.py
中添加了SiliconCloudVLMAPI
类,支持图像输入和文本提示。encode_comfy_image
函数处理输入图像,支持多图输入。2. 重构模型获取逻辑
llm.py
中新增fetch_all_models
函数,用于获取所有可用模型。get_silicon_cloud_llm_models_endpoint
和get_silicon_cloud_vlm_models_endpoint
。3. 更新utils.py
get_vlm_response
函数,处理VLM API的请求逻辑。4. 前端JavaScript更新(js/siliconcloud_llm_api.js)
createModelFetchExtension
函数。5. 其他优化
NODE_CLASS_MAPPINGS
和NODE_DISPLAY_NAME_MAPPINGS
中添加了新的VLM节点。6. 测试工作流
这次更新使BizyAir节点库能够处理更复杂的多模态AI任务。用户现在可以在ComfyUI中无缝集成文本和图像输入,实现更丰富的AI辅助创作和分析工作流。测试工作流文件提供了一个实际使用示例,有助于用户快速理解和应用新功能。