search

langgenius / dify

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.
https://dify.ai
Other
48.71k stars 6.98k forks source link

Add ComfyUI workflow api #2927

Closed vindia9 closed 7 months ago

vindia9 commented 7 months ago

Self Checks

1. Is this request related to a challenge you're experiencing?

I am trying to use LLM in conjunction with controlling image generation, for which I need to use Stable Diffusion. However, the Stable Diffusion WebUI does not offer enough flexibility and cannot facilitate high degrees of creative freedom through workflows like Stable Diffusion ComfyUI can.

2. Describe the feature you'd like to see

Add API for Stable Diffusion ComfyUI in the Tools - Stable Diffusion.

3. How will this feature improve your workflow or experience?

I can create an assistant based on Stable Diffusion ComfyUI that uses JSON-formatted workflows for calls, allowing for image generation with a higher degree of freedom.

4. Additional context or comments

Stable Diffusion ComfyUI API Example https://github.com/comfyanonymous/ComfyUI/blob/master/script_examples/basic_api_example.py

5. Can you help us with this feature?

dosubot[bot] commented 7 months ago

Thank you for your suggestion. Adding an API for Stable Diffusion ComfyUI to the Tools - Stable Diffusion in the repository could indeed enhance the creative freedom for image generation. This could be achieved by extending the existing YAML configuration to include API endpoint definitions, request, and response formats.

Currently, the YAML file defines parameters for image generation, such as prompt, model, lora, steps, width, height, and negative_prompt. To enable API functionality, additional fields for API endpoint configuration, HTTP method, request body schema, and response handling would need to be introduced. This would allow the tool to interact with Stable Diffusion ComfyUI through HTTP requests, passing parameters in a JSON format and receiving generated images or related metadata in response. You can refer to the current YAML configuration here.

As for the current structure of the Tools - Stable Diffusion, it is defined in the StableDiffusionProvider class in the stablediffusion.py file. The class includes a method _validate_credentials that validates the credentials for the Stable Diffusion tool. You can refer to the current structure here.

However, without explicit endpoint information, it's not possible to detail how these parameters are used in API requests or what the response format would be. Typically, in an API, you would expect to see endpoints described along with HTTP methods (e.g., GET, POST) and examples of request and response bodies. This information is not present in the provided YAML configuration.

Unfortunately, I couldn't find information about the current functionality of the Stable Diffusion ComfyUI and its differences from the Stable Diffusion WebUI, or the current version of the Stable Diffusion ComfyUI API and any upcoming updates or changes that could affect its integration with the repository.

If you could provide more details about the Stable Diffusion ComfyUI API, such as the key endpoints and their respective request and response formats, or any upcoming updates or changes, it would be very helpful for further discussion on this feature request.

Sources

#### About Dosu This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.
crazywoola commented 7 months ago

This is in alpha test right now.