search

siliconflow / BizyAir

BizyAir: Comfy Nodes that can run in any environment.
https://siliconflow.github.io/BizyAir/
MIT License
390 stars 20 forks source link

[FEATURE REQUEST] Support for Flux.1 ControlNet node and Lora in BizyAir #184

Open syl1130 opened 1 month ago

syl1130 commented 1 month ago

Hello @doombeaker , I request to add the following enhanced features and support for models such as nodes. I really like BizyAir and am actively promoting it to more users. I hope it will continue to improve.

ControlNet

FLUX.1-dev ControlNet [XLabs Canny] Repository

FLUX.1-dev ControlNet [XLabs depth] Repository

Inpaint (Fill, Expand, Add/Remove Object) FLUX.1-dev ControlNet Inpainting - Beta Repository

Our latest inpainting model brings significant improvements compared to the previous version:

1024 Resolution Support: Capable of directly processing and generating 1024x1024 resolution images without additional upscaling steps, providing higher quality and more detailed output results. Enhanced Detail Generation: Fine-tuned to capture and reproduce finer details in inpainted areas. Improved Prompt Control: Offers more precise control over generated content through enhanced prompt interpretation.

[FLUX.1-Turbo-Alpha] 8-step Lora Repository

Improve image efficiency and increase speed by 50%:

This checkpoint is a 8-step distilled Lora, trained based on FLUX.1-dev model. We use a multi-head discriminator to improve the distill quality. Our model can be used for T2I, inpainting controlnet and other FLUX related models. The recommended guidance_scale=3.5 and lora_scale=1.

Flux IP-Adapter Stylization Repository

It can also be used to mix or style images (although I'm still working out the settings for this)

doombeaker commented 1 month ago

You can upload the loras and use it by you own, please refer to https://siliconflow.github.io/BizyAir/model-host/introduce.html to get details.

flux controlnet will be supported in a few weeks, you can upload your own controlnet and use it by BizyAir then.

RF Inversion is not on the to do list so far.

Thank you for your advice!

syl1130 commented 1 month ago

thank you for your support

jasoncow007 commented 2 weeks ago

You can upload the loras and use it by you own, please refer to https://siliconflow.github.io/BizyAir/model-host/introduce.html to get details.

flux controlnet will be supported in a few weeks, you can upload your own controlnet and use it by BizyAir then.

I tried the controlnet apply node works well with samplercustomadvanced, but get error while using Ksampler

Unexpected error accessing result["data"]["payload"]. Result: {'type': 'error', 'message': 'Error processing task e5b421cd-d6d8-4057-8dcb-36bc40ab5904: Exceeded limit for instance UNETLoader count <= 0.', 'details': 'Traceback (most recent call last):\n File "/workspace/comfybridge/comfybridge/bizyair/schedulers.py", line 287, in process_request\n result = await self._execute_task(\n File "/workspace/comfybridge/comfybridge/bizyair/schedulers.py", line 341, in _execute_task\n return await loop.run_in_executor(self.pool, executor.execute)\n File "/opt/conda/lib/python3.10/concurrent/futures/thread.py", line 58, in run\n result = self.fn(*self.args, self.kwargs)\n File "/workspace/comfybridge/comfybridge/bizyair/schedulers.py", line 80, in execute\n return self.execute_func(*self.args, *self.kwargs)\n File "/workspace/comfybridge/comfybridge/plugins/comfy_pipeline/utils/cuda_memory_utils.py", line 22, in wrapper\n result = func(args, kwargs)\n File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context\n return func(*args, **kwargs)\n File "/workspace/comfybridge/comfybridge/plugins/comfy_pipeline/server.py", line 124, in run\n user_params: ServerFormat = convert_to_server_format(\n File "/workspace/comfybridge/comfybridge/plugins/comfy_pipeline/transpiler/transpiler.py", line 81, in convert_to_server_format\n prompt, node_id_registry = allocator.generate_workflow_with_allocated_ids(\n File "/workspace/comfybridge/comfybridge/plugins/comfy_pipeline/transpiler/node_id_allocator.py", line 105, in generate_workflow_with_allocated_ids\n raise RuntimeError(\nRuntimeError: Exceeded limit for instance UNETLoader count <= 0\n', 'extra_info': None, 'data_status': 'COMPLETED', 'start_time': 1731129655.359425, 'end_time': 1731129655.4180846, 'task_id': 'e5b421cd-d6d8-4057-8dcb-36bc40ab5904', 'upload_to_s3': None}