[gradio] cuda not release gpu memory?

[t00350320]

diffusers 0.28.0.dev0

first time runs well, the second runtime changed the content scale to 2 , CUDA out of memory errors. 1) guess the content scale will be transfered to float ? 2) gradio will not release gpu after the first error :

  File "/home/notebook/code/personal/CSGO/gradio/app.py", line 172, in create_image
    images = csgo.generate(pil_content_image=content_image, pil_style_image=style_image,
  File "/home/notebook/code/personal/CSGO/./ip_adapter/ip_adapter.py", line 735, in generate
    images = self.pipe(
  File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/notebook/code/group/diffusers/src/diffusers/pipelines/controlnet/pipeline_controlnet_sd_xl.py", line 1235, in __call__
    self.check_inputs(
  File "/home/notebook/code/group/diffusers/src/diffusers/pipelines/controlnet/pipeline_controlnet_sd_xl.py", line 753, in check_inputs
    raise TypeError("For single controlnet: `controlnet_conditioning_scale` must be type `float`.")
TypeError: For single controlnet: `controlnet_conditioning_scale` must be type `float`.

    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 26.00 MiB (GPU 0; 31.75 GiB total capacity; 28.56 GiB already allocated; 8.75 MiB free; 29.92 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

PTAL

[xingp-ng]

diffusers 0.28.0.dev0

first time runs well, the second runtime changed the content scale to 2 , CUDA out of memory errors.

1. guess the content scale will be transfered to float ?
2. gradio will not release gpu after the first error :

  File "/home/notebook/code/personal/CSGO/gradio/app.py", line 172, in create_image
    images = csgo.generate(pil_content_image=content_image, pil_style_image=style_image,
  File "/home/notebook/code/personal/CSGO/./ip_adapter/ip_adapter.py", line 735, in generate
    images = self.pipe(
  File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/notebook/code/group/diffusers/src/diffusers/pipelines/controlnet/pipeline_controlnet_sd_xl.py", line 1235, in __call__
    self.check_inputs(
  File "/home/notebook/code/group/diffusers/src/diffusers/pipelines/controlnet/pipeline_controlnet_sd_xl.py", line 753, in check_inputs
    raise TypeError("For single controlnet: `controlnet_conditioning_scale` must be type `float`.")
TypeError: For single controlnet: `controlnet_conditioning_scale` must be type `float`.

    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 26.00 MiB (GPU 0; 31.75 GiB total capacity; 28.56 GiB already allocated; 8.75 MiB free; 29.92 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

PTAL

controlnet_conditioning_scale should be of type float like the controlNet model parameter.

[t00350320]

diffusers 0.28.0.dev0 first time runs well, the second runtime changed the content scale to 2 , CUDA out of memory errors.

1. guess the content scale will be transfered to float ?
2. gradio will not release gpu after the first error :

  File "/home/notebook/code/personal/CSGO/gradio/app.py", line 172, in create_image
    images = csgo.generate(pil_content_image=content_image, pil_style_image=style_image,
  File "/home/notebook/code/personal/CSGO/./ip_adapter/ip_adapter.py", line 735, in generate
    images = self.pipe(
  File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/notebook/code/group/diffusers/src/diffusers/pipelines/controlnet/pipeline_controlnet_sd_xl.py", line 1235, in __call__
    self.check_inputs(
  File "/home/notebook/code/group/diffusers/src/diffusers/pipelines/controlnet/pipeline_controlnet_sd_xl.py", line 753, in check_inputs
    raise TypeError("For single controlnet: `controlnet_conditioning_scale` must be type `float`.")
TypeError: For single controlnet: `controlnet_conditioning_scale` must be type `float`.

    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 26.00 MiB (GPU 0; 31.75 GiB total capacity; 28.56 GiB already allocated; 8.75 MiB free; 29.92 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

PTAL

controlnet_conditioning_scale should be of type float like the controlNet model parameter.

it's sad that, even i input 2.0 , or modify scale_c = float(scale_c), the error also comes. i have no idea why this happen.

[xingp-ng]

diffusers 0.28.0.dev0 first time runs well, the second runtime changed the content scale to 2 , CUDA out of memory errors.

1. guess the content scale will be transfered to float ?
2. gradio will not release gpu after the first error :

  File "/home/notebook/code/personal/CSGO/gradio/app.py", line 172, in create_image
    images = csgo.generate(pil_content_image=content_image, pil_style_image=style_image,
  File "/home/notebook/code/personal/CSGO/./ip_adapter/ip_adapter.py", line 735, in generate
    images = self.pipe(
  File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/notebook/code/group/diffusers/src/diffusers/pipelines/controlnet/pipeline_controlnet_sd_xl.py", line 1235, in __call__
    self.check_inputs(
  File "/home/notebook/code/group/diffusers/src/diffusers/pipelines/controlnet/pipeline_controlnet_sd_xl.py", line 753, in check_inputs
    raise TypeError("For single controlnet: `controlnet_conditioning_scale` must be type `float`.")
TypeError: For single controlnet: `controlnet_conditioning_scale` must be type `float`.

    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 26.00 MiB (GPU 0; 31.75 GiB total capacity; 28.56 GiB already allocated; 8.75 MiB free; 29.92 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

PTAL

controlnet_conditioning_scale should be of type float like the controlNet model parameter.

it's sad that, even i input 2.0 , or modify scale_c = float(scale_c), the error also comes. i have no idea why this happen.

The maximum value of content_C in Gradio is 2.0. However, in our experience, it is not recommended to set it higher than 1.2, which can result in style information not being injected. Is it ok to set the rest of the values please?

[t00350320]

diffusers 0.28.0.dev0 first time runs well, the second runtime changed the content scale to 2 , CUDA out of memory errors.

1. guess the content scale will be transfered to float ?
2. gradio will not release gpu after the first error :

  File "/home/notebook/code/personal/CSGO/gradio/app.py", line 172, in create_image
    images = csgo.generate(pil_content_image=content_image, pil_style_image=style_image,
  File "/home/notebook/code/personal/CSGO/./ip_adapter/ip_adapter.py", line 735, in generate
    images = self.pipe(
  File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/notebook/code/group/diffusers/src/diffusers/pipelines/controlnet/pipeline_controlnet_sd_xl.py", line 1235, in __call__
    self.check_inputs(
  File "/home/notebook/code/group/diffusers/src/diffusers/pipelines/controlnet/pipeline_controlnet_sd_xl.py", line 753, in check_inputs
    raise TypeError("For single controlnet: `controlnet_conditioning_scale` must be type `float`.")
TypeError: For single controlnet: `controlnet_conditioning_scale` must be type `float`.

    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 26.00 MiB (GPU 0; 31.75 GiB total capacity; 28.56 GiB already allocated; 8.75 MiB free; 29.92 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

PTAL

controlnet_conditioning_scale should be of type float like the controlNet model parameter.

it's sad that, even i input 2.0 , or modify scale_c = float(scale_c), the error also comes. i have no idea why this happen.

The maximum value of content_C in Gradio is 2.0. However, in our experience, it is not recommended to set it higher than 1.2, which can result in style information not being injected. Is it ok to set the rest of the values please?

your gradio app.py

content_scale=1.0,
controlnet_conditioning_scale=scale_c,

so i suppose the controlnet_conditioning_scale should be configured below 1, but it's overwrited by content_scale? feel confused with these two parameters. I think you should put two parameters content_scale，controlnet_conditioning_scale respectively in gradio ui?

[xingp-ng]

Yes could cause confusion. This will be changed in the next version. controlnet_conditioning_scale will not be overridden by content_scale. This is only written so that people understand that scale_c affects content control and scale_s affects style control.

[xingp-ng]

Yes could cause confusion. This will be changed in the next version. controlnet_conditioning_scale will not be overridden by content_scale. This is only written so that people understand that scale_c affects content control and scale_s affects style control.

fixed.