MC-E / DragonDiffusion

ICLR 2024 (Spotlight)
Apache License 2.0
695 stars 21 forks source link

Notes on installation bugs + problem with Resize in Object Pasting #18

Open legel opened 5 months ago

legel commented 5 months ago

Thanks for the great research and for releasing the code.

Unfortunately the code was difficult to install, at least on an Ubuntu Linux machine (e.g. bugs relating to the very old versions being used here, a bug relating to a line in basicsr which did not have a version... I recommend trying to do an install on a "fresh" Linux machine to confirm.)

At first I was getting this error:

Traceback (most recent call last):
  File "/home/photon/miniconda3/envs/diffedit/lib/python3.10/site-packages/gradio/queueing.py", line 407, in call_prediction
    output = await route_utils.call_process_api(
  File "/home/photon/miniconda3/envs/diffedit/lib/python3.10/site-packages/gradio/route_utils.py", line 226, in call_process_api
    output = await app.get_blocks().process_api(
  File "/home/photon/miniconda3/envs/diffedit/lib/python3.10/site-packages/gradio/blocks.py", line 1550, in process_api
    result = await self.call_function(
  File "/home/photon/miniconda3/envs/diffedit/lib/python3.10/site-packages/gradio/blocks.py", line 1185, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/photon/miniconda3/envs/diffedit/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/photon/miniconda3/envs/diffedit/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2134, in run_sync_in_worker_thread
    return await future
  File "/home/photon/miniconda3/envs/diffedit/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "/home/photon/miniconda3/envs/diffedit/lib/python3.10/site-packages/gradio/utils.py", line 661, in wrapper
    response = f(*args, **kwargs)
  File "/home/photon/sense/3cology/research/DragonDiffusion/src/demo/model.py", line 137, in run_appearance
    img_base, input_scale = resize_numpy_image(img_base, max_resolution*max_resolution)
  File "/home/photon/sense/3cology/research/DragonDiffusion/src/utils/utils.py", line 8, in resize_numpy_image
    h, w = image.shape[:2]
AttributeError: 'NoneType' object has no attribute 'shape'

It was not clear to me that the user is required to create bounding boxes on the images in Gradio. (Mentioning here for others.)

After I did that, the system is at least running, but I'm seeing a very strange result with the Resize object param.

Things work properly for object pasting with default Resize param = 1: Screenshot from 2024-02-08 14-41-10

But, with everything else completely the same, when I change to Resize param = 0.4, something seems to break with the stable diffusion: Screenshot from 2024-02-08 14-39-43

The degradation seems to be somewhere in the middle you can see for Resize param = 0.8: Screenshot from 2024-02-08 14-33-02

Any ideas?

MC-E commented 5 months ago

Thanks for your attention. Yes, the mask is needed. Could you provide the examples you use? I will check the quality (when Resize!=1).

legel commented 5 months ago

Sure, here are the photos of the wood table and house. Would love to discover that something on my end is not configured properly!

house wood_table