can't run img2img sdxl turbo: 'MODULE' 'ProxyModule' object 'model.down_blocks.1.attentions.0' in nn.Graph has no attribute

[noah1320]

Describe the bug

A clear and concise description of what the bug is.

Your environment

Colab (Linux)

OS

OneDiff git commit id

OneFlow version info

path: ['/usr/local/lib/python3.10/dist-packages/oneflow']
version: 0.9.1.dev20240515+cu118
git_commit: ec7b682
cmake_build_type: Release
rdma: True
mlir: True
enterprise: False

How To Reproduce

pipe = AutoPipelineForImage2Image.from_pretrained("stabilityai/sdxl-turbo", torch_dtype=torch.float16, variant="fp16")
pipe.unet = oneflow_compile(pipe.unet)
pipe.vae = AutoencoderTiny.from_pretrained("madebyollin/taesdxl", torch_dtype=torch.float16)
pipe.to("cuda")
img1 = np.zeros((512, 512, 3) , dtype=np.uint8)
img1 = Image.fromarray(img1)
img2 = pipe(prompt, image=image, num_inference_steps=i, strength=strength, guidance_scale=1).images[0]

The complete error message

ERROR:onediff.utils.log_utils:Exception in forward: e=AttributeError("'MODULE' 'ProxyModule' object 'model.down_blocks.1.attentions.0' in nn.Graph has no attribute 'caption_projection'")
WARNING:onediff.utils.log_utils:Recompile oneflow module ...
[ERROR](GRAPH:OneflowGraph_1:OneflowGraph) building graph got error.
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/onediff/infer_compiler/backends/oneflow/deployable_module.py](https://localhost:8080/#) in wrapper(self, *args, **kwargs)
     34             try:
---> 35                 return func(self, *args, **kwargs)
     36             except Exception as e:

48 frames
AttributeError: 'MODULE' 'ProxyModule' object 'model.down_blocks.1.attentions.0' in nn.Graph has no attribute 'caption_projection'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/oneflow/nn/graph/proxy.py](https://localhost:8080/#) in __getattr__(self, name)
    512                 attr = types.MethodType(attr.__func__, self)
    513             return attr
--> 514         raise AttributeError(
    515             "'{}' '{}' object '{}' in nn.Graph has no attribute '{}'".format(
    516                 self.to(GraphModule)._type,

AttributeError: 'MODULE' 'ProxyModule' object 'model.down_blocks.1.attentions.0' in nn.Graph has no attribute 'caption_projection'
### Additional context

Add any other context about the problem here.

[strint]

@noah1320 what is your diffusers version?

[strint]

I can successfully run this. Here is my:

env

• hf diffusers version: 0.26.0
• onediff: main branch, commit e8978c91eb228001e2c1a39b48cb47e00280e78a
• oneflow: 0.9.1+cu122.git.665bcf8 ( python3 -m oneflow --doctor)
• device: NVIDIA A100-PCIE-40GB

code

import torch
from diffusers import AutoPipelineForImage2Image, AutoencoderTiny
from onediff.infer_compiler import oneflow_compile
from PIL import Image
import numpy as np

pipe = AutoPipelineForImage2Image.from_pretrained("tabilityai/sdxl-turbo", torch_dtype=torch.float16, variant="fp16")
pipe.unet = oneflow_compile(pipe.unet)
pipe.vae = AutoencoderTiny.from_pretrained("madebyollin/taesdxl", torch_dtype=torch.float16)
pipe.to("cuda")

img1 = np.zeros((512, 512, 3) , dtype=np.uint8)
img1 = Image.fromarray(img1)
prompt = "A photo of a cat"
# warmup
img2 = pipe(prompt, image=img1, num_inference_steps=8, strength=0.5, guidance_scale=1).images[0]

# run
img2 = pipe(prompt, image=img1, num_inference_steps=8, strength=0.5, guidance_scale=1).images[0]

speed

# warmup
100%|██████████████████| 4/4 [00:44<00:00, 11.07s/it]
# run
100%|██████████████████| 4/4 [00:00<00:00, 56.41it/s]

[strint]

@noah1320

[noah1320]

I was using 0.19.3

[strint]

I was using 0.19.3

this is no AutoencoderTiny in 0.19.3 ?

ImportError: cannot import name 'AutoencoderTiny' from 'diffusers' 

You can update diffusers and have a try

pip install diffusers==0.26.0

[strint]

@noah1320