[Question]: NotImplementedError: oneflow.nn.functional.scaled_dot_product_attention is not implemented, please submit an issue at 'https://github.com/Oneflow-Inc/oneflow/issues' including the log information of the error, the minimum reproduction code, and the system information.

[chenmeng666chen]

Description

Traceback (most recent call last): File "worker_prod.py", line 3, in from app import viewer File "/opt/app/app/init.py", line 5, in from . import api File "/opt/app/app/api.py", line 74, in controlnet = ControlNetModel.from_pretrained("/opt/app/sd-controlnet-canny", torch_dtype=flow.float16) File "/opt/conda/lib/python3.8/site-packages/diffusers/models/modeling_utils.py", line 610, in from_pretrained model = cls.from_config(config, unused_kwargs) File "/opt/conda/lib/python3.8/site-packages/diffusers/configuration_utils.py", line 210, in from_config model = cls(init_dict) File "/opt/conda/lib/python3.8/site-packages/diffusers/configuration_utils.py", line 569, in inner_init init(self, *args, *init_kwargs) File "/opt/conda/lib/python3.8/site-packages/diffusers/models/controlnet.py", line 209, in init down_block = get_down_block( File "/opt/conda/lib/python3.8/site-packages/diffusers/models/unet_2d_blocks.py", line 89, in get_down_block return CrossAttnDownBlock2D( File "/opt/conda/lib/python3.8/site-packages/diffusers/models/unet_2d_blocks.py", line 771, in init Transformer2DModel( File "/opt/conda/lib/python3.8/site-packages/diffusers/configuration_utils.py", line 569, in inner_init init(self, args, **init_kwargs) File "/opt/conda/lib/python3.8/site-packages/diffusers/models/transformer_2d.py", line 179, in init [ File "/opt/conda/lib/python3.8/site-packages/diffusers/models/transformer_2d.py", line 180, in BasicTransformerBlock( File "/opt/conda/lib/python3.8/site-packages/diffusers/models/attention.py", line 223, in init self.attn1 = CrossAttention( File "/opt/conda/lib/python3.8/site-packages/diffusers/models/cross_attention.py", line 105, in init processor = AttnProcessor2_0() if hasattr(F, "scaled_dot_product_attention") else CrossAttnProcessor() File "/opt/conda/lib/python3.8/site-packages/oneflow/mock_torch/init.py", line 42, in getattr raise NotImplementedError(self.module.name + "." + name + error_msg) NotImplementedError: oneflow.nn.functional.scaled_dot_product_attention is not implemented, please submit an issue at 'https://github.com/Oneflow-Inc/oneflow/issues' including the log information of the error, the minimum reproduction code, and the system information.

Alternatives

No response

[doombeaker]

controlnet seems cannot be run by mock_torch recently. But as I know, it has been supported in oneflow forked diffusers repository, see issue https://github.com/Oneflow-Inc/diffusers/issues/127 . You can try it as the comment in issue 127 described.