langgenius / dify

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.
https://dify.ai
Other
53.25k stars 7.77k forks source link

The ComfyUI tool cannot process prompt words containing double quotation marks or carriage returns. #10298

Closed majestichou closed 3 weeks ago

majestichou commented 3 weeks ago

Self Checks

Dify version

0.10.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

The ComfyUI tool is used to build an application. The yaml of this applicaiton is as follows:

app:
  description: ''
  icon: 🤖
  icon_background: '#FFEAD5'
  mode: advanced-chat
  name: 文生图--异常版本
  use_icon_as_answer_icon: false
kind: app
version: 0.1.2
workflow:
  conversation_variables: []
  environment_variables: []
  features:
    file_upload:
      allowed_file_extensions:
      - .JPG
      - .JPEG
      - .PNG
      - .GIF
      - .WEBP
      - .SVG
      allowed_file_types:
      - image
      allowed_file_upload_methods:
      - local_file
      - remote_url
      enabled: false
      fileUploadConfig:
        audio_file_size_limit: 50
        batch_count_limit: 5
        file_size_limit: 15
        image_file_size_limit: 10
        video_file_size_limit: 100
      image:
        enabled: false
        number_limits: 3
        transfer_methods:
        - local_file
        - remote_url
      number_limits: 3
    opening_statement: ''
    retriever_resource:
      enabled: true
    sensitive_word_avoidance:
      enabled: false
    speech_to_text:
      enabled: false
    suggested_questions: []
    suggested_questions_after_answer:
      enabled: false
    text_to_speech:
      enabled: false
      language: ''
      voice: ''
  graph:
    edges:
    - data:
        isInIteration: false
        sourceType: tool
        targetType: answer
      id: 1730706119831-source-answer-target
      selected: false
      source: '1730706119831'
      sourceHandle: source
      target: answer
      targetHandle: target
      type: custom
      zIndex: 0
    - data:
        isInIteration: false
        sourceType: start
        targetType: tool
      id: 1730376121968-source-1730706119831-target
      source: '1730376121968'
      sourceHandle: source
      target: '1730706119831'
      targetHandle: target
      type: custom
      zIndex: 0
    nodes:
    - data:
        desc: ''
        selected: false
        title: 开始
        type: start
        variables: []
      height: 54
      id: '1730376121968'
      position:
        x: 930
        y: 281
      positionAbsolute:
        x: 930
        y: 281
      selected: false
      sourcePosition: right
      targetPosition: left
      type: custom
      width: 244
    - data:
        answer: '{{#1730706119831.files#}}'
        desc: ''
        selected: false
        title: 直接回复
        type: answer
        variables: []
      height: 103
      id: answer
      position:
        x: 1568
        y: 281
      positionAbsolute:
        x: 1568
        y: 281
      selected: false
      sourcePosition: right
      targetPosition: left
      type: custom
      width: 244
    - data:
        desc: ''
        provider_id: comfyui
        provider_name: comfyui
        provider_type: builtin
        selected: false
        title: 工作流
        tool_configurations:
          image_ids: null
          workflow_json: '{   "6": {     "inputs": {       "text": "{{positive_prompt}}",       "clip":
            [         "11",         0       ]     },     "class_type": "CLIPTextEncode",     "_meta":
            {       "title": "CLIP Text Encode (Positive Prompt)"     }   },   "8":
            {     "inputs": {       "samples": [         "13",         0       ],       "vae":
            [         "10",         0       ]     },     "class_type": "VAEDecode",     "_meta":
            {       "title": "VAE Decode"     }   },   "9": {     "inputs": {       "filename_prefix":
            "ComfyUI",       "images": [         "8",         0       ]     },     "class_type":
            "SaveImage",     "_meta": {       "title": "Save Image"     }   },   "10":
            {     "inputs": {       "vae_name": "ae.safetensors"     },     "class_type":
            "VAELoader",     "_meta": {       "title": "Load VAE"     }   },   "11":
            {     "inputs": {       "clip_name1": "t5xxl_fp16.safetensors",       "clip_name2":
            "clip_l.safetensors",       "type": "flux"     },     "class_type": "DualCLIPLoader",     "_meta":
            {       "title": "DualCLIPLoader"     }   },   "12": {     "inputs": {       "unet_name":
            "flux1-dev.safetensors",       "weight_dtype": "default"     },     "class_type":
            "UNETLoader",     "_meta": {       "title": "Load Diffusion Model"     }   },   "13":
            {     "inputs": {       "noise": [         "25",         0       ],       "guider":
            [         "22",         0       ],       "sampler": [         "16",         0       ],       "sigmas":
            [         "17",         0       ],       "latent_image": [         "27",         0       ]     },     "class_type":
            "SamplerCustomAdvanced",     "_meta": {       "title": "SamplerCustomAdvanced"     }   },   "16":
            {     "inputs": {       "sampler_name": "euler"     },     "class_type":
            "KSamplerSelect",     "_meta": {       "title": "KSamplerSelect"     }   },   "17":
            {     "inputs": {       "scheduler": "simple",       "steps": 20,       "denoise":
            1,       "model": [         "30",         0       ]     },     "class_type":
            "BasicScheduler",     "_meta": {       "title": "BasicScheduler"     }   },   "22":
            {     "inputs": {       "model": [         "30",         0       ],       "conditioning":
            [         "26",         0       ]     },     "class_type": "BasicGuider",     "_meta":
            {       "title": "BasicGuider"     }   },   "25": {     "inputs": {       "noise_seed":
            949997509248481     },     "class_type": "RandomNoise",     "_meta": {       "title":
            "RandomNoise"     }   },   "26": {     "inputs": {       "guidance": 3.5,       "conditioning":
            [         "6",         0       ]     },     "class_type": "FluxGuidance",     "_meta":
            {       "title": "FluxGuidance"     }   },   "27": {     "inputs": {       "width":
            1024,       "height": 1024,       "batch_size": 1     },     "class_type":
            "EmptySD3LatentImage",     "_meta": {       "title": "EmptySD3LatentImage"     }   },   "30":
            {     "inputs": {       "max_shift": 1.15,       "base_shift": 0.5,       "width":
            1024,       "height": 1024,       "model": [         "12",         0       ]     },     "class_type":
            "ModelSamplingFlux",     "_meta": {       "title": "ModelSamplingFlux"     }   }
            }'
        tool_label: 工作流
        tool_name: workflow
        tool_parameters:
          negative_prompt:
            type: mixed
            value: ''
          positive_prompt:
            type: mixed
            value: '{{#sys.query#}}'
        type: tool
      height: 116
      id: '1730706119831'
      position:
        x: 1256.0000000000002
        y: 281
      positionAbsolute:
        x: 1256.0000000000002
        y: 281
      selected: true
      sourcePosition: right
      targetPosition: left
      type: custom
      width: 244
    viewport:
      x: -371.8135330515688
      y: -1.080979998461487
      zoom: 1.0000000000000002

When I type in the usual prompt words, for example, draw a puppy, the workflow generates the picture normally.

But when I type a prompt word with double quotation marks or carriage return line feeds, everything becomes abnormal and the app doesn't produce pictures properly.

The following is an example prompt word: A blackboard that says "Hello World." The tracing log is as follows: image

The following is another example prompt word:

On a grassy area, the children were playing happily.
On the other grass, adults are playing golf.

The tracing log is as follows: image

✔️ Expected Behavior

The ComfyUI tool can process prompt words containing double quotation marks or carriage returns.

❌ Actual Behavior

The ComfyUI tool can not process prompt words containing double quotation marks or carriage returns.

dosubot[bot] commented 3 weeks ago

The issue you're experiencing with the ComfyUI tool not processing prompt words containing double quotation marks or carriage returns might be related to how the input is parsed or handled within the tool. Unfortunately, I couldn't find specific information on how ComfyUI handles special characters like double quotation marks and carriage returns. However, here are a few general suggestions that might help:

  1. Escape Special Characters: Try escaping double quotation marks in your prompt by using a backslash (\"). This might help if the issue is related to JSON parsing or string handling.

  2. Remove Carriage Returns: If possible, try to format your input without carriage returns. You can replace them with spaces or other delimiters that don't affect the meaning of your prompt.

  3. Check Input Handling: Review the code or configuration where the input is processed to ensure that it correctly handles special characters. This might involve modifying the input parsing logic to accommodate these characters.

  4. Update or Patch: Ensure that you are using the latest version of the ComfyUI tool, as updates might include fixes for such issues.

If these suggestions don't resolve the issue, you might need to look into the specific implementation details of the ComfyUI tool to identify where the input handling might be failing.

To continue talking to Dosu, mention @dosu.