Issue with Image Uploads Using “remote_url” on Dify Cloud Solution Post v0.10 Release

[nguyenphan]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.10

Cloud or Self Hosted

Cloud

Steps to reproduce

1. Use Dify GUI or API to send a request to a Text Generation app or Workflow app.
2. Use an image file that references a “remote_url” rather than uploading directly to Dify.
3. Observe that the request fails regardless of the app type (e.g., Text Generation, Workflow).

Prior to the release of version 0.10, the same apps worked correctly without any issues. Direct requests to Google Gemini through the Google JS SDK using the same image URL work without problems, suggesting the issue is specific to Dify’s handling of “remote_url” after the update.

These problems do not occur if I use the "Upload from Computer" option.

✔️ Expected Behavior

Image uploads using “remote_url” should work seamlessly as they did prior to version 0.10, allowing apps like Google Gemini Pro and GPT-4o to process requests without errors.

❌ Actual Behavior

• Google Gemini Pro: Returns [google] Server Unavailable Error, 500 An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting. • GPT-4o: Returns [openai] Bad Request Error, Error code: 400 - {'error': {'message': 'Invalid MIME type. Only image types are supported.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_image_format'}}. • Dify GUI: Image uploads consistently fail, as shown in the attached screenshot (image error and network console logs).

This is the error from the GUI:

[dosubot[bot]]

Hey @nguyenphan! 👋 I'm here to help you with any bugs, questions, or contributions you have for Dify Cloud Solution. Let's tackle this issue together!

In Dify version 0.10, the handling of 'remote_url' for image uploads has changed due to the transition from a standalone image upload feature to a more comprehensive "file upload" feature. This change means that the previous method of handling 'remote_url' is now part of a broader file upload process, which supports various file types, including images ^[1][2].

The issues you're experiencing with 'remote_url' uploads could be related to several factors:

1. Content Security Policy (CSP): Ensure that the remote URLs you are using are included in the CSP whitelist. The CSP_WHITELIST environment variable specifies allowed domains, and if your URLs are not included, it could lead to loading issues ^[3].

2. MIME Type Validation: The error message from GPT-4o indicates an invalid MIME type. Ensure that the images you are trying to upload have the correct MIME type that is supported by Dify and the external services you are using.

3. Error Handling: Dify has specific error handling mechanisms for image uploads, such as file size checks and error callbacks. Make sure that the images meet the size requirements and that any errors during the upload process are being handled appropriately ^[4][5].

To resolve these issues, you may need to adjust your configuration settings, ensure that the remote URLs are correctly whitelisted, and verify that the images conform to the expected MIME types.

To continue talking to Dosu, mention @dosu.

[nguyenphan]

For more infos, the image I am using is this https://inkr-upload-global.s3.us-east-1.amazonaws.com/figma-plugin/2024-9-24-Vs5IfUVERbwGb4kYiqpqd.jpg a public available image hosted on s3. I am sorry but I am using your cloud solution. How do I white list the domains? Will it happen on my side? or yours? If mine, then how has it been working prior 0.10?

If the issue is with the image MIME, then please take the image and help point me to how should I fix the data of the image. Because on my side, I can download that image normally without any issue.

FYI, I used this files upload on both Text Generation app and workflow, and we have been using the app for almost a year without any issue.

[dosubot[bot]]

To address the issues you're facing with image uploads using remote_url in Dify Cloud:

1. Domain Whitelisting: For cloud users, domain whitelisting is typically managed by the service provider, not on your side. If the issue is related to Content Security Policy (CSP), you might need to contact the Dify support team to ensure that the domain hosting your image (s3.us-east-1.amazonaws.com) is whitelisted.

2. MIME Type Validation: The error message suggests there might be an issue with the MIME type of the image. You can check the MIME type of your image using tools like file command in Unix-based systems or online MIME type checkers. Ensure that the image is in a supported format (JPG, JPEG, PNG, GIF, WEBP, or SVG) and that the MIME type matches the file extension. If the MIME type is incorrect, you might need to re-encode the image using an image processing tool to ensure it matches the expected format.

If these steps don't resolve the issue, further investigation into the specific error messages and logs from Dify might be necessary to pinpoint the exact cause.

To continue talking to Dosu, mention @dosu.

[nguyenphan]

its detected mime is image/jpeg. So please allow dev to look at this and fix this asap. Given the amount of issue filed post 0.10 regarding file handling, there might be some major hole in the codebase

[laipz8200]

Before v0.10.0, we only supported images. Now, for files using remote_url, we send a HEAD request to this URL and use the returned Content-Type. For the provided URL, S3 returned application/octet-stream.

[nguyenphan]

@laipz8200 so, how should you recommend me to fix this?

[nguyenphan]

@laipz8200 explicitly setting ContentType when upload to s3 will fix this problem.

[laipz8200]

I created a PR to optimize the guessing of file mimetypes, which will make using files via remote_url more stable. However, this will not resolve the issue with file uploads on the web, as that problem is caused by CSP.

[nguyenphan]

@laipz8200 sure, thanks for the follow up, you can reopen the issue if you want, it is true that the website issue is still there.

[nguyenphan]

I created a PR to optimize the guessing of file mimetypes, which will make using files via remote_url more stable. However, this will not resolve the issue with file uploads on the web, as that problem is caused by CSP.

hopefully you guys can update this flow, since all of the images I upload to s3 were not explicitly set with ContentType...