Severe problem in OpenAI model (gpt-4o)

[QichangZheng]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.7.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I have just switched from FastGPT to Dify. I developed an application and use API to request it. I configured OpenAI and Ollama models. When I use Ollama local models, my OpenAI tokens were consumed! I am sure that I published the model and updated the configuration. In this case, I disabled my OpenAI apikey, and I can still use the application. Moreover, in development, for OpenAI models, I only used gpt-4o-mini, and never used gpt-4o, but it showed on the OpenAI offical API board that I consumed gpt-4o tokens instead of gpt-4o-mini.

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

[dosubot[bot]]

@QichangZheng Hello! I'm here to assist you with any bugs, questions, or contributions. Let's work together to resolve the issue you're experiencing.

I found some similar discussions and issues that might be relevant to your problem:

1. I cannot find the GPT-4o-mini model listed in the model settings after adding my OpenAI API key - This discussion mentions that the GPT-4o-mini model is not supported in the Docker version of Dify ^[1].
2. Community version not support gpt 4o min - This discussion also highlights issues with the GPT-4o-mini model in the community version ^[2].
3. GPT-4o-mini - Another discussion about the GPT-4o-mini model ^[3].

For your specific issue, it seems like there might be a configuration problem causing the OpenAI tokens to be consumed even when using Ollama local models. Here are a few steps you can take to troubleshoot and potentially resolve the issue:

1. Check Model Configuration:

   • Ensure that the model providers are correctly configured in the Settings --> Model Providers section. Verify that the API keys and endpoints are correctly set for both OpenAI and Ollama models.

2. API Endpoint Configuration:

   • For Docker setups, make sure to use host.docker.internal to connect to the host machine. For example, if you are using LMStudio to provide local access to a model, the API address should be something like http://host.docker.internal:1234/v1 ^[1].

3. Token Consumption Monitoring:

   • Use Dify's logging and debugging tools to trace and diagnose token consumption. Integrating tools like LangSmith and Langfuse can help provide comprehensive tracking and deep evaluation capabilities ^[4][5].

4. Disable OpenAI API Key:

   • If disabling the OpenAI API key still allows the application to function, it suggests that the local model configuration might not be correctly isolated. Double-check the model selection logic in your application to ensure it is explicitly using the Ollama models.

If these steps do not resolve the issue, you might want to look into the specific logs and debugging information provided by Dify to get more insights into why the OpenAI tokens are being consumed.

To continue talking to Dosu, mention @dosu.

[crazywoola]

[QichangZheng]

But when I disabled my OpenAI apikey, my Dify application still works well. Here is my application structure: