search

langgenius / dify

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.
https://dify.ai
Other
50.14k stars 7.18k forks source link

wrong max base model token limit when using gpt-4 turbo #1698

Closed Ps7ch3 closed 10 months ago

Ps7ch3 commented 11 months ago

Dify version

0.3.32

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Long user input message or prompt with Azure GPT-4 (1106-preview)

✔️ Expected Behavior

Add extra model option for Azure GPT-4 turbo

❌ Actual Behavior

image

Ps7ch3 commented 11 months ago

I believe the root cause is that Azure gpt-4 turbo did not assign a new model name and only updated the model verison to "1106-preview"

See docs for details

And current gpt-4 base model max token is 8K

image
crazywoola commented 11 months ago

This is a known issue. Because the gpt-4-preview has a 

Input: 128,000
Output: 4096

But we haven't split them into two parts in the backend code.

Ps7ch3 commented 11 months ago

This is a known issue. Because the gpt-4-preview has a

Input: 128,000
Output: 4096

But we haven't split them into two parts in the backend code.

I simply caught the raised exception as a temp workaround

Any plan to add 'gpt-4-turbo' option to the base model selection or custom max model token limit in the future?

crazywoola commented 11 months ago

Yes.