[BUG] Bad token estimation in Prompty

[ianchi]

Describe the bug Token estimation is wrong, as it doesn't take into account the functions/tools definition as part of the prompt. In cases with heavy use of tools this can make a big diference and make you take a wrong decision regarding the token limit of a request.

How To Reproduce the bug Steps to reproduce the behavior, how frequent can you experience the bug:

1. Create a prompty file with a large tools definition
2. Make an estimation with a small question
3. compare with the actual token consumption reported by oai response

Expected behavior A clear and concise description of what you expected to happen.

• Add an additional function to also estimate the tools definition token consumption

Screenshots If applicable, add screenshots to help explain your problem.

Running Information(please complete the following information):

• Promptflow Package Version using pf -v: [e.g. 0.0.102309906]
• Operating System: [e.g. Ubuntu 20.04, Windows 11]: Debian
• Python Version using python --version: [e.g. python==3.10.12]: 3.11.5

Additional context Add any other context about the problem here.

[ianchi]

Additionally token from messages is also incorrect. It doesn't take into account when an assistant's message has tool_calls

And when using Azure OpenAI the wrong model type may be selected. num_tokens_from_messages expects the model as string using one of OpenAI model's labels, but in this case the azure_deployment is used. This is a user generated label and not necessarily a valid model name, so the token count raises error and can´t be used.

[github-actions[bot]]

Hi, we're sending this friendly reminder because we haven't heard back from you in 30 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 7 days of this comment, the issue will be automatically closed. Thank you!