Error get_num_tokens_from_messages() with gpt-3.5-turbo-0613

Zabrell commented 1 year ago

System Info

Windows(server on linux), python, poetry

Who can help?

@hwchase17 @agola11

Information

[X] The official example notebooks/scripts
[ ] My own modified scripts

Related Components

[X] LLMs/Chat Models
[ ] Embedding Models
[ ] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[ ] Document Loaders
[ ] Vector Stores / Retrievers
[X] Memory
[ ] Agents / Agent Executors
[ ] Tools / Toolkits
[X] Chains
[ ] Callbacks/Tracing
[ ] Async

Reproduction

Langchain has problems with working with model casts

 File "C:\Users\zabre\PycharmProjects\sales-bot\bot\src\sales_bot\text_handler.py", line 445, in get_response
    memory.prune()
  File "C:\Users\zabre\AppData\Local\pypoetry\Cache\virtualenvs\sales-bot-f2_0xw7_-py3.11\Lib\site-packages\langchain\memory\summary_buffer.py", line 65, in prune
    curr_buffer_length = self.llm.get_num_tokens_from_messages(buffer)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\zabre\AppData\Local\pypoetry\Cache\virtualenvs\sales-bot-f2_0xw7_-py3.11\Lib\site-packages\langchain\chat_models\openai.py", line 420, in get_num_tokens_from_messages
    raise NotImplementedError(
NotImplementedError: get_num_tokens_from_messages() is not presently implemented for model gpt-3.5-turbo-0613.See https://github.com/openai/openai-python/blob/main/chatml.md for information on how messages are converted to tokens.

For example, when working with gpt-3.5-turbo-0613, he is not able to count the number of tokens, because the moment of checking the model is hard-coded Please correct the following in def get_num_tokens_from_messages(langchain\chat_models\openai.py)":

        if model == "gpt-3.5-turbo":
            # gpt-3.5-turbo may change over time.
            # Returning num tokens assuming gpt-3.5-turbo-0301.
            model = "gpt-3.5-turbo-0301"
        elif model == "gpt-4":
            # gpt-4 may change over time.
            # Returning num tokens assuming gpt-4-0314.
            model = "gpt-4-0314"

Replase:

        if "gpt-3.5-turbo" in model:
            # gpt-3.5-turbo may change over time.
            # Returning num tokens assuming gpt-3.5-turbo-0301.
            model = "gpt-3.5-turbo-0301"
        elif "gpt-4" in model:
            # gpt-4 may change over time.
            # Returning num tokens assuming gpt-4-0314.
            model = "gpt-4-0314"

Expected behavior

Sorry, because the project is closed, there is no way to provide sources, but the problem arises when using blind models and when calculating the number of tokens

memory = ConversationSummaryBufferMemory(
    llm=self.analyse_llm,
    ai_prefix="1",
    human_prefix="2",
    max_token_limit=self.max_memory_token_limit,
)

#  loaded old memory, and check length with prune causes an error

memory.prune()

dosubot[bot] commented 1 year ago

🤖

Thank you for bringing this issue to our attention! It seems like there is an error with the get_num_tokens_from_messages() function when using the gpt-3.5-turbo-0613 model in LangChain.

Based on your description, it looks like you have identified a potential solution to the problem by modifying the code in openai.py. We appreciate your effort in proposing a fix!

To contribute your solution to LangChain, you can create a pull request (PR) with your code changes. The maintainers will review your proposed fix and decide if it should be merged into the project.

Please follow the contribution guidelines of LangChain and submit your PR with the proposed code changes. Your contribution will help improve the framework for everyone using it.

Thank you for your willingness to contribute to LangChain! Let us know if you have any further questions or need any assistance with the contribution process.

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

Zabrell commented 1 year ago

Sorry, my version of the library was older, there is no such error in the new one

langchain-ai / langchain