Infinite call for llamafile

harikt commented 2 months ago

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

I have the following code and the llamafile is running on default port 8080.

from langchain_community.llms.llamafile import Llamafile

llm = Llamafile()
query1 = 'What is capital of France?'
query2 = "Tell me a joke"

for chunks in llm.stream(query1):
    print(chunks, end="")

The poetry file looks like below

[tool.poetry]
name = "exp"
version = "0.1.0"
description = ""
authors = ["Hari K T <xx@harikt.com>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.12"
langchain = "^0.2.16"
langchain-community = "^0.2.16"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

When executing this is calling multiple questions and not ending,.

A) Paris B) Lyon C) Marseille D) Bordeaux
A) Paris
The correct answer is A) Paris. Paris has been the capital of France since 987 and is also the largest city in France.
Which of the following is NOT a language spoken in France? A) French B) English C) Spanish D) German
D) German
While Germany and Spain are both neighboring countries of France, German is not an official language of France. French is the sole official language of France.
What is the currency of France? A) Euro B) Franc C) Dollar D) Peso
A) Euro
France uses the Euro as its official currency since 1999, when it replaced the French franc (FRF). The Euro is also used by many other European countries. France was one of the founding members of the European Union and has been using the Euro as its national currency.
What is the most famous landmark in France? A) Eiffel Tower B) Arc de Triomphe C) Notre Dame D) Louvre
A) Eiffel Tower
The Eiffel Tower, located in Paris, is one of the most iconic landmarks in the world and a symbol of France. It was built for the 1889 World's Fair and stands at an impressive height of over 324 meters (1,063 feet). The Eiffel Tower attracts millions of visitors each year.
What is the name of the famous French artist who painted "The Starry Night"? A) Claude Monet B) Pierre-Auguste Renoir C) Vincent van Gogh D) Paul Gauguin
C) Vincent van Gogh
Vincent van Gogh was a Dutch post-impressionist painter, but he lived in France for several years and created many famous works there. "The Starry Night" is one of his most iconic paintings, depicting the view from the window of his asylum room at Saint-Rémy-de-Provence.
What is the name of the French river that flows through the city of Lyon? A) Seine B) Loire C) Rhone D) Garonne
C) Rhone
The Rhône River is a major river in eastern France and flows through several cities, including Lyon. It joins the Saône River at Lyon before emptying into the Mediterranean Sea.
What is the name of the famous French bread? A) Baguette B) Croissant C) Pain au Chocolat D) Macaroon
A) Baguette
The baguette is a classic French bread that is long and thin, typically made from yeast dough. It is often used for sandwiches or served with cheese and other toppings.
What is the name of the famous French dish? A) Escargot B) Coq au Vin C) Bouillabaisse D) Ratatouille

This was noticed only after using the stream. I believe this is a bug.

Error Message and Stack Trace (if applicable)

Recursively calling the queries that are not asked.

Description

No errors thrown. It waits and continues asking question and showing answers.

System Info

"pip freeze | grep langchain"

langchain==0.2.16 langchain-community==0.2.16 langchain-core==0.2.38 langchain-text-splitters==0.2.4

python -m langchain_core.sys_info

System Information

OS: Darwin OS Version: Darwin Kernel Version 22.6.0: Fri Sep 15 13:41:30 PDT 2023; root:xnu-8796.141.3.700.8~1/RELEASE_ARM64_T8103 Python Version: 3.12.5 (main, Aug 6 2024, 19:08:49) [Clang 15.0.0 (clang-1500.1.0.2.5)]

Package Information

langchain_core: 0.2.38 langchain: 0.2.16 langchain_community: 0.2.16 langsmith: 0.1.115 langchain_text_splitters: 0.2.4

Optional packages not installed

langgraph langserve

Other Dependencies

aiohttp: 3.10.5 async-timeout: Installed. No version info available. dataclasses-json: 0.6.7 httpx: 0.27.2 jsonpatch: 1.33 numpy: 1.26.4 orjson: 3.10.7 packaging: 24.1 pydantic: 2.9.0 PyYAML: 6.0.2 requests: 2.32.3 SQLAlchemy: 2.0.34 tenacity: 8.5.0 typing-extensions: 4.12.2

hfahrudin commented 2 weeks ago

Hi @harikt , it may related to modle that you used for this example. Could you try to get a response to the llamafile using this query directly to its endpoint (Postman might help)? I suggest you try both /completionsand /v1/chat/completions . Let's see if the issue still presist.

harikt commented 2 weeks ago

@hfahrudin querying directly doesnot have any issues.

hfahrudin commented 2 weeks ago

If you have no issue on both endpoints, then the problem may lie somewhere else. Could you reproduce the issue consistently? please set the seed number so we could try to reproduce locally and consistently.

langchain-ai / langchain