search

langchain-ai / langchain

🦜🔗 Build context-aware reasoning applications
https://python.langchain.com
MIT License
92.32k stars 14.75k forks source link

`temperature` Parameter Seems to Have No Effect on ChatOllama #25326

Open GuyPaddock opened 1 month ago

GuyPaddock commented 1 month ago

Checked other resources

Example Code

from langchain_ollama import ChatOllama
from langchain_core.prompts import ChatPromptTemplate

llm = ChatOllama(model='mistral-nemo:12b', temperature=0)

# Prompt
system = """You're a prolific writer of poetry."""
prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system),
        ("human", "Please write a poem about dogs."),
    ]
)

chain = prompt | llm
response = chain.invoke({})

print(response)

Error Message and Stack Trace (if applicable)

No response

Description

When this code is run multiple times, you get different results each time even though temperature=0. I get the same result regardless of the model (I've tried: mistral-nemo:12b, mixtral:8x7b, and ``)

For example, the results of two different runs (note that they start the same but vary in the middle):

content='In twilight\'s hush, where shadows play,\nThere dances a creature, swift and gray.\nWith eyes that gleam like distant stars,\nAnd heart that beats with ancient scars.\n\nA dog, they call him, loyal friend,\nWhose love transcends the world\'s trend.\nThrough fields of gold or city streets,\nHis faithful spirit never retreats.\n\nHe bounds through life on paws so light,\nIn joy or sorrow, day or night.\nWith wagging tail and lolling tongue,\nHe sings his song, both long and lung.\n\nBeneath the moon or sun\'s warm kiss,\nHe guards his pack with fierce caress.\nYet in his eyes, there lies a plea:\n"Love me, protect me, set me free."\n\nFor dogs are bound by chains unseen,\nTo humans\' hearts, their love serene.\nThey ask for naught but scraps and crumbs,\nAnd give us all the world\'s sweet sums.\n\nSo here\'s to dogs, both big and small,\nWho stand tall against life\'s cruel thrall.\nMay we be worthy of their grace,\nAnd find in them our resting place.' response_metadata={'model': 'mistral-nemo:12b', 'created_at': '2024-08-13T01:48:01.337930892Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 10320603800, 'load_duration': 22148447, 'prompt_eval_count': 18, 'prompt_eval_duration': 91910000, 'eval_count': 230, 'eval_duration': 10076617000} id='run-bb06db26-3791-4d6f-be86-98e6ee2ee699-0' usage_metadata={'input_tokens': 18, 'output_tokens': 230, 'total_tokens': 248}
content="In twilight's hush, where shadows play,\nThere dances a creature, swift and gray.\nWith eyes that gleam like distant stars,\nAnd heart that beats with ancient scars.\n\nA dog, they call him, loyal friend,\nHis lineage traced back to the bend\nOf time itself, when wolf and man\nFirst forged their bond on blood-stained sand.\n\nHe trots through fields of emerald green,\nWhere whispers of wind through wheat are seen.\nHis nose, a compass, guides his way,\nThrough scent-laden paths he'll never stray.\n\nIn city streets, where neon lights glow,\nHe pads along, his coat aglow.\nA symphony of senses stirred,\nBy every smell that's ever occurred.\n\nBeneath the moon's silver caress,\nHe guards his pack with fierce possessiveness.\nYet in daylight hours, he'll loll and play,\nWith children's laughter echoing ballet.\n\nHis dreams are wild, as free as he,\nChasing deer through forests deep and green.\nBut waking finds him by our side,\nA companion steadfast, ever-true guide.\n\nOh, dog of mine, with soul so bright,\nYou light my path in darkest night.\nYour love is pure, your spirit bold,\nA tale forevermore to be told." response_metadata={'model': 'mistral-nemo:12b', 'created_at': '2024-08-13T01:48:12.964980836Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 11613020626, 'load_duration': 21177489, 'prompt_eval_count': 18, 'prompt_eval_duration': 43662000, 'eval_count': 260, 'eval_duration': 11418682000} id='run-2d28422d-a5b0-490c-9147-75fa2599d487-0' usage_metadata={'input_tokens': 18, 'output_tokens': 260, 'total_tokens': 278}

System Info

System Information
------------------
> OS:  Linux
> OS Version:  #3 SMP PREEMPT_DYNAMIC Wed Aug  7 22:17:28 EDT 2024
> Python Version:  3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]

Package Information
-------------------
> langchain_core: 0.2.24
> langchain: 0.2.11
> langchain_community: 0.2.10
> langsmith: 0.1.93
> langchain_chroma: 0.1.2
> langchain_experimental: 0.0.63
> langchain_ollama: 0.1.0
> langchain_text_splitters: 0.2.2
> langgraph: 0.1.15

Packages not installed (Not Necessarily a Problem)
--------------------------------------------------
The following packages were not found:

> langserve

Pip freeze:

aiohttp @ file:///home/conda/feedstock_root/build_artifacts/aiohttp_1713964853148/work
aiosignal @ file:///home/conda/feedstock_root/build_artifacts/aiosignal_1667935791922/work
annotated-types @ file:///home/conda/feedstock_root/build_artifacts/annotated-types_1716290248287/work
anyio @ file:///home/conda/feedstock_root/build_artifacts/anyio_1717693030552/work
argon2-cffi @ file:///home/conda/feedstock_root/build_artifacts/argon2-cffi_1692818318753/work
argon2-cffi-bindings @ file:///home/conda/feedstock_root/build_artifacts/argon2-cffi-bindings_1695386546427/work
arrow @ file:///home/conda/feedstock_root/build_artifacts/arrow_1696128962909/work
asgiref==3.8.1
asttokens @ file:///home/conda/feedstock_root/build_artifacts/asttokens_1698341106958/work
async-lru @ file:///home/conda/feedstock_root/build_artifacts/async-lru_1690563019058/work
async-timeout @ file:///home/conda/feedstock_root/build_artifacts/async-timeout_1691763562544/work
attrs @ file:///home/conda/feedstock_root/build_artifacts/attrs_1704011227531/work
Babel @ file:///home/conda/feedstock_root/build_artifacts/babel_1702422572539/work
backoff==2.2.1
bcrypt==4.2.0
beautifulsoup4 @ file:///home/conda/feedstock_root/build_artifacts/beautifulsoup4_1705564648255/work
bleach @ file:///home/conda/feedstock_root/build_artifacts/bleach_1696630167146/work
Brotli @ file:///home/conda/feedstock_root/build_artifacts/brotli-split_1695989787169/work
build==1.2.1
cached-property @ file:///home/conda/feedstock_root/build_artifacts/cached_property_1615209429212/work
cachetools==5.4.0
certifi @ file:///home/conda/feedstock_root/build_artifacts/certifi_1720457958366/work/certifi
cffi @ file:///home/conda/feedstock_root/build_artifacts/cffi_1696001684923/work
chardet==5.2.0
charset-normalizer @ file:///home/conda/feedstock_root/build_artifacts/charset-normalizer_1698833585322/work
chroma-hnswlib==0.7.6
chromadb==0.5.5
click==8.1.7
coloredlogs==15.0.1
comm @ file:///home/conda/feedstock_root/build_artifacts/comm_1710320294760/work
dataclasses-json==0.6.7
debugpy @ file:///home/conda/feedstock_root/build_artifacts/debugpy_1719378659226/work
decorator @ file:///home/conda/feedstock_root/build_artifacts/decorator_1641555617451/work
defusedxml @ file:///home/conda/feedstock_root/build_artifacts/defusedxml_1615232257335/work
Deprecated==1.2.14
elastic-transport==8.13.1
elasticsearch==8.14.0
entrypoints @ file:///home/conda/feedstock_root/build_artifacts/entrypoints_1643888246732/work
exceptiongroup @ file:///home/conda/feedstock_root/build_artifacts/exceptiongroup_1720869315914/work
executing @ file:///home/conda/feedstock_root/build_artifacts/executing_1698579936712/work
faiss-gpu==1.7.2
fastapi==0.112.0
fastjsonschema @ file:///home/conda/feedstock_root/build_artifacts/python-fastjsonschema_1718477020893/work/dist
filelock==3.15.4
flatbuffers==24.3.25
fqdn @ file:///home/conda/feedstock_root/build_artifacts/fqdn_1638810296540/work/dist
frozenlist @ file:///home/conda/feedstock_root/build_artifacts/frozenlist_1702645481127/work
fsspec==2024.6.1
google-auth==2.32.0
googleapis-common-protos==1.63.2
greenlet @ file:///home/conda/feedstock_root/build_artifacts/greenlet_1703201576006/work
grpcio==1.65.4
h11 @ file:///home/conda/feedstock_root/build_artifacts/h11_1664132893548/work
h2 @ file:///home/conda/feedstock_root/build_artifacts/h2_1634280454336/work
hpack==4.0.0
httpcore @ file:///home/conda/feedstock_root/build_artifacts/httpcore_1711596990900/work
httptools==0.6.1
httpx @ file:///home/conda/feedstock_root/build_artifacts/httpx_1708530890843/work
huggingface-hub==0.24.5
humanfriendly==10.0
hyperframe @ file:///home/conda/feedstock_root/build_artifacts/hyperframe_1619110129307/work
idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1713279365350/work
importlib_metadata==8.0.0
importlib_resources @ file:///home/conda/feedstock_root/build_artifacts/importlib_resources_1711040877059/work
ipykernel @ file:///home/conda/feedstock_root/build_artifacts/ipykernel_1719845459717/work
ipython @ file:///home/conda/feedstock_root/build_artifacts/ipython_1719582526268/work
isoduration @ file:///home/conda/feedstock_root/build_artifacts/isoduration_1638811571363/work/dist
jedi @ file:///home/conda/feedstock_root/build_artifacts/jedi_1696326070614/work
Jinja2 @ file:///home/conda/feedstock_root/build_artifacts/jinja2_1715127149914/work
json5 @ file:///home/conda/feedstock_root/build_artifacts/json5_1712986206667/work
jsonpatch @ file:///home/conda/feedstock_root/build_artifacts/jsonpatch_1695536281965/work
jsonpointer @ file:///home/conda/feedstock_root/build_artifacts/jsonpointer_1718283388110/work
jsonschema @ file:///home/conda/feedstock_root/build_artifacts/jsonschema_1720529478715/work
jsonschema-specifications @ file:///tmp/tmpkv1z7p57/src
jupyter-events @ file:///home/conda/feedstock_root/build_artifacts/jupyter_events_1710805637316/work
jupyter-lsp @ file:///home/conda/feedstock_root/build_artifacts/jupyter-lsp-meta_1712707420468/work/jupyter-lsp
jupyter_client @ file:///home/conda/feedstock_root/build_artifacts/jupyter_client_1716472197302/work
jupyter_core @ file:///home/conda/feedstock_root/build_artifacts/jupyter_core_1710257277185/work
jupyter_server @ file:///home/conda/feedstock_root/build_artifacts/jupyter_server_1720816649297/work
jupyter_server_terminals @ file:///home/conda/feedstock_root/build_artifacts/jupyter_server_terminals_1710262634903/work
jupyterlab @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_1721396525904/work
jupyterlab_pygments @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_pygments_1707149102966/work
jupyterlab_server @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_server-split_1721163288448/work
kubernetes==30.1.0
langchain @ file:///home/conda/feedstock_root/build_artifacts/langchain_1721798007509/work
langchain-chroma==0.1.2
langchain-community==0.2.10
langchain-core @ file:///home/conda/feedstock_root/build_artifacts/langchain-core_1722041751363/work
langchain-experimental==0.0.63
langchain-ollama==0.1.0
langchain-text-splitters @ file:///home/conda/feedstock_root/build_artifacts/langchain-text-splitters_1719389695345/work
langgraph==0.1.15
langsmith @ file:///home/conda/feedstock_root/build_artifacts/langsmith_1721479018518/work
markdown-it-py==3.0.0
MarkupSafe @ file:///home/conda/feedstock_root/build_artifacts/markupsafe_1706899921127/work
marshmallow==3.21.3
matplotlib-inline @ file:///home/conda/feedstock_root/build_artifacts/matplotlib-inline_1713250518406/work
mdurl==0.1.2
mistune @ file:///home/conda/feedstock_root/build_artifacts/mistune_1698947099619/work
mmh3==4.1.0
monotonic==1.6
mpmath==1.3.0
multidict @ file:///home/conda/feedstock_root/build_artifacts/multidict_1707040698785/work
mypy-extensions==1.0.0
nbclient @ file:///home/conda/feedstock_root/build_artifacts/nbclient_1710317608672/work
nbconvert @ file:///home/conda/feedstock_root/build_artifacts/nbconvert-meta_1718135430380/work
nbformat @ file:///home/conda/feedstock_root/build_artifacts/nbformat_1712238998817/work
nest_asyncio @ file:///home/conda/feedstock_root/build_artifacts/nest-asyncio_1705850609492/work
notebook_shim @ file:///home/conda/feedstock_root/build_artifacts/notebook-shim_1707957777232/work
numpy @ file:///home/conda/feedstock_root/build_artifacts/numpy_1707225380409/work/dist/numpy-1.26.4-cp310-cp310-linux_x86_64.whl#sha256=51131fd8fc130cd168aecaf1bc0ea85f92e8ffebf211772ceb16ac2e7f10d7ca
oauthlib==3.2.2
ollama==0.3.0
onnxruntime==1.18.1
opentelemetry-api==1.26.0
opentelemetry-exporter-otlp-proto-common==1.26.0
opentelemetry-exporter-otlp-proto-grpc==1.26.0
opentelemetry-instrumentation==0.47b0
opentelemetry-instrumentation-asgi==0.47b0
opentelemetry-instrumentation-fastapi==0.47b0
opentelemetry-proto==1.26.0
opentelemetry-sdk==1.26.0
opentelemetry-semantic-conventions==0.47b0
opentelemetry-util-http==0.47b0
orjson @ file:///home/conda/feedstock_root/build_artifacts/orjson_1719968921648/work/target/wheels/orjson-3.10.6-cp310-cp310-linux_x86_64.whl#sha256=6a7a7934d9964ea3709a0f11ae8c0506ca6d8e7d4f8a437be382be2f14da02f4
overrides @ file:///home/conda/feedstock_root/build_artifacts/overrides_1706394519472/work
packaging @ file:///home/conda/feedstock_root/build_artifacts/packaging_1696202382185/work
pandocfilters @ file:///home/conda/feedstock_root/build_artifacts/pandocfilters_1631603243851/work
parso @ file:///home/conda/feedstock_root/build_artifacts/parso_1712320355065/work
pexpect @ file:///home/conda/feedstock_root/build_artifacts/pexpect_1706113125309/work
pickleshare @ file:///home/conda/feedstock_root/build_artifacts/pickleshare_1602536217715/work
pkgutil_resolve_name @ file:///home/conda/feedstock_root/build_artifacts/pkgutil-resolve-name_1694617248815/work
platformdirs @ file:///home/conda/feedstock_root/build_artifacts/platformdirs_1715777629804/work
posthog==3.5.0
prometheus_client @ file:///home/conda/feedstock_root/build_artifacts/prometheus_client_1707932675456/work
prompt_toolkit @ file:///home/conda/feedstock_root/build_artifacts/prompt-toolkit_1718047967974/work
protobuf==4.25.4
psutil @ file:///home/conda/feedstock_root/build_artifacts/psutil_1719274566094/work
ptyprocess @ file:///home/conda/feedstock_root/build_artifacts/ptyprocess_1609419310487/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl
pure_eval @ file:///home/conda/feedstock_root/build_artifacts/pure_eval_1721585709575/work
pyasn1==0.6.0
pyasn1_modules==0.4.0
pycparser @ file:///home/conda/feedstock_root/build_artifacts/pycparser_1711811537435/work
pydantic @ file:///home/conda/feedstock_root/build_artifacts/pydantic_1720293063581/work
pydantic_core @ file:///home/conda/feedstock_root/build_artifacts/pydantic-core_1720041207964/work
Pygments @ file:///home/conda/feedstock_root/build_artifacts/pygments_1714846767233/work
pypdf==4.3.1
PyPika==0.48.9
pyproject_hooks==1.1.0
PySocks @ file:///home/conda/feedstock_root/build_artifacts/pysocks_1661604839144/work
pysrt==1.1.2
python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/python-dateutil_1709299778482/work
python-dotenv==1.0.1
python-json-logger @ file:///home/conda/feedstock_root/build_artifacts/python-json-logger_1677079630776/work
pytz @ file:///home/conda/feedstock_root/build_artifacts/pytz_1706886791323/work
PyYAML @ file:///home/conda/feedstock_root/build_artifacts/pyyaml_1695373428874/work
pyzmq @ file:///home/conda/feedstock_root/build_artifacts/pyzmq_1715024398995/work
referencing @ file:///home/conda/feedstock_root/build_artifacts/referencing_1714619483868/work
regex==2024.7.24
requests @ file:///home/conda/feedstock_root/build_artifacts/requests_1717057054362/work
requests-oauthlib==2.0.0
rfc3339-validator @ file:///home/conda/feedstock_root/build_artifacts/rfc3339-validator_1638811747357/work
rfc3986-validator @ file:///home/conda/feedstock_root/build_artifacts/rfc3986-validator_1598024191506/work
rich==13.7.1
rpds-py @ file:///home/conda/feedstock_root/build_artifacts/rpds-py_1721860942725/work
rsa==4.9
Send2Trash @ file:///home/conda/feedstock_root/build_artifacts/send2trash_1712584999685/work
shellingham==1.5.4
six @ file:///home/conda/feedstock_root/build_artifacts/six_1620240208055/work
sniffio @ file:///home/conda/feedstock_root/build_artifacts/sniffio_1708952932303/work
soupsieve @ file:///home/conda/feedstock_root/build_artifacts/soupsieve_1693929250441/work
SQLAlchemy @ file:///home/conda/feedstock_root/build_artifacts/sqlalchemy_1718781149566/work
stack-data @ file:///home/conda/feedstock_root/build_artifacts/stack_data_1669632077133/work
starlette==0.37.2
sympy==1.13.1
tavily-python==0.3.5
tenacity @ file:///home/conda/feedstock_root/build_artifacts/tenacity_1720351771156/work
terminado @ file:///home/conda/feedstock_root/build_artifacts/terminado_1710262609923/work
tiktoken==0.7.0
tinycss2 @ file:///home/conda/feedstock_root/build_artifacts/tinycss2_1713974937325/work
tokenizers==0.19.1
tomli @ file:///home/conda/feedstock_root/build_artifacts/tomli_1644342247877/work
tornado @ file:///home/conda/feedstock_root/build_artifacts/tornado_1717722796999/work
tqdm==4.66.4
traitlets @ file:///home/conda/feedstock_root/build_artifacts/traitlets_1713535121073/work
typer==0.12.3
types-python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/types-python-dateutil_1710589910274/work
typing-inspect==0.9.0
typing-utils @ file:///home/conda/feedstock_root/build_artifacts/typing_utils_1622899189314/work
typing_extensions @ file:///home/conda/feedstock_root/build_artifacts/typing_extensions_1717802530399/work
Unidecode==1.3.8
uri-template @ file:///home/conda/feedstock_root/build_artifacts/uri-template_1688655812972/work/dist
urllib3 @ file:///home/conda/feedstock_root/build_artifacts/urllib3_1719391292974/work
uvicorn==0.30.5
uvloop==0.19.0
watchfiles==0.22.0
wcwidth @ file:///home/conda/feedstock_root/build_artifacts/wcwidth_1704731205417/work
webcolors @ file:///home/conda/feedstock_root/build_artifacts/webcolors_1717667289718/work
webencodings @ file:///home/conda/feedstock_root/build_artifacts/webencodings_1694681268211/work
websocket-client @ file:///home/conda/feedstock_root/build_artifacts/websocket-client_1713923384721/work
websockets==12.0
wrapt==1.16.0
yarl @ file:///home/conda/feedstock_root/build_artifacts/yarl_1705508292061/work
zipp @ file:///home/conda/feedstock_root/build_artifacts/zipp_1718013267051/work
zstandard==0.23.0
GuyPaddock commented 1 month ago

Potentially related: https://github.com/ollama/ollama/issues/5321

GuyPaddock commented 1 month ago

Question -- per https://github.com/langchain-ai/langchain/issues/24703, there isn't a way to set the random seed. How is that handled when temperature is 0? Is there still randomness? If so, what controls the random seed?