ERROR: Exception in ASGI application when running with Ollama.

[banifou]

Not working with Ollama and llama3. Before it worked fine with Chatgpt 4-o.

From the logs:

Finalized research step. 💸 Total Research Costs: $0.01411754 🤔 Generating subtopics...

🤖 Calling llama3...

📋Subtopics: subtopics=[Subtopic(task='Learn more about BLABLA'), Subtopic(task='Analyze your competitors'), Subtopic(task='Content optimization')] INFO: connection closed Error in generating report introduction: 🔎 Starting the research task for 'Learn more about BLABLA'... ERROR: Exception in ASGI application Traceback (most recent call last): File "/python/gpt-researcher/venv/lib/python3.10/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 244, in run_asgi result = await self.app(self.scope, self.asgi_receive, self.asgi_send) # type: ignore[func-returns-value] File "/python/gpt-researcher/venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in call return await self.app(scope, receive, send) File "/python/gpt-researcher/venv/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in call await super().call(scope, receive, send) File "/python/gpt-researcher/venv/lib/python3.10/site-packages/starlette/applications.py", line 123, in call await self.middleware_stack(scope, receive, send) File "/python/gpt-researcher/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 151, in call await self.app(scope, receive, send) File "/python/gpt-researcher/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in call await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/python/gpt-researcher/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/python/gpt-researcher/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/python/gpt-researcher/venv/lib/python3.10/site-packages/starlette/routing.py", line 756, in call await self.middleware_stack(scope, receive, send) File "/python/gpt-researcher/venv/lib/python3.10/site-packages/starlette/routing.py", line 776, in app await route.handle(scope, receive, send) File "/python/gpt-researcher/venv/lib/python3.10/site-packages/starlette/routing.py", line 373, in handle await self.app(scope, receive, send) File "/python/gpt-researcher/venv/lib/python3.10/site-packages/starlette/routing.py", line 96, in app await wrap_app_handling_exceptions(app, session)(scope, receive, send) File "/python/gpt-researcher/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/python/gpt-researcher/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/python/gpt-researcher/venv/lib/python3.10/site-packages/starlette/routing.py", line 94, in app await func(session) File "/python/gpt-researcher/venv/lib/python3.10/site-packages/fastapi/routing.py", line 348, in app await dependant.call(**values) File "/python/gpt-researcher/backend/server.py", line 68, in websocket_endpoint report = await manager.start_streaming( File "/python/gpt-researcher/backend/websocket_manager.py", line 56, in start_streaming report = await run_agent(task, report_type, report_source, tone, websocket) File "/python/gpt-researcher/backend/websocket_manager.py", line 88, in run_agent report = await researcher.run() File "/python/gpt-researcher/backend/report_type/detailed_report/detailedreport.py", line 65, in run , report_body = await self._generate_subtopic_reports(subtopics) File "/python/gpt-researcher/backend/report_type/detailed_report/detailed_report.py", line 105, in _generate_subtopic_reports result = await fetch_report(subtopic) File "/python/gpt-researcher/backend/report_type/detailed_report/detailed_report.py", line 93, in fetch_report subtopic_report = await self._get_subtopic_report(subtopic) File "/python/gpt-researcher/backend/report_type/detailed_report/detailed_report.py", line 131, in _get_subtopic_report await subtopic_assistant.conduct_research() File "/python/gpt-researcher/gpt_researcher/master/agent.py", line 88, in conduct_research await stream_output( File "/python/gpt-researcher/gpt_researcher/master/actions.py", line 382, in stream_output await websocket.send_json({"type": type, "output": output}) File "/python/gpt-researcher/venv/lib/python3.10/site-packages/starlette/websockets.py", line 198, in send_json await self.send({"type": "websocket.send", "text": text}) File "/python/gpt-researcher/venv/lib/python3.10/site-packages/starlette/websockets.py", line 112, in send raise RuntimeError('Cannot call "send" once a close message has been sent.') RuntimeError: Cannot call "send" once a close message has been sent.

[arsaboo]

Working fine for me with Ollama.

Did it work with Ollama for you earlier?

[banifou]

No, I tried twice. Ollama works fine from the command line.

[arsaboo]

I meant, did GPTR ever work with Ollama for you?

[banifou]

What is GPTR?! Ah, gpt-researcher. Like I said, I tried twice with Ollama and failed twice. So it never worked with Ollama.

[assafelovic]

Hey @banifou can you try pulling the latest and try again? Also make sure to upgrade the gptr pip package: pip install gpt-researcher -U

[banifou]

Yes, it's working now! It takes much more longer until llama3 finished the work than with Openai!

[ElishaKay]

Sup @banifou @arsaboo

Is it possible to get a sample env file / setup guidance for running with Ollama? I'd love to create some documentation around this for the community.

I deployed an Open WebUI server with Elestio, but have yet to get it working

Here's my .env:

LLM_PROVIDER=ollama
OLLAMA_BASE_URL=https://ollama-ug3qr-u21899.vm.elestio.app:11434/api

OPENAI_API_KEY=OLLAMA
EMBEDDING_PROVIDER=ollama
FAST_LLM_MODEL=llama3
SMART_LLM_MODEL=llama3
OLLAMA_EMBEDDING_MODEL=snowflake-arctic-embed:l

Any recommendations on a working setup would be greatly appreciated

[arsaboo]

@ElishaKay Here's my env:

RETRIEVER=searx
SEARX_URL=http://host.docker.internal:4002
RETRIEVER_ENDPOINT=http://host.docker.internal:4002
# Ollama Config
LLM_PROVIDER=ollama
OLLAMA_BASE_URL=http://host.docker.internal:11434
OPENAI_API_KEY=OLLAMA
EMBEDDING_PROVIDER=ollama
FAST_LLM_MODEL=qwen2:72b-instruct
SMART_LLM_MODEL=llama3
OLLAMA_EMBEDDING_MODEL=snowflake-arctic-embed:l
DOC_PATH=./my-docs

Here's my compose file (the nextJS frontend is not working for me):

version: '3'
services:
  gpt-researcher:
    pull_policy: build
    image: gptresearcher/gpt-researcher
    build: ./
    environment:
      OPENAI_API_KEY: ${OPENAI_API_KEY}
      TAVILY_API_KEY: ${TAVILY_API_KEY}
      LANGCHAIN_API_KEY: ${LANGCHAIN_API_KEY}
    restart: always
    ports:
      - 8001:8000
    extra_hosts:
      - "host.docker.internal:host-gateway"
  gptr-nextjs:
    pull_policy: build
    image: gptresearcher/gptr-nextjs
    stdin_open: true
    environment:
      - CHOKIDAR_USEPOLLING=true
    build:
      dockerfile: Dockerfile.dev
      context: multi_agents/frontend
    volumes:
      - /app/node_modules
      - ./multi_agents/frontend:/app
    restart: always
    ports:
      - 3000:3000
    extra_hosts:
      - "host.docker.internal:host-gateway"

[ElishaKay]

Cool, thank @arsaboo

What error does docker compose throw for the nextjs app?

Try opening localhost:3000 in an incognito tab - sometimes nextjs has caching issues