Is Anthropic / Claude use broken at the moment? -- NOW WITH FIX IN REPLIES

[Speedway1]

Running with Anthropic I get: Error choosing agent: 1 validation error for ChatAnthropic max_tokens_to_sample none is not an allowed value (type=type_error.none.not_allowed) Default Agent ERROR: Exception in ASGI application Traceback (most recent call last): File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 244, in run_asgi result = await self.app(self.scope, self.asgi_receive, self.asgi_send) # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in call return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in call await super().call(scope, receive, send) File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/applications.py", line 123, in call await self.middleware_stack(scope, receive, send) File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/middleware/errors.py", line 151, in call await self.app(scope, receive, send) File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in call await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/routing.py", line 756, in call await self.middleware_stack(scope, receive, send) File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/routing.py", line 776, in app await route.handle(scope, receive, send) File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/routing.py", line 373, in handle await self.app(scope, receive, send) File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/routing.py", line 96, in app await wrap_app_handling_exceptions(app, session)(scope, receive, send) File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/routing.py", line 94, in app await func(session) File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/fastapi/routing.py", line 348, in app await dependant.call(values) File "/data/misc/gpt-researcher/backend/server.py", line 53, in websocket_endpoint report = await manager.start_streaming(task, report_type, report_source, websocket) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/misc/gpt-researcher/backend/websocket_manager.py", line 57, in start_streaming report = await run_agent(task, report_type, report_source, websocket) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/misc/gpt-researcher/backend/websocket_manager.py", line 75, in run_agent report = await researcher.run() ^^^^^^^^^^^^^^^^^^^^^^ File "/data/misc/gpt-researcher/backend/report_type/detailed_report/detailed_report.py", line 33, in run await self._initial_research() File "/data/misc/gpt-researcher/backend/report_type/detailed_report/detailed_report.py", line 54, in _initial_research await self.main_task_assistant.conduct_research() File "/data/misc/gpt-researcher/gpt_researcher/master/agent.py", line 96, in conduct_research self.context = await self.get_context_by_search(self.query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/misc/gpt-researcher/gpt_researcher/master/agent.py", line 179, in get_context_by_search await get_sub_queries(query=query, agent_role_prompt=self.role, File "/data/misc/gpt-researcher/gpt_researcher/master/actions.py", line 101, in get_sub_queries response = await create_chat_completion( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/misc/gpt-researcher/gpt_researcher/utils/llm.py", line 88, in create_chat_completion provider = get_llm(llm_provider, model=model, temperature=temperature, max_tokens=max_tokens, llm_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/misc/gpt-researcher/gpt_researcher/utils/llm.py", line 52, in get_llm return llm_provider(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^ File "/data/misc/gpt-researcher/gpt_researcher/llm_provider/anthropic/anthropic.py", line 19, in init self.llm = self.get_llm_model() ^^^^^^^^^^^^^^^^^^^^ File "/data/misc/gpt-researcher/gpt_researcher/llm_provider/anthropic/anthropic.py", line 36, in get_llm_model llm = ChatAnthropic( ^^^^^^^^^^^^^^ File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/pydantic/v1/main.py", line 341, in init raise validation_error pydantic.v1.error_wrappers.ValidationError: 1 validation error for ChatAnthropic max_tokens_to_sample none is not an allowed value (type=type_error.none.not_allowed)

[Nizier193]

i have dealt with this problem. the issue is in gpt_researcher/llm_provider/anthropic/anthropic.py

line 36

llm = ChatAnthropic(
    model=self.model,
    temperature=self.temperature,
    max_tokens=self.max_tokens,
    api_key=self.api_key
)

error occurs because ChatAnthropic needs max_tokens_to_sample parameter to work but for some reason it is a None in this case

i simply deleted max_tokens parameter and replaced it with max_tokens_to_sample then i got this

llm = ChatAnthropic(
    model=self.model,
    temperature=self.temperature,
    max_tokens_to_sample = 1024, # or any other int value, 1024 is a default value
    api_key=self.api_key
)

works perfectly for me

[Speedway1]

Yes, I found that too, but I did the code slightly differently, I set it to 4096 because this is the maximum that Claude 3.5 will return:

    def get_llm_model(self):
        # Initializing the chat model
        llm = ChatAnthropic(
            model=self.model,
            temperature=self.temperature,
            max_tokens=self.max_tokens,
            max_tokens_to_sample= 4096,
            api_key=self.api_key
        )

Also I am sharing the max tokens in the .env file for the new Sonnet3.5 model:

FAST_LLM_MODEL=claude-3-5-sonnet-20240620 SMART_LLM_MODEL=claude-3-5-sonnet-20240620 FAST_TOKEN_LIMIT=200000 SMART_TOKEN_LIMIT=200000 SUMMARY_TOKEN_LIMIT=200000

Also, to support the added contact window size, you need to change: utils/llm.py

    # validate input
    if model is None:
        raise ValueError("Model cannot be None")
    if max_tokens is not None and max_tokens > 200001:
        raise ValueError(
            f"Max tokens cannot be more than 200001, but got {max_tokens}")

[assafelovic]

Thanks for the fix @Speedway1, next time would love to give you a shoutout with a PR merge :) Fixed here: https://github.com/assafelovic/gpt-researcher/commit/95eb94c8c60f48294dc3b9ef61de5c026464fd40