search

Pythagora-io / gpt-pilot

The first real AI developer
Other
31.82k stars 3.2k forks source link

[Bug]: Claude 3 api usage still generally unstable. #779

Closed luyandadhlamini2 closed 7 months ago

luyandadhlamini2 commented 7 months ago

Version

Command-line (Python) version

Operating System

Ubuntu Linux

What happened?

The addition of Claude 3 support has been great. The biggest advantage of Claude 3 over current openai models is the costs saving that comes with the smaller claude model (Haiku).

However I've found that Claude is still very prone to producing errors that lead to gpt-pilot crashing. These errors mostly relate to:

  1. Extra data errors (gpt-pilot sometimes struggles with handling responses from Claude models. These are harder to workaround & sometimes require several restarts of a project to overcome)
  2. 429 errors (rate limits - which can be fixed by upgrading to a higher tier of Anthropic subscription).

Some examples in screenshots below Extra data: Screenshot 2024-03-22 070528 None type handling: Screenshot 2024-03-22 070502

Hopefully some of these issues will be resolved in coming days, with more robust error handling by gpt-pilot.

senko commented 7 months ago

@luyandadhlamini2 Haiku is known to struggle in a few places, so using Sonnet or Opus is recommended.