Error: Failed to complete: Connection error - with Anthropic

[kgilpin]

When running cli.js navie, the following stack trace is observed:

Error: Failed to complete: Connection error.
    at AnthropicCompletionService.complete_1 (/Users/kgilpin/source/appland/appmap-js/packages/navie/dist/services/anthropic-completion-service.js:165:23)
    at complete_1.throw (<anonymous>)
    at resume (/Users/kgilpin/source/appland/appmap-js/packages/navie/dist/services/anthropic-completion-service.js:24:44)
    at reject (/Users/kgilpin/source/appland/appmap-js/packages/navie/dist/services/anthropic-completion-service.js:27:30)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Handling exception: Error: Failed to complete: Connection error.
    at AnthropicCompletionService.complete_1 (/Users/kgilpin/source/appland/appmap-js/packages/navie/dist/services/anthropic-completion-service.js:165:23)
    at complete_1.throw (<anonymous>)
    at resume (/Users/kgilpin/source/appland/appmap-js/packages/navie/dist/services/anthropic-completion-service.js:24:44)
    at reject (/Users/kgilpin/source/appland/appmap-js/packages/navie/dist/services/anthropic-completion-service.js:27:30)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Stack trace: Error: Failed to complete: Connection error.
    at AnthropicCompletionService.complete_1 (/Users/kgilpin/source/appland/appmap-js/packages/navie/dist/services/anthropic-completion-service.js:165:23)
    at complete_1.throw (<anonymous>)
    at resume (/Users/kgilpin/source/appland/appmap-js/packages/navie/dist/services/anthropic-completion-service.js:24:44)
    at reject (/Users/kgilpin/source/appland/appmap-js/packages/navie/dist/services/anthropic-completion-service.js:27:30)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Note that the initial completion to build vector terms seems to work.

Environment

ANTHROPIC_API_KEY=sk-ant-api03-0Eke...
APPMAP_NAVIE_MODEL=claude-3-5-sonnet-20240620

Reproduce

Command that fails

RuntimeError: Failed to execute command  /Users/kgilpin/source/appland/appmap-js/packages/cli/built/cli.js -v navie --log-navie -i /Users/kgilpin/source/appland/navie-benchmark/solve/pytest-dev__pytest-7490/navie/generate-test/1/choose/search/search.txt -p /Users/kgilpin/source/appland/navie-benchmark/solve/pytest-dev__pytest-7490/navie/generate-test/1/choose/search/search.format.txt -o /Users/kgilpin/source/appland/navie-benchmark/solve/pytest-dev__pytest-7490/navie/generate-test/1/choose/search/search.output.txt > /Users/kgilpin/source/appland/navie-benchmark/solve/pytest-dev__pytest-7490/navie/generate-test/1/choose/search/search.log 2>&1. See /Users/kgilpin/source/appland/navie-benchmark/solve/pytest-dev__pytest-7490/navie/generate-test/1/choose/search/search.log for details.

Navie log

search.zip

Source code

pytest-dev__pytest-7490

source.zip

[dividedmind]

I can't reproduce this. Looks like it was a genuine connection error. Sonnet 3.5 is often overloaded.

[kgilpin]

Ok. So that manifests as a connection error? Should we back off and retry?

[dividedmind]

Ok. So that manifests as a connection error?

Not usually, what I've seen is a 4xx HTTP status code. But I suppose it happened in this instance? Or maybe there was a network problem. 🤷🏼‍♂️

Should we back off and retry?

I'm pretty sure langchain already does that by default.