`max_tokens_to_sample: 9016` - Is this the Anthropic API maximum?

[hhooeezz]

This is regarding this line.

We cannot find the 9016 token maximum anywhere in the Anthropic or any third-party documentation. The max_tokens_to_sample _is_ mandatory, though, so we must set something (obviously <<10000).

If there is no hard limit in Anthropic's API, why not explicitly leave it to the script user:

// Config constant for max tokens 
const MAX_TOKENS = 9016;

async function handleRequest(request) {

  // ...

  const claudeRequestBody = {
    prompt,
    model: claudeModel,
    temperature,
    stop_sequences: stop,
    stream,

    // Always include max_tokens, using constant as default
    max_tokens_to_sample: MAX_TOKENS  
  };

  // ...

}

Either way, we appreciate having found your nifty script.

[jtsang4]

The 9016 is an empirical value, because Cloudflare Worker only supports API response time within 10ms for free plan. However, this value does not seem to work recently, Proxy API endpoint deployed using Cloudflare Worker now often have the problem of returning truncated. I have adopted the Docker deployment solution myself.

You are right, this value can be set as a constant for easy modification by those in need, I will extract it.

[jtsang4]

The value has been extracted.