`ChatCloudflareWorkersAI` Chat Model Integration configuration

[demosjarco]

I see baseUrl as a configuration object, but not being used anywhere in the code. This feature is necessary to be able to run requests through Cloudflare's AI Gateway

langchain/src/chat_models/cloudflare_workersai.ts On line 35 it's declared and line 47 the value is set But on line 122, it's not being used at all

[jacoblee93]

Hi @demosjarco, thanks for the heads up - will push a fix today and let you know here!

[dosubot[bot]]

🤖

Hello,

Thank you for bringing this to our attention. You're correct, the baseUrl configuration object is declared and set, but it's not being used in the _call method where the Cloudflare API call is made. This is because the URL for the API call is currently hardcoded.

Here's the hardcoded URL on line 122:

const url = `https://api.cloudflare.com/client/v4/accounts/${this.cloudflareAccountId}/ai/run/${this.model}`;

If you want to use the baseUrl for constructing the URL, you can modify the _call method like this:

async _call(
  prompt: string,
  options: this["ParsedCallOptions"]
): Promise<string> {
  this.validateEnvironment();

  const url = `${this.baseUrl}/client/v4/accounts/${this.cloudflareAccountId}/ai/run/${this.model}`;
  const headers = {
    Authorization: `Bearer ${this.cloudflareApiToken}`,
    "Content-Type": "application/json",
  };

  const data = { prompt };
  const responseData = await this.caller.call(async () => {
    const response = await fetch(url, {
      method: "POST",
      headers,
      body: JSON.stringify(data),
      signal: options.signal,
    });
    if (!response.ok) {
      const error = new Error(
        `Cloudflare LLM call failed with status code ${response.status}`
      );
      // eslint-disable-next-line @typescript-eslint/no-explicit-any
      (error as any).response = response;
      throw error;
    }
    return response.json();
  });

  return responseData.result.response;
}

This way, the baseUrl will be used in the API call. Please make sure to provide a valid baseUrl when instantiating the CloudflareWorkersAI class.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Best regards, Dosu

Sources

• langchain/src/llms/cloudflare_workersai.ts

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.