AWS Bedrock isn't accessible from SvelteKit Project folder, getting a socket connection timeout error

[Aashw1n]

I'm trying to access the llama 3.1 405B model on AWS bedrock. Here's the code in my Sveltekit server side file: ` import { json } from '@sveltejs/kit';

import {PRIVATE_AWS_ACCESS_KEY, PRIVATE_AWS_SECRET_KEY} from '$env/static/private'

import { BedrockRuntimeClient, InvokeModelCommand } from "@aws-sdk/client-bedrock-runtime";

const client = new BedrockRuntimeClient({ credentials:{ accessKeyId:PRIVATE_AWS_ACCESS_KEY, secretAccessKey: PRIVATE_AWS_SECRET_KEY},region: "us-west-2" });

console.log(client)

const modelId = "meta.llama3-1-405b-instruct-v1:0"; /* @type {import('./$types').RequestHandler} /

/* @type {import('./$types').PageLoad} / export async function load() { try{

const userMessage = Tell me about donuts

const prompt = <|begin_of_text|> <|start_header_id|>user<|end_header_id|> ${userMessage} <|eot_id|> <|start_header_id|>assistant<|end_header_id|> ;

// Format the request payload using the model's native structure. const request = { prompt, // Optional inference parameters: max_gen_len: 1024, temperature: 0.7, top_p: 0.9, }

const response = await client.send( new InvokeModelCommand({ contentType: "application/json", accept: "application/json", body: JSON.stringify(request), modelId, }), );

// Decode the native response body. /* @type {{ generation: string }} / const nativeResponse = JSON.parse(new TextDecoder().decode(response.body));

// Extract and print the generated text. const responseText = nativeResponse.generation;

console.log(responseText); return {resp:responseText} }catch(err){ console.log(err) return{message: "This isn't working"} }

}`

All I'm able to get is this: Error [ERR_SOCKET_CONNECTION_TIMEOUT]: Socket connection timeout at new NodeError (node:internal/errors:399:5) at internalConnectMultiple (node:net:1099:20) at Timeout.internalConnectMultipleTimeout (node:net:1638:3) at listOnTimeout (node:internal/timers:575:11) at process.processTimers (node:internal/timers:514:7) { code: 'ERR_SOCKET_CONNECTION_TIMEOUT', '$metadata': { attempts: 1, totalRetryDelay: 0 } }

It does work on a separate, simple typescript project that only calls the model and prints.

Anything, even a clue as to what could be causing this will be a huge help.

Thank you.

[aBurmeseDev]

Hi @Aashw1n - thanks for reaching out.

The errorERR_SOCKET_CONNECTION_TIMEOUT means that your application is unable to establish a connection with the Bedrock service within the specified timeout period. This could be due to several reasons including network issues, misconfigured AWS credentials or incorrect service endpoint settings.

• Network issue: If you're running this code on a local machine or development environment, check if there are any network restrictions or firewalls that might be blocking the outgoing connection to the service.
• Timeout Settings: By default, JavaScript SDK has a socket timeout of 60 seconds. In some cases, this timeout might be too short especially if you're working with larger models or if there's network latency. You can try increasing the socket timeout by configuring the socketTimeout option when creating the BedrockRuntimeClient instance. For example:

  const client = new BedrockRuntimeClient({
  credentials: {
  ....
  },
  region: "us-west-2",
  socketTimeout: 120000 // Increase the timeout to 120 seconds (2 minutes)
  });

• Separate Project Comparison: Since you mentioned that the code works in a separate TypeScript project, try to identify any differences in the project setup, dependencies or configurations that might be causing the issue in your Sveltekit server-side file. Also note that we cannot offer guidance on third-party tool, framework or software that's not AWS.

Hope it helps! Best, John