AWS credentials resolution for Sagemaker models

chat-ui is excellent, thanks for all your amazing work here!

I have been experimenting with a model in Sagemaker and am having some issues with the model endpoint configuration. It currently requires credentials to be provided explicitly. This does work, but the ergonomics are not great for our use cases:

in development, my team uses AWS SSO and it would be great to use our session credentials and not need to update our MODELS environment variable manually every time our sessions refresh
in deployments, we would want to use an instance or task execution role to sign requests

In my investigation I found this area of code https://github.com/huggingface/chat-ui/blob/eb071be4c938b0a2cf2e89a152d68305d4714949/src/lib/server/endpoints/aws/endpointAws.ts#L22-L37, which uses the aws4fetch library that only support signing with explicitly passed AWS credentials.

I was able to update this area of code locally and support AWS credential resolution by switching this to use a different library aws-sigv4-fetch like so:

try {
    createSignedFetcher = (await import("aws-sigv4-fetch")).createSignedFetcher;
} catch (e) {
    throw new Error("Failed to import aws-sigv4-fetch");
}

const { url, accessKey, secretKey, sessionToken, model, region, service } =
    endpointAwsParametersSchema.parse(input);

const signedFetch = createSignedFetcher({
    service,
    region,
    credentials:
        accessKey && secretKey
            ? { accessKeyId: accessKey, secretAccessKey: secretKey, sessionToken }
            : undefined,
});

// Replacer `aws.fetch` with `signedFetch` below when passing `fetch` to `textGenerationStream#options`

My testing has found this supports passing credentials like today, or letting the AWS SDK resolve them through the default chain.

Would you be open to a PR with this change? Or is there a different/better/more suitable way to accomplish AWS credential resolution here?

huggingface / chat-ui

AWS credentials resolution for Sagemaker models #926