Use external website instead of S3

[GregoryWullimann]

Hello!

The images I want to use are not stored in S3, but rather on a website, e.g. https://mywebsite.com/images/cat.png .

I've been trying to modify the code that simply download the image from the website rather than S3. I thought it was straight forward but I'm getting Internal Server Error.

What would be the correct way of solving this?

const fetch = require("node-fetch");
// ...
       const BASE_URL = "https://mywebsite.com";
        let originalImageBuffer;
    let contentType;
    try {
        const originalImageUrl = `${BASE_URL}/${originalImagePath}`
        const response = await fetch(originalImageUrl);
        if (!response.ok) {
            throw new Error(`HTTP error! status: ${response.status}`);
        }
        originalImageBuffer = await response.buffer();
        contentType = response.headers.get("content-type");
    } catch (error) {
        return sendError(500, "error downloading original image", error);
    }

    let transformedImage = Sharp(originalImageBuffer, {
        failOn: "none",
        animated: true,
    });

[achrafsouk]

To facilitate the troubleshooting, can you share the logs of the Lambda execution ?

[GregoryWullimann]

To facilitate the troubleshooting, can you share the logs of the Lambda execution ?

Hi!

Turns out node-fetch doesn't support require. I ended up importing it like this:

const fetch = (...args) => import('node-fetch').then(({default: fetch}) => fetch(...args));

The rest of the code works fine!

[piotrekwitkowski]

Hi Gregory - glad to hear it worked! There is one more thing you can try to simplify your function: if you are not using S3 and therefore not using AWS SDK at all(?), you may try updating AWS Lambda runtime to Node 18 or Node 20. Node-native fetch API was added in v18 as an experimental feature, and is a stable feature now in v20. v20 brings performance improvements to fetch, so out of the two it's better to use the newer version. You don't need to import/require node-fetch at all then. And the function will have smaller bundle size and memory footprint too.

Lambda announced support for Node v20 runtime earlier this week.

Please keep in mind that Lambda Node runtimes include different AWS SDK versions: runtimes up to version 16 have AWS SDK for JS v2, while those from version 18 onwards include AWS SDK for JS v3. If you're not using the AWS SDK to make S3 calls but are using it for other purposes within your function code, you might need to modify those specific sections of the code.