OP-Engineering / link-preview-js

⛓ Extract web links information: title, description, images, videos, etc. [via OpenGraph], runs on mobiles and node.
MIT License
770 stars 124 forks source link

Timeout not respected #122

Closed wouterlemcke closed 2 years ago

wouterlemcke commented 2 years ago

We noticed the configured timeout is not respected.

await getLinkPreview(url, {
    imagesPropertyType: 'og',
    headers: {
       'user-agent': 'googlebot'
    },
    timeout: 1000,
}))

When trying to generate a link preview for for example https://video013.kerkomroep.nl/pages/afspelen.php?mp=11115&file=2022-07-03-08-41-15.mp4 it never times out.

The weird thing is that when I set timeout to 10 it works, but as soon as I put it on 100 or above it just freezes.

ospfranco commented 2 years ago

That's weird, the timeout code is very simple... maybe you are running it in an environment where the AbortController is not supported?

In any case, it would be great if you can post a reproducible example.

Screenshot 000728
ospfranco commented 2 years ago

I can see the tests fail on the github runner, but on my local machine they work fine, so it probably has something to do with the node version. ping @wouterlemcke, can you provide more info where are you running this? node? which version? etc.

wouterlemcke commented 2 years ago

Sorry for the slow response, I was on holiday 😄 We are running Node 14.19.3 on AWS EC2s, but also locally on my Macbook (running Node 14.19.0) I can reproduce it

ospfranco commented 2 years ago

Welp 🤷‍♂️

Screenshot 000817

Kinda torn on this though... no idea if there is any other mechanism to timeout a fetch API request

wouterlemcke commented 2 years ago

Haha, that's annoying. I can share my temp. solution:

const linkPreview = (await Promise.race([
                getLinkPreview(url, {
                    imagesPropertyType: 'og',
                    headers,
                    timeout,
                    followRedirects: 'manual',
                    handleRedirects: (baseURL: string, forwardedURL: string) => {
                        return LinkPreviewRedirect.isWhiteListed(baseURL, forwardedURL)
                    },
                    resolveDNSHost: async (url: string) => {
                        const hostname = new URL(url).hostname
                        const result = await dns.promises.lookup(hostname)
                        return result.address
                    },
                }),
                new Promise((resolve, reject) => {
                    setTimeout(() => {
                        reject(new GeneratingLinkPreviewFailedException('Timeout'))
                    }, timeout)
                }),
            ]))
ospfranco commented 2 years ago

ospfranco commented 2 years ago

Going to close this, node 14 has already reached eol on active support (despite security patches). And there is the existing workaround you posted.