Closed undergroundwires closed 7 months ago
Hello @undergroundwires, do I understand correctly that you are trying to fetch using ipv6?
I @Alexey-Ayupov. According to my research, the error UND_ERR_CONNECT_TIMEOUT
related to DNS, and community reports mostly about IPv6, that's why I brought up IPv6. I cannot relate this to any outbound request limitation, even second/third requests fails after 10 seconds interval.
The test failing is simple:
const urls = [
'https://web.archive.org/web/20221029145712/https://kb.mozillazine.org/Downloads.rdf',
'https://web.archive.org/web/20240120213614/https://techcommunity.microsoft.com/t5/windows-it-pro-blog/group-configuration-search-highlights-in-windows/ba-p/3263989',
'https://web.archive.org/web/20230929132845/https://support.microsoft.com/en-us/topic/compatibility-update-for-keeping-windows-up-to-date-in-windows-server-2012-r2-and-windows-server-2008-r2-sp1-c62197fb-d711-f7d3-f135-172844b9f322',
'https://web.archive.org/web/20231004100105/https://nvd.nist.gov/vuln/search/results?form_type=Basic&results_type=overview&query=microsoft+store&queryType=phrase&search_type=all&isCpeNameSearch=false',
'https://web.archive.org/web/20240219205516/https://wiki.archlinux.org/title/spotify',
'https://web.archive.org/web/20240119160347/https://github.com/undergroundwires/privacy.sexy/issues/247',
'https://web.archive.org/web/20231206171559/https://bestgamingtips.com/fix-xbox-identity-provider-not-working/',
'https://web.archive.org/web/20230806192800/https://www.hexacorn.com/blog/2018/09/02/beyond-good-ol-run-key-part-86/',
'https://web.archive.org/web/20231020012236/https://answers.microsoft.com/es-es/windows/forum/all/windows-10-carpeta-y-archivos-sih/4d318121-fed6-4202-8b92-d4dc236b468e',
'https://web.archive.org/web/20221029141626/https://kb.mozillazine.org/Places.sqlite'
];
for (const url of urls) {
console.log('Fetching: ', url);
const response = await fetch(url, { method: 'HEAD'});
await sleep(10000);
expect(response.status).to.equal(200);
}
The error I'm getting after first, sometimes second request:
Caused by: ConnectTimeoutError: Connect Timeout Error
hosted runners do not support ipv6 yet, that is why every "real" ipv6 inquiry will fail, and there is no ETA on ipv6 fully available unfortunately.
Hi @mikhailkoliada, I tested more and verify that this is not about IPv6 requests being made.
This is how I prove it's not about IPv6 resolution directly:
dns.setDefaultResultOrder('ipv4first');
which forces IPv4 resolution. I added that test to tests in the repository.
Configuration: vite-test.spec.ts
sudo sh -c "echo 'precedence ::ffff:0:0/96 100' >> /etc/gai.conf"
Configuration: .github/workflows/test.yaml
I disable IPv6 on host container completely:
sysctl -w net.ipv6.conf.all.disable_ipv6=1
sysctl -w net.ipv6.conf.default.disable_ipv6=1
sysctl -w net.ipv6.conf.lo.disable_ipv6=1
Configuration: .github/workflows/test.yaml
I get UND_ERR_CONNECT_TIMEOUT
regardless.
Can it be possible that this is because of some ongoing connection (network/DNS requests) limitation in the image or cloud service you're running?
Here's my workaround (open-source and documented) that I hope that can help you too:
force-ipv4.sh
.: Script for general use.After days of research and trial/error, this is how I got this working:
force-ipv4.sh
, that configures system to prefer IPv4 over IPv6, call it to configure the machine. It was not easy to find a reliable cross-platform solution and I went Cloudflare WARP for DNS resolution along with some system configurations.force-ipv4
and call it in GitHub runners.fetch
API from Node.Related commit introducing this fix: https://github.com/undergroundwires/privacy.sexy/commit/52fadcd6177ed06216be9c67dad57192ae02a4f9
Description
Sending more than one (sometimes two or three) requests using
fetch
results in DNS errors.I get
UND_ERR_CONNECT_TIMEOUT
errors which is related to IPv6 issues in node. Related issue for node: nodejs/node#41625, Related issue for node fetch: nodejs/undic#1531 Related issue for GitHub runners not being able to do IPv6: actions/runner-images#668, actions/runner#3138Platforms affected
Runner images affected
Image version and build link
Failed build: https://github.com/undergroundwires/node-fetch-ipv6/actions/runs/8350276298
Is it regression?
Yes, successful build runs are old
Expected behavior
The requests are being sent fine, this was the case before.
Actual behavior
I created a repository to reproduce this in as minimal way as possible: https://github.com/undergroundwires/node-fetch-ipv6
I run fetch on using vite test runner on node and without any test runner. I get same results. I realize that the first two requests are successful, I start getting the error after the second or third one.
See the test: test file
Second test (multiple fetch in order) fails, and all run fine locally. I run tests on default GitHub runners using both latest node 20 and 18. I get same errors:
Repro steps