lycheeverse / lychee

⚑ Fast, async, stream-based link checker written in Rust. Finds broken URLs and mail addresses inside Markdown, HTML, reStructuredText, websites and more!
https://lychee.cli.rs
Apache License 2.0
2.03k stars 121 forks source link

windows: all links timeout with >=0.10.0 versions #902

Closed soredake closed 1 year ago

soredake commented 1 year ago

For some reason, lychee version 0.10.0 and above now shows absolutely all links as timeout'ed, but in reality lychee does zero network activity (checked with task manager).

mre commented 1 year ago

Hey @soredake, how did you install lychee? I just tried it on a Windows 11 machine with the latest version from winget and it worked as expected:

> echo 'https://endler.dev' | lychee -
πŸ” 1 Total βœ… 1 OK 🚫 0 Errors
soredake commented 1 year ago

how did you install lychee?

From winget, running .exe's downloaded from github gives same result. Maybe my windows 11 install is broken somehow, latest version of lychee worked fine some time ago.

mre commented 1 year ago

Strange. curl works as expected? E.g. curl https://endler.dev?

soredake commented 1 year ago

Yes, curl works fine.

mre commented 1 year ago

That's confusing. Did you try my exact example? Might be blocking on longer lists, but I doubt it to be honest.

soredake commented 1 year ago

Did you try my exact example?

Yes, it fails with any file/link.

ΠΈΠ·ΠΎΠ±Ρ€Π°ΠΆΠ΅Π½ΠΈΠ΅

mre commented 1 year ago

Could it be the Windows Defender Firewall which blocks lychee's network access maybe? πŸ€”

soredake commented 1 year ago

Could it be the Windows Defender Firewall which blocks lychee's network access maybe? πŸ€”

I've thought about it, I've reset all firewall rules to default, allowed lychee again, no luck, dunno why old versions work.

mre commented 1 year ago

Very weird. A few more thoughts:

soredake commented 1 year ago

In Windows Defender, did you have to allow curl as well?

No.

Does it have the same settings? Probably it's allowed by default.

curl is not listed in windows defender firewall rules at all.

Does lychee --dump work?

Yes.

Does --offline plus a local file as input work?

I got this πŸ” 120 Total βœ… 0 OK 🚫 0 Errors πŸ’€ 120 Excluded

Can you try building lychee from master and test with that?

Same timeout. It'll be nice if lychee have builds in ci/cd for much easier testing without needing to compile from source.

Can you try an Administrator shell?

No luck.

mre commented 1 year ago

Running out of ideas here. It works on my Windows machine, but that's inside a VM. Maybe you have another Windows box to test this on or somebody else has more thoughts, sorry.

soredake commented 1 year ago

I've found my issue, lychee is for some reason using Hamachi network instead of regular, when disable hamachi lychee works as expected, the question is, why newer versions use hamachi network?

mre commented 1 year ago

Congrats on figuring this out!

We don't use hamachi anywhere specifically. For requests, we just depend on reqwest and I checked but they also don't use hamachi. Maybe it's a proxy config? Also check your hamachi VPN settings. It's weird that only lychee uses it. πŸ€”

mre commented 1 year ago

In any case, I'm glad it's not a bug in lychee and you're able to check links again. πŸ˜‰

soredake commented 1 year ago

Digging deeper into this i found this article https://cloudrun.co.uk/windows10/set-network-interface-priority-in-windows-10-using-set-netipinterface/, hamachi interface was selected because it was higher priority due to providing 10 Gbps versus my 100 Mbps main connection. Maybe something was changed in reqwest that makes it use windows priority for selecting interface, i will report this to them later.

mre commented 1 year ago

I would guess that if it's configured somewhere at all then it would be in hyper, not reqwest. But honestly I doubt it. In any case, nice find and I'd be thrilled to know the answer. Please keep us posted. 😊

soredake commented 1 year ago

https://github.com/hyperium/hyper/issues/3122 reported to hyper.