Open BelgianNoise opened 5 months ago
Could not get go-rod to work with rotating proxies. Mayb need a static proxy (1 external IP) for this to work.
https://github.com/BelgianNoise/colruyt-products-scraper/commit/2dcff9fefea95ffcf81ffe69453a92c12c45f0a1 circumvents this issue for now. (Quick fix) Might look in to it later if this workaround stops working.
Issue
The headless chrome browser doesn't seem to work in gh actions. For now I assume it gets served the bot prevention page from colruyt.
Notes
Is gh IP address' range blacklisted ?
Possible solution:
Configure go-rod to use SSL proxies as well.