search

flathunters / flathunter

A bot to help people with their rental real-estate search. 🏠🤖
GNU Affero General Public License v3.0
830 stars 179 forks source link

Does the Geetest Captcha resolver actually work (Immobilienscout24) #502

Closed LuanG01 closed 10 months ago

LuanG01 commented 10 months ago

Hey,

I just tried to build my own web-scraper for immobilienscout24 and wanted to solve the geetest captcha with the help of 2captcha. However, I ran into the problem that I couldn't find out where Immobilienscout24 gets its challengeKey from Geetest.

If I understand the concept correctly, the generated challengeKey that is visible on the page is no longer valid and you always have to generate a new one (see: https://2captcha.com/de/p/geetest).

Unfortunately, I'm not a python expert, but if I understand the code from flathunters correctly, the code only takes the challengeKey from immoscout, which in my opinion should be invalid.

So does the Geetest solver work here at all? If so, am I misunderstanding the code and it draws a new challengeKey every time or am I misunderstanding the principle of the challengeKey?

Please help me!:)

codders commented 10 months ago

Hi @LuanG01,

The captcha-solving in Flathunter is a little bit magical. It's had so many contributions over the time that I don't think any one person understands it all. But to try and answer your question, when we crawl the Immoscout page, we have the browser setup to drop all HTTP requests to the Geetest API. Because the requests are blocked during the first fetch of the page, the challengeKey we received is still valid, and we can use that to submit the captcha response.

Hope that solves it for you.

LuanG01 commented 10 months ago

Thanks, that helped a lot!