apify / got-scraping

HTTP client made for scraping based on got.
422 stars 32 forks source link

Need more control over TLS #137

Closed JTJag closed 2 months ago

JTJag commented 2 months ago

Quite often it happens that tls interceptors do not bypass the protection. I checked manually and found that on some sites not only cipher suites are checked, but also the presence of GREASE and the contents of some extensions. I understand that to control this, need to rewrite the entire HTTPS agent and create your own TLS implementation. This solution will definitely be slower than native libraries. I have already achieved some results in this direction. Are you considering adding such functionality to got-scrapping?