search

lwthiker / curl-impersonate

curl-impersonate: A special build of curl that can impersonate Chrome & Firefox
MIT License
3.64k stars 243 forks source link

Failing to curl https://archive.today #235

Open karl-saville opened 2 weeks ago

karl-saville commented 2 weeks ago

I can no longer scrape archive.today, or its (many) aliases using curl-impersonate, using either the pre-built binaries or the pre-built docker images with ANY of the contained browser impersonators.

Sample -v output:

~/Temp/CurlImpersonate$ docker run --rm lwthiker/curl-impersonate:0.6.1-chrome curl_chrome110 -v -v https://archive.ph/lxPIW> /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 192.124.216.120:443...
* Connected to archive.ph (192.124.216.120) port 443 (#0)
* ALPN: offers h2,http/1.1
* Cipher selection: TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256,ECDHE-ECDSA-AES128-GCM-SHA256,ECDHE-RSA-AES128-GCM-SHA256,ECDHE-ECDSA-AES256-GCM-SHA384,ECDHE-RSA-AES256-GCM-SHA384,ECDHE-ECDSA-CHACHA20-POLY1305,ECDHE-RSA-CHACHA20-POLY1305,ECDHE-RSA-AES128-SHA,ECDHE-RSA-AES256-SHA,AES128-GCM-SHA256,AES256-GCM-SHA384,AES128-SHA,AES256-SHA
* ALPS: offers h2
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
  0     0    0     0    0     0      0      0 --:--:--  0:01:00 --:--:--     0* BoringSSL SSL_connect: SSL_ERROR_SYSCALL in connection to archive.ph:443 
  0     0    0     0    0     0      0      0 --:--:--  0:01:00 --:--:--     0
* Closing connection 0
curl: (35) BoringSSL SSL_connect: SSL_ERROR_SYSCALL in connection to archive.ph:443
curl-impersonate-v0.6.1$ ./curl_safari15_5 -vv --cacert /etc/ssl/certs/ca-certificates.crt --capath /etc/ssl/certs  https://archive.ph/lxPIW> /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 192.124.216.120:443...
* Connected to archive.ph (192.124.216.120) port 443 (#0)
* ALPN: offers h2,http/1.1
* Cipher selection: TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384:TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256:TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384:TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256:TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256:TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA:TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA:TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA:TLS_RSA_WITH_AES_256_GCM_SHA384:TLS_RSA_WITH_AES_128_GCM_SHA256:TLS_RSA_WITH_AES_256_CBC_SHA:TLS_RSA_WITH_AES_128_CBC_SHA:TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA:TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA:TLS_RSA_WITH_3DES_EDE_CBC_SHA
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
  0     0    0     0    0     0      0      0 --:--:--  0:01:00 --:--:--     0* BoringSSL SSL_connect: SSL_ERROR_SYSCALL in connection to archive.ph:443 
  0     0    0     0    0     0      0      0 --:--:--  0:01:00 --:--:--     0
* Closing connection 0
curl: (35) BoringSSL SSL_connect: SSL_ERROR_SYSCALL in connection to archive.ph:443
karl-saville commented 2 weeks ago

Same issue with a freshly built docker image :(