thp / urlwatch

Watch (parts of) webpages and get notified when something changes via e-mail, on your phone or via other means. Highly configurable.
https://thp.io/2008/urlwatch/
Other
2.85k stars 349 forks source link

Ignore 'Connection aborted' / 'Connection Reset by peer' errors #623

Open runa opened 3 years ago

runa commented 3 years ago

Hi! I'm using urlwatch to monitor changes on a site that is not yet up, so I'm getting "Connection reset by peer" errors (port is open, but someone closes the request without returning a response.

I would like to ignore this error; but ignore_connection_errors: true doesn't seems to catch it.

===========================================================================
01. ERROR: API covid
===========================================================================

---------------------------------------------------------------------------
ERROR: API covid (https://foo)
---------------------------------------------------------------------------
('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
---------------------------------------------------------------------------

--
urlwatch 2.17, Copyright 2008-2019 Thomas Perl
Website: https://thp.io/2008/urlwatch/
watched 2 URLs in 0 seconds

This is the output of curl on this URL

% curl -vvvv https://foo/
*   Trying 200.16.89.120:443...
* TCP_NODELAY set
* Connected to foo (x.x.x.x) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 /GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: 
*  start date: May 26 00:00:00 2020 GMT
*  expire date: Apr  6 23:59:59 2021 GMT
*  subjectAltName: host ",,,"
*  issuer: 
*  SSL certificate verify ok.
> GET / HTTP/1.1
> Host: foo
> User-Agent: curl/7.68.0
> Accept: */*
>
* OpenSSL SSL_read: Connection reset by peer, errno 104
* Closing connection 0
curl: (56) OpenSSL SSL_read: Connection reset by peer, errno 104
mborsetti commented 3 years ago

It should work since ConnectionResetError is a subclass of ConnectionError per documentation. However, without being able to reproduce the behavior you're seeing, it's impossible to fix.

$ curl -vvvv https://foo/
* Could not resolve host: foo
* Closing connection 0
curl: (6) Could not resolve host: foo

It would be great if you could share a reproducible example!