sergot / http-useragent

Web user agent class for Perl 6.
MIT License
36 stars 39 forks source link

Timeout doesn't work if TCP connection is established but HTTP server delays returning content #234

Open ichalov opened 3 years ago

ichalov commented 3 years ago

E.g. I have this PHP script on the server:

<?php

sleep(200);

echo "test\n";

Then the following script returns content in 200 seconds and not 10 or 180:

use HTTP::UserAgent;

my $ua = HTTP::UserAgent.new;
$ua.timeout = 10;

my $start = DateTime.now;
my $r = $ua.get( "http://<<server>>/test.php" );
if $r.is-success {
    say $r.content;
} else {
    die $r.status-line;
}
say DateTime.now - $start;

perl5 LWP::UserAgent handles this case just fine (though it had issues with DNS timeouts and maybe non-responsive TCP ports IIRC).

I think I was able to work it around like:

    my $r;
    my $p = Promise.anyof(
      Promise.in( $ua.timeout );
      start { $r = $ua.get( $url ); }
    );
    await $p;

It would be good to insert similar code in the library itself but I understand it may have unexpected side effects. This timeout seems to be important in cron scripts - if the server stops responding in time (e.g. due to DB locking) then client host RAM starts to be consumed by increasing number of hanging instances of the cron script.