Odd interaction of HTTP::UserAgent and Promises

p6rt commented 6 years ago

Migrated from rt.perl.org#133057 (status was 'open')

Searchable as RT133057$

p6rt commented 6 years ago

From @briandfoy

I have this little program where I want to fetch web thingys concurrently. There's a problem with the combination of HTTP::UserAgent and Promises although I don't know which one doesn't work:

#!perl6

use HTTP::UserAgent; my $ua = HTTP::UserAgent.new: :useragent\<chrome_linux>;

my @urls = \< http://www.perl.org http://www.perl6.org http://www.yahoo.com >;

loop { my @promises; for @urls -> $url { @promises.push: Promise .start({ $ua.get: $url }) .then({ put "Got {.result.^name}" }); } await @promises; }

Running this program sometimes works a little and often if fails in diverse and curious ways. I couldn't get it to fail unless I used a Promise:

$ perl6 cycle.p6 Got HTTP::Response Segmentation fault: 11

$ perl6 cycle.p6 Got HTTP::Response Illegal instruction: 4

$ perl6 cycle.p6 Got HTTP::Response An operation first awaited: in block \ at cycle.p6 line 20

Died with the exception: Tried to get the result of a broken Promise in block at cycle.p6 line 18

Original exception: Please install IO::Socket::SSL in order to fetch https sites: Failed to seek in filehandle: 22 in method get-connection at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 350 in method get-connection at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 343 in method request at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 156 in method request at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 176 in method get at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 102 in method get at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 105 in block at cycle.p6 line 17

$ perl6 cycle.p6 Got HTTP::Response moar(77217,0x70001052c000) malloc: *** error for object 0x7fd7b89a4af0: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug Abort trap: 6

Here's the setup

$ perl6 -v This is Rakudo Star version 2018.01 built on MoarVM version 2018.01 implementing Perl 6.c. $ uname -a Darwin . 17.4.0 Darwin Kernel Version 17.4.0: Sun Dec 17 09:19:54 PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64 x86_64

p6rt commented 6 years ago

From @briandfoy

Ah, there's even an HTTP::UserAgent issue for this I think: https://github.com/sergot/http-useragent/issues/191

p6rt commented 6 years ago

From simon.proctor@gmail.com

Looks to me like one of the sites is redirecting to an https page and you don't have IO::Socket::SSL installed.

On Wed, 4 Apr 2018, 1:42 am brian d foy, \perl6\-bugs\-followup@perl\.org wrote:

# New Ticket Created by "brian d foy" # Please include the string: [perl #133057] # in the subject line of all future correspondence about this issue. # \<URL: https://rt-archive.perl.org/perl6/Ticket/Display.html?id=133057 >

I have this little program where I want to fetch web thingys concurrently. There's a problem with the combination of HTTP::UserAgent and Promises although I don't know which one doesn't work:
\#\!perl6

use HTTP&#8203;::UserAgent;
my $ua = HTTP&#8203;::UserAgent\.new&#8203;: :useragent\<chrome\_linux>;

my @&#8203;urls = \<
    http://www.perl.org
    http://www.perl6.org
    http://www.yahoo.com
    >;

loop \{
    my @&#8203;promises;
    for @&#8203;urls \-> $url \{
        @&#8203;promises\.push&#8203;: Promise
            \.start$\{ $ua\.get&#8203;: $url \}$
            \.then$\{ put "Got \{\.result\.^name\}" \}$;
        \}
    await @&#8203;promises;
    \}
Running this program sometimes works a little and often if fails in diverse and curious ways. I couldn't get it to fail unless I used a Promise:
$ perl6 cycle\.p6
Got HTTP&#8203;::Response
Segmentation fault&#8203;: 11

$ perl6 cycle\.p6
Got HTTP&#8203;::Response
Illegal instruction&#8203;: 4

$ perl6 cycle\.p6
Got HTTP&#8203;::Response
An operation first awaited&#8203;:
  in block \<unit> at cycle\.p6 line 20

Died with the exception&#8203;:
    Tried to get the result of a broken Promise
      in block  at cycle\.p6 line 18

    Original exception&#8203;:
        Please install IO&#8203;::Socket&#8203;::SSL in order to fetch https
sites: Failed to seek in filehandle: 22 in method get-connection at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 350 in method get-connection at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 343 in method request at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 156 in method request at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 176 in method get at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 102 in method get at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 105 in block at cycle.p6 line 17
$ perl6 cycle\.p6
Got HTTP&#8203;::Response
moar$77217,0x70001052c000$ malloc&#8203;: \*\*\* error for object
0x7fd7b89a4af0: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug Abort trap: 6

Here's the setup
$ perl6 \-v
This is Rakudo Star version 2018\.01 built on MoarVM version 2018\.01
implementing Perl 6\.c\.
$ uname \-a
Darwin \. 17\.4\.0 Darwin Kernel Version 17\.4\.0&#8203;: Sun Dec 17 09&#8203;:19&#8203;:54
PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64 x86_64

p6rt commented 6 years ago

The RT System itself - Status changed from 'new' to 'open'

p6rt commented 6 years ago

From simon.proctor@gmail.com

There are also issues with libssl (which I think the IO::Socket::SSL module calls using Native Call) and threads.

At the moment the only threads safe https client is the Cro one, I believe.

Sorry I can't be more help, 2am insomnia should really be sleeping.

On Wed, 4 Apr 2018, 2:30 am Simon Proctor, \simon\.proctor@gmail\.com wrote:

Looks to me like one of the sites is redirecting to an https page and you don't have IO::Socket::SSL installed.

On Wed, 4 Apr 2018, 1:42 am brian d foy, \perl6\-bugs\-followup@perl\.org wrote:
# New Ticket Created by "brian d foy" # Please include the string: [perl #133057] # in the subject line of all future correspondence about this issue. # \<URL: https://rt-archive.perl.org/perl6/Ticket/Display.html?id=133057 >

I have this little program where I want to fetch web thingys concurrently. There's a problem with the combination of HTTP::UserAgent and Promises although I don't know which one doesn't work:
\#\!perl6

use HTTP&#8203;::UserAgent;
my $ua = HTTP&#8203;::UserAgent\.new&#8203;: :useragent\<chrome\_linux>;

my @&#8203;urls = \<
    http://www.perl.org
    http://www.perl6.org
    http://www.yahoo.com
    >;

loop \{
    my @&#8203;promises;
    for @&#8203;urls \-> $url \{
        @&#8203;promises\.push&#8203;: Promise
            \.start$\{ $ua\.get&#8203;: $url \}$
            \.then$\{ put "Got \{\.result\.^name\}" \}$;
        \}
    await @&#8203;promises;
    \}
Running this program sometimes works a little and often if fails in diverse and curious ways. I couldn't get it to fail unless I used a Promise:
$ perl6 cycle\.p6
Got HTTP&#8203;::Response
Segmentation fault&#8203;: 11

$ perl6 cycle\.p6
Got HTTP&#8203;::Response
Illegal instruction&#8203;: 4

$ perl6 cycle\.p6
Got HTTP&#8203;::Response
An operation first awaited&#8203;:
  in block \<unit> at cycle\.p6 line 20

Died with the exception&#8203;:
    Tried to get the result of a broken Promise
      in block  at cycle\.p6 line 18

    Original exception&#8203;:
        Please install IO&#8203;::Socket&#8203;::SSL in order to fetch https
sites: Failed to seek in filehandle: 22 in method get-connection at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 350 in method get-connection at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 343 in method request at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 156 in method request at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 176 in method get at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 102 in method get at /Users/brian/Desktop/lib/HTTP/UserAgent.pm (HTTP::UserAgent) line 105 in block at cycle.p6 line 17
$ perl6 cycle\.p6
Got HTTP&#8203;::Response
moar$77217,0x70001052c000$ malloc&#8203;: \*\*\* error for object
0x7fd7b89a4af0: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug Abort trap: 6

Here's the setup
$ perl6 \-v
This is Rakudo Star version 2018\.01 built on MoarVM version 2018\.01
implementing Perl 6\.c\.
$ uname \-a
Darwin \. 17\.4\.0 Darwin Kernel Version 17\.4\.0&#8203;: Sun Dec 17 09&#8203;:19&#8203;:54
PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64 x86_64

p6rt commented 6 years ago

From @briandfoy

The error message isn't useful because you get that no matter what happens. It's really the IO::Socket::SSL is not thread safe.

But, I'd not expect a segfault.

Raku / old-issue-tracker