POST body parsing fails

[swaggboi]

I have the following route to process a POST form submission:

$router.post(-> $request, $response {
    my Str $return-url = $request.content.{'hyperlink'};
    my Str $url-scheme = $request.headers.{'X-Forwarded-Proto'} || 'http';
    my Str $url-host   = $request.headers.{'Host'};
    my Str $base-url   = $url-scheme ~ '://' ~ $url-host ~ '/';
    my Str $hyperlink  = $base-url ~ encode-base64(gzip($return-url), :str);

    $response.html($templates.process: 'index', :$hyperlink);
});

This seems to work but today I noticed there's a couple strings that it won't accept, for example if I submit https://youtube.com/watch?v=xvFZjo5PgG0 I get the following in my console:

[Context: 91141] | [Time: 2023-10-31T21:51:07.598590-04:00] | POST | / | HTTP/1.1 | Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0
Failed trying to parse a body of type application/x-www-form-urlencoded
  in block  at /home/daniel/.raku/sources/1F9F48EAAE7A585D01855E91A4F8E592CF5AB1E1 (Humming-Bird::Core) line 123 
[Context: 91141] | [Time: 2023-10-31T21:51:07.628344-04:00] | 500 Internal Server Error | / | text/html

I thought maybe the question mark/query string confused it but I've tried other URLs with query strings and they work, it's just this link and https://youtube.com/watch?v=oHg5SJYRHA0 it doesn't seem to like so far.

[rawleyfowler]

I'm guessing this is a bug with the ? as you suspect. Try printing the body of the request rather than content, to see what it looks like to make sure. The parsing from body to content is handled by another Raku module so it may be a big with that as well.

[swaggboi]

I'm getting Nil it looks like:

[Context: 54286] | [Time: 2023-10-31T22:26:16.115241-04:00] | POST | / | HTTP/1.1 | Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0
Failed trying to parse a body of type application/x-www-form-urlencoded
  in block  at /home/daniel/.raku/sources/1F9F48EAAE7A585D01855E91A4F8E592CF5AB1E1 (Humming-Bird::Core) line 123
Nil
Failed trying to parse a body of type application/x-www-form-urlencoded
  in block  at /home/daniel/.raku/sources/1F9F48EAAE7A585D01855E91A4F8E592CF5AB1E1 (Humming-Bird::Core) line 123
[Context: 54286] | [Time: 2023-10-31T22:26:16.215051-04:00] | 500 Internal Server Error | / | text/html

Here's what I get for just https://www.youtube.com:

[Context: 87200] | [Time: 2023-10-31T22:28:50.166111-04:00] | POST | / | HTTP/1.1 | Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0
https://youtube.com
[Context: 87200] | [Time: 2023-10-31T22:28:50.186918-04:00] | 200 OK | / | text/html

[swaggboi]

This is what I see in the POST body according to Firefox dev tools:

hyperlink=https%3A%2F%2Fyoutube.com%2Fwatch%3Fv%3DxvFZjo5PgG0

[rawleyfowler]

Hmm what process are you going through to make the request? It says url form encoded but that body does not look url encoded to me.

[rawleyfowler]

Hmm I will play around with this this evening. Not 100% sure if this is a bug with Humming::Bird or, URI::Encode.

[swaggboi]

I dropped it into the first result for 'URL decoder' and it decoded fine. It's just a web form rendered to the web browser:

<form method="post">
  <label for="hyperlink">Hyperlink</label>
  <input id="hyperlink" name="hyperlink" type="text" minlength="4"
         maxlength="2097152" required>
  <button type="submit">Process</button>
</form>

I can reproduce with curl, I tried commenting out the gzip stuff but then it just complains when the base64 stuff is run. I'm thinking about trying a different Raku version/distro I've only tried this on Debian 12 with the version it ships with:

daniel@netburst:~$ raku --version
Welcome to Rakudo™ v2022.12.
Implementing the Raku® Programming Language v6.d.
Built on MoarVM version 2022.12.

[rawleyfowler]

That version should be fine, but I would recommend using rakubrew to install the latest.

[rawleyfowler]

Also @swaggboi can you verify the version of Humming-Bird you are running?

[swaggboi]

2.1.5

[rawleyfowler]

@swaggboi I figured it out. A very small bug in the URL-encoded pre-parser. I'll have it PR'd and published asap. Good find!

[swaggboi]

I cannot tell you how relieved I am this wasn't me doing something dumb/wrong 💪

[rawleyfowler]

I was actually warned of this happening last year, but hadn't seen it break anything until today lol. So next time i'll listen :D

[rawleyfowler]

@swaggboi It will take a bit for it to hit the Zef ecosystem. So in the meantime, you can run

zef install https://github.com/rawleyfowler/Humming-Bird --force-install

to get the latest version.