Orange-OpenSource / hurl

Hurl, run and test HTTP requests with plain text.
https://hurl.dev
Apache License 2.0
12.8k stars 478 forks source link

CLI option `--max-filesize` not always enforced #3245

Open lambrospetrou opened 3 hours ago

lambrospetrou commented 3 hours ago

What is the current bug behavior?

A response of 147KB Javascript file is not rejected with --max-filesize 100.

Steps to reproduce

# filename: test-filesize.hurl

GET https://unpkg.com/vue@3.4.27/dist/vue.global.prod.js
[Options]
output:test-vue.prod.js
HTTP 200

Run the above with:

hurl --test --max-filesize 100 test-filesize.hurl

The download of the file succeeds just fine.

What is the expected correct behavior?

The download should be rejected.

For other file types like images or JSON, the max filesize option applies successfully, for example:

GET https://unpkg.com/vue@3.4.27/dist/vue.global.prod.js
[Options]
output:test-vue.prod.js
HTTP 200

GET https://api6.ipify.org?format=json
[Options]
output:test-https-ipv6.json
ipv6: true
HTTP 200

GET https://cf-assets.www.cloudflare.com/slt3lc6tev37/76x52jIsr93tqZq0h3HCFW/33a7575a9dc880e0a45c0f69fcbcfc8f/cc-diagram-orange-2024.png
[Options]
output:cloudflare.png
HTTP 200

Running the above will give out:

$ hurl --report-json test-report-json --max-filesize 100 test-report-json.hurl
error: HTTP connection
  --> test-report-json.hurl:13:5
   |
13 | GET https://cf-assets.www.cloudflare.com/slt3lc6tev37/76x52jIsr93tqZq0h3HCFW/33a7575a9dc880e0a45c0f69fcbcfc8f/cc-diagram-orange-2024.png
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (63) Maximum file size exceeded
   |

With a low enough max filesize (e.g. 10) it will also fail the second JSON request. Instead of failing at the first entry for the JS file.

Execution context

Possible fixes

lambrospetrou commented 2 hours ago

I realized now that curl (curl 7.81.0) also does not fail for that file...

curl --max-filesize 10 https://unpkg.com/vue@3.4.27/dist/vue.global.prod.js

According to https://everything.curl.dev/usingcurl/downloads/max-filesize.html though, it should stop after it exceeds the max filesize even if it doesn't know it upfront. But it doesn't.

edit: According to https://curl.se/docs/manpage.html#--max-filesize after curl 8.4.0 it should be rejected:

NOTE: before curl 8.4.0, when the file size is not known prior to download, for such files this option has no effect even if the file transfer ends up being larger than this given limit.

Starting with curl 8.4.0, this option aborts the transfer if it reaches the threshold during transfer.

It seems though that Hurl uses libcurl/7.81.0 (based on hurl --version).

So, is there a way to update the version used by Hurl, or does that depend on my system's version of curl?