search

tag1consulting / goose

Load testing framework, inspired by Locust
https://tag1.com/goose
Apache License 2.0
759 stars 69 forks source link

Request Headers #501

Closed kazimir-malevich closed 2 years ago

kazimir-malevich commented 2 years ago

I have a request that fails (403) because the "browser is not recognised". My understanding is that reqwest inserts request headers by default, however when I log to the Goose RequestLog, I am not seeing any headers "headers":[].

My question then is, will I need to create a RequestBuilder for each request to pass HeaderMaps or could this not be set at the Goose config level for every request?

jeremyandrews commented 2 years ago

See https://docs.rs/goose/latest/goose/goose/struct.GooseUser.html#method.set_client_builder for details on how you can set custom headers.

That said, also be aware of this Reqwest bug which can confuse things: https://github.com/tag1consulting/goose/issues/336

kazimir-malevich commented 2 years ago

I'll close this, I think I need to drop down to reqwest to see why it's request is being forbidden by a site that requires javascript to be enabled and how I can workaround this. It's difficult to determine what headers are being sent by the upstream bugs referenced above.

jeremyandrews commented 2 years ago

You can use something like tcpdump to view the headers sent.

kazimir-malevich commented 2 years ago

Good advice. Thanks.

kazimir-malevich commented 2 years ago

The site I am testing has a noscript tag that throws a JavaScript disabled message if JavaScript is disabled. I am not sure I can get around this with an HTTP client.

jeremyandrews commented 2 years ago

When writing load tests, you'll often need to first load pages with a real web browser with Web Developer Tool enabled. Load a page and check the developer tools to see if there are any XHR requests being made on the page, which you'll also need to simulate in your load test.

kazimir-malevich commented 2 years ago

Thanks for the advice. I'll add the tips into the documentation. There has also been some experimentation with running load tests using the chrome developer protocol as described here: https://k6.io/blog/announcing-xk6-browser-testing/.

I don't know how realistic this would be with Goose, but I think it is worth highlighting and there is a Puppeteer Rust port: https://github.com/atroche/rust-headless-chrome

matthiasg commented 4 months ago

I found this issue looking for an easy way to set some headers. Sadly this seems rather convoluted. Documentation says to use a custom client, but when trying to integrate that into an existing project I got the error that the reqwest version differs.

Now I have to create a custom project for this which is ok, just a little more work than I had hoped. Is there a specific reason why the builder does not have a header method ? Seems a pretty common request for me. Especially when load testing API endpoints instead of Web apps.