tag1consulting / goose

Load testing framework, inspired by Locust
https://tag1.com/goose
Apache License 2.0
813 stars 71 forks source link

Error when generating HTML report #533

Closed Vincsz closed 1 year ago

Vincsz commented 1 year ago

Hi,

I have an issue when running my Goose loadtest. The HTML report can't be generated because of the following error : thread 'main' panicked at 'called Option::unwrap() on a None value', /Users/vincau/.cargo/registry/src/github.com-1ecc6299db9ec823/goose-0.17.0/src/graph.rs:425:41

I was able to run my test and generate the report until last month, it seems that the report can't be generated but the API calls are executed.

Command executed : cargo run --release -- --host https://my_api --report-file=report.html -u 750 -r 500 --throttle-requests 1125 -t 60s

Am I missing something new ?

Thanks, Vincsz

jeremyandrews commented 1 year ago

Looking at that line of the code, it seems this could panic if data is empty. Are any requests actually being made? Can you past the full output, for example are there command line metrics being displayed?

Vincsz commented 1 year ago

Ok, my API calls are failing :

=== PER SCENARIO METRICS ===
 ------------------------------------------------------------------------------
 Name                     |  # users |  # times run | scenarios/s | iterations
 ------------------------------------------------------------------------------
 1: LoadTest API Receiver |      750 |        57100 |      951.67 |      76.13
 ------------------------------------------------------------------------------
 Name                     |    Avg (ms) |        Min |         Max |     Median
 ------------------------------------------------------------------------------
   1: LoadTest API Rece.. |      788.78 |        567 |         812 |        567

 === PER TRANSACTION METRICS ===
 ------------------------------------------------------------------------------
 Name                     |   # times run |        # fails |  trans/s |  fail/s
 ------------------------------------------------------------------------------
 1: LoadTest API Receiver
   1:                     |        57,850 |  57,850 (100%) |   964.17 |  964.17
 ------------------------------------------------------------------------------
 Name                     |    Avg (ms) |        Min |         Max |     Median
 ------------------------------------------------------------------------------
 1: LoadTest API Receiver
   1:                     |      783.96 |         11 |         812 |         80
 ------------------------------------------------------------------------------
 Slowest page load within specified percentile of requests (in ms):
 ------------------------------------------------------------------------------
 Name                     |    50% |    75% |    98% |    99% |  99.9% | 99.99%
 ------------------------------------------------------------------------------
 ------------------------------------------------------------------------------
 Name                     |                                        Status codes 
 ------------------------------------------------------------------------------
 -------------------------+----------------------------------------------------

I'm pretty sure that previously when I had that kind of issue I was able to get an empty report showing at least the status codes. Even at warm up, I don't get any status code and my api gateway logs confirm that no call was received.

I'm trying to call my API using a POST call. I do it this way :

async fn loadtest_index(user: &mut GooseUser) -> TransactionResult {
    let params = &serde_json::json!({my_json});
    let request_builder = user.get_request_builder(&GooseMethod::Post, "")?.form(&params).header("x-api-key", "my_api_key");

    let goose_request = GooseRequest::builder().set_request_builder(request_builder).expect_status_code(200).build();

    let _goose = user.request(goose_request).await?;
    Ok(())
}

Could my JSON be responsible for this fail ? I guess I would get an explicit error if my JSON wasn't correct (400 Bad Request ?).

jeremyandrews commented 1 year ago

Goose shouldn't panic, so there's definitely a bug here we need to track down/fix.

As for your specific issue, perhaps enable the debug log.

That should show you what your server is responding that you don't expect.

(I also always enable the error log ).

Vincsz commented 1 year ago

The error log and the debug log files are generated but they stay empty (0kb, as the report.html file btw), the process seems to panic before the logs are written.

jeremyandrews commented 1 year ago

Due to the bug you are reporting, temporarily disable --report-file=report.html and re-run: this should allow you to collect the debug log.

Vincsz commented 1 year ago

I tried to modify my json input, and I was able to get errors and the report file. So I guess I can't just copy/paste a json string in a variable.

Reading json from a file this way doesn't work either, but as I'm new to Rust... let params = &serde_json::json!(std::fs::read_to_string("body.json").expect("Unable to read file"));

jeremyandrews commented 1 year ago

fixed by #555