tag1consulting / goose

Load testing framework, inspired by Locust
https://tag1.com/goose
Apache License 2.0
813 stars 71 forks source link

[Question] Recommended way to handle errors? #537

Open lukehsiao opened 1 year ago

lukehsiao commented 1 year ago

I'm working on a load test, and am wondering what the right way to handle errors is. For example, suppose I have a load test which logs in to a service to get a JWT, and sets that in the session. Then, the next transactions use .get_session_data().

If I use get_session_data_unchecked, I'll see panics like

thread 'tokio-runtime-worker' panicked at 'Missing session data!', /home/lukehsiao/.cargo/registry/src/github.com-1ecc6299db9ec823/goose-0.17.0/src/goose.rs:985:28

So, I'd like to handle the error more gracefully. I assume the root cause here is that the service I am load-testing timed-out on the initial log-in transaction, thus failing to set the session data.

However, the transaction functions all return a TransactionResult, and the TransactionError they box seem quite specific: https://docs.rs/goose/latest/goose/goose/enum.TransactionError.html

Is there an example of how to use the non-panicing get_session_data and bubble up the error, or otherwise handle it gracefully?

Should I just return Ok(()) immediately, skipping the following requests, which require the JWT? Will doing so result in one of my GooseUsers running a scenario's transactions over and over that don't actually do anything?

lukehsiao commented 1 year ago

I see this on the best practices page:

When writing load tests, avoid unwrap() (and variations) in your transaction functions -- Goose generates a lot of load, and this tends to trigger errors. Embrace Rust's warnings and properly handle all possible errors, this will save you time debugging later.

So, I suppose I'm largely asking what the right way to properly handle errors is.

For example, in my particular case, my load test is just like

#[tokio::main]
async fn main() -> Result<(), GooseError> {
    GooseAttack::initialize()?
        // In this example, we only create a single scenario, named "WebsiteUser".
        .register_scenario(
            scenario!("WebsiteUser")
                // This transaction only runs one time when the user first starts.
                .register_transaction(transaction!(website_signup).set_on_start())
                .register_transaction(transaction!(authenticated_index)),
        )
        .execute()
        .await?;

    Ok(())
}

Where authenticated_index might panic if it is using get_session_data_unchecked. I think ideally, I could raise an error, and cause the website_signup to be tried again to set the session. But, I just don't know what the Goose-recommended pattern is.

onichandame commented 1 year ago

I'd like TransactionError to have another variant able to take any error. For example TransactionError::Unknown(String). This would allow it to integrate well with all sorts of external libraries, like anyhow.