tag1consulting / goose

Load testing framework, inspired by Locust
https://tag1.com/goose
Apache License 2.0
799 stars 71 forks source link

Ability to GooseTaskError::Custom{ msg ? } #250

Open jrouaix opened 3 years ago

jrouaix commented 3 years ago

Hello,

I'm trying to implement a full story scenario in a goose task. Not sure it's the way to do that, but it allows me to reuse generated data from the server or fake data from Faker; I'd like to ensure the responses are correct, and fail in a nice way (no panic) when it does. Right now, i'm using assert! macros but i'd like to do better and use the goose machinery instead.

pub(crate) async fn user_story(user: &GooseUser) -> GooseTaskResult {
    let create_user: CreateUserRequest = Faker.fake();
    let json = serde_json::to_string(&create_user).expect("This should never fail");
    let response = user.post("/v2/user/create", &json).await?.response?;

    if response.status() == http::StatusCode::CONFLICT {
        // Sometimes, the random values collide, so it's better not to test again this user ;-)
        return Ok(());
    }

    // BAD, because it makes the task panic !
    // assert_eq!(response.status(), http::StatusCode::CREATED);
    if response.status() != http::StatusCode::CREATED {
        return Err(GooseTaskResult::Custom("Response status should be CREATED"));
    }

    // ... long story, reusing responses, username, passwords from faked data

Could we imagine having a GooseTaskResult::Custom() ?

jeremyandrews commented 3 years ago

That's an interesting idea. PR's welcome! You should be able to implement this with a PR that changes only goose.rs. Otherwise I'll try and circle back to this, as I agree it would be useful when using Goose as a library.

jeremyandrews commented 2 years ago

How/where would you consume these errors? While it's easy enough to add a custom error type, I'm not sure where you'd receive and handle it?

Currently Transaction functions are invoked here: https://github.com/tag1consulting/goose/blob/main/src/user.rs#L278

We could improve that logic to bubble up the error to the parent function, which would be here: https://github.com/tag1consulting/goose/blob/main/src/user.rs#L71

But at that point we can't bubble it up any further. So, @jrouaix, @dpc what would your goal/use-case be?

The current design would recommend you use set_failure() as that would get all information into the debug log. https://docs.rs/goose/latest/goose/goose/struct.GooseUser.html#method.set_failure

dpc commented 2 years ago

My goal would be for goose to bubble these errors all the way to the report (aggregate) and tell me that N transactions failed on valuation errors.