Group result in summary from multiple "run"

[Alphapage]

Hello, I can't find a way to made a global summary from multiple run:

#[tokio::main]
async fn main() {
    let result1=World::run("tests/features/readme.feature").await;
    let result2=World2::run("tests/features/readme2.feature").await;
    // cucumber::summarize([result1,result2]);
}

For now, I get 2 summaries in the terminal. Is there a way to get a final global merged summary ?

Thank you in advance for your answer.

[ilslv]

Unfortunately we don't have similar feature right now. Can you share a use case that drove you to use 2 separate cucumber runs?

[Alphapage]

Ok, I must create a global world... Thank you +1

[RobbieMcKinstry]

Hello! 👋🏻 I have a use case for two separate Cucumber runs.

I'm testing a web application with multiple features, and I'm using Cucumber to make HTTP requests and check the results. I have multiple .feature files, each describing a different resource in the application.

Here's a quick example:

Feature: Users should be able to create new accounts.
  Rule: Users with valid emails and passwords should be allowed to create accounts.
    Example: This email is valid and the password is sufficiently strong.
      Given the user enters the email "robbie@email.com" 
      Given the user enters the password "HelloWor-ld213"
      When she creates an account
      Then the request should succeed

  Rule: Users' emails must be at least four characters long.
    Example: This illegal user email has three characters.
      Given the user enters the email "a@b"
      Given the user enters the password "HelloWor-ld213"
      When she creates an account
      Then the request should fail

Right now, I...

1. Make an API call for the step When she creates an account
2. Store the response on the World
3. Inspect the status code as part of the step Then the request should fail.

#[derive(cucumber::World, Debug, Default)]
pub struct NewUser {
    email: Option<String>,
    password: Option<String>,
    response: Option<Result<UserCreateSuccess, Error<CreateUserError>>>,
}

#[given(expr = "the user enters the email {string}")]
pub async fn provide_email(w: &mut NewUser, email: String) {
    w.email = Some(email);
}

#[given(expr = "the user enters the password {string}")]
pub async fn provide_password(w: &mut NewUser, password: String) {
    w.password = Some(password);
}

#[when(regex = r"^(?:he|she|they) creates an account")]
pub async fn create_account(w: &mut NewUser) {
    w.response = Some(
        create_user(
            &Configuration::default(),
            UserCreateRequest {
                account_name: w.account_name.clone().unwrap(),
                email: w.email.clone().unwrap(),
                password: w.password.clone().unwrap(),
            },
        )
        .await,
    );
}

#[then("the request should succeed")]
pub async fn request_succeeds(w: &mut NewUser) {
    let observed = w.response.as_ref().is_some_and(|resp| resp.is_ok());
    assert!(
        observed,
        "Expected OK status code, found {:?}",
        w.response.as_ref().map(|resp| resp.is_ok())
    );
}

#[then("the request should fail")]
pub async fn request_fails(w: &mut NewUser) {
    let observed = w.response.as_ref().is_some_and(|resp| resp.is_err());
    assert!(
        observed,
        "Expected failed status code, found {:?}",
        w.response.as_ref().map(|resp| resp.is_err())
    );
}

This works well for one type of request, but because the response types are all different (which is good because I want to strongly-type my API calls), the number of fields in my World grows each time I check a new API call. In particular, the field:

struct World {
    response: Option<Result<UserCreateSuccess, Error<CreateUserError>>>,
}

needs to be duplicated for each API response type:

struct World {
    user_create_response: Option<Result<UserCreateSuccess, UserCreateError>>,
    user_read_response: Option<Result<UserReadSuccess, UserReadError>>,
    user_delete_response: Option<Result<UserDeleteSuccess, UserDeleteError>>,
    user_update_response: Option<Result<UserUpdateSuccess, UserUpdateError>>,

    team_create_response: Option<Result<TeamCreateSuccess, TeamCreateError>>,
    /* .... */
}

My solution so far has been to use one world per API call, and also one .feature per API call.

// file tests/user/create.rs
#[derive(cucumber::World)]
struct CreateUserWorld {}

// file tests/user/read.rs
#[derive(cucumber::World)]
struct ReadUserWorld {}
...

However, it'd like to have one summary instead of many. That's why this feature is important to me.

Alternatively, if I could scope the state into smaller chunks (like have one World composed of multiple smaller World, that would work too. However, I noticed almost all of the functions that would allow me to implement this are parameterized by the trait bound W: World (e.g. cucumber::Cucumber, writer::Normalized), which prevents me from easily writing an aggregation function.

A side effect of my design is that it's hard to reuse steps across worlds (which I understand to be an anti-pattern). Any tips or insights would be appreciated. ❤️