taskset => scenario, task => step

[kazimir-malevich]

I know Goose is based on Locust and this would be a massive refactor. But I think vocabulary is important and testers would see tasksets as scenarios and tasks as steps. Is a rename worth it?

[jeremyandrews]

I think whatever names are chosen, someone will be confused if they're used to different names from a previous load testing tool. I'd prefer to see improved documentation that explains "if you have a jMeter background, you can consider a TaskSet similar to a Scenario" etc.

[kazimir-malevich]

See PR glossary

[jeremyandrews]

Doing a little more research on other load testing tools, I'm leaning more and more toward renaming GooseTaskSet to Scenario. Ideally the goal will to allow either name for a major version or two with a deprecation warning.

[kazimir-malevich]

Scenario has been used since manual testing days. I am also thinking removing tasks and having the API more granular. We make requests using HTTP verbs, we validate a 200 or 300 as a success. We make additional assertions against the HTML body. I just don't know enough as yet, but things come across as confusing. We don't need a DSL, we still want to be writing against a Rust API. DSLs confuse things.

[jeremyandrews]

Whatever they are named, the abstraction is fairly simple: Individual Requests (which indeed map to HTTP verbs) are grouped together in Tasks, which can be further grouped together in TaskSets.

I recommend taking a look at this example for a better understanding of this works, and the added flexibility it offers beyond what you're describing: https://github.com/tag1consulting/goose-eggs/tree/main/examples/umami

[kazimir-malevich]

I was thinking last night, the design is correct. You'd want to measure timings of transactions/tasks to determine bottlenecks in specific areas of the AUT, which no other load test I am aware of does. But I would suggest scenarios and transactions for naming tasksets and tasks. I need to learn rust now, which will take a few days, but feel free to assign me simple tasks to begin with. I am not intentionally criticising, I just believe this has the potential to be the best and most comprehensive load test tool around..

[jeremyandrews]

But I would suggest scenarios and transactions for naming tasksets and tasks.

The plan is to rename GooseTaskSet to Scenario (also dropping the Goose prefix), we're onboard with this change. We aim to simplify upgrades by temporarily also supporting the old naming, issuing a deprecation notice on compilation. It should be a relatively easy change, I'll dive into that after I finish reworking the internals to support test plans (see https://github.com/tag1consulting/goose/pull/422).

Renaming GooseTask to Transaction does not seem more clear as by design a Task is more arbitrary than a single transaction, but I intend to do more research to see if the majority of tools actually use this nomenclature. (And at minimum we'd also likely drop the Goose prefix, renaming GooseTask to Task)

feel free to assign me simple tasks to begin with

Happy to answer any questions about existing Issues and to provide any guidance if you dive into anything there.

I am not intentionally criticising

No need to apologize, feedback and open discussions are welcome!

[kazimir-malevich]

A transaction would be a subset of a user-journey, like logging into a website, adding to a shopping cart or checking out etc. I don't know if Goose does this but timings of the full transaction should be displayed, so we can clearly discover where bottlenecks lie in the system.

[jeremyandrews]

As a name, it makes more sense when I consider it terms of a Database Transaction.

Yes, Goose collects and reports on per-Task metrics in addition to per-Request metrics (both in ASCII and in the HTML Report and graphs). The only downside is currently each Request in a Task is executed sequentially, instead of executing several at a time like a web browser would.

[kazimir-malevich]

The only downside is currently each Request in a Task is executed sequentially, instead of executing several at a time like a web browser would.

That would be good. I will out of action for few days while I learn Rust. Thanks.

[jeremyandrews]

@kazimir-malevich https://github.com/tag1consulting/goose/pull/449 can use testing if you're able/interested

[kazimir-malevich]

Sorry for the delay. I will write a test with the new nomenclature.

[kazimir-malevich]

Looks good to me: https://github.com/kazimir-malevich/goose-load-test/commit/fd4f0ce6cd1450fc2eaade8726b5d96b7aa34ff8

 === OVERVIEW ===                
 ------------------------------------------------------------------------------                                                        
 Action       Started               Stopped             Elapsed    Users                                                               
 ------------------------------------------------------------------------------                                                        
 Increasing:  2022-04-24 18:58:40 - 2022-04-24 18:58:45 (00:00:05, 0 -> 4)                                                             
 Maintaining: 2022-04-24 18:58:45 - 2022-04-24 18:58:55 (00:00:10, 4)                                                                  
 Decreasing:  2022-04-24 18:58:55 - 2022-04-24 18:58:55 (00:00:00, 0 <- 4)                                                             

 Target host: https://www.stage.bbc.co.uk/weatherwatchers                                                                              
 goose v0.16.0-dev               
 ------------------------------------------------------------------------------ 

I will elaborate the test as Weather Watchers is quite a complex user journey, then I can give more feedback.

[jeremyandrews]

Fixed by https://github.com/tag1consulting/goose/pull/451