add Job-Matrix support

[maxklenk]

When I create Jobs I see myself copying the same Job over and over while I only change one parameter in each copy. To get an idea of the performance of a site my setup most often looks like this:

• X Locations (e.g Continents)
• Y Pages/Scripts
• Z Connections

I end up creating X Y Z Jobs to test all combinations. A Job-Matrix could work like the normal Job-creation with the only difference that the three mentioned parameters would be multi selects. As a start the Job-Matrix could simply create all the Jobs, but to be able to update them easily afterwards a persistent Job-Matrix entity would be even better.

What do you think about this feature? Is there another solution for my problem?

[sburnicki]

Being able to generically define a job to run with multiple locations and connections makes sense. In our case, we mostly have one script for a specific job group, so here it wouldn't make sense to also define mutliple scripts to run.

How do you use scripts so that different measurements should end up in the same job group. Do you simply differentiate page names? Would be nice to get an insight on your workflow.

I'm afraid the only possibility for you to do this right now is to edit an existing job, modify it and save it as a copy. Still much work, but at least you don't need to start from scratch ;)

[maxklenk]

My scripts are very simple, mainly because I use them to define the page name. I was not sure how the usage of page names was planned, because the ::: syntax was not documented. Most of my scripts just looks like this:

setEventName landing:::someName
navigate https://www.domain.tld/

Multiple Jobs then share the same Job Group, so I can aggregate them in the visualizations.

How do you use the scripts?

[nilskuhn]

We measure multiple pages with one script. So for example to measure a typical customer journey in german apple store:

setEventName homepage_entry:::Apple_homepage_entry
navigate https://www.apple.com/de/
setEventName category:::Apple_category_accessories
navigate http://www.apple.com/de/shop/mac/mac-accessories
setEventName productList:::Apple_productList_mac
execAndWait document.querySelector('.ac-gn-link-mac').click();
setEventName search:::Apple_search
navigate http://www.apple.com/de/search/earpods?src=globalnav
setEventName productDetails:::Apple_productDetails
navigate http://www.apple.com/de/shop/product/MD827ZM/B/apple-earpods-mit-fernbedienung-und-mikrofon

The benefit of this approach is that local browser cache fills up in a realistic way while walking through the application. Scripting your tests like real users customer journeys is a better approach than webpagetests single step measurement first and repeated view in our opinion. So we mostly have one JobGroup per application. if we need to measure more than one page as entry page we have some additional scripts for that.

That's the reason for using atributes location and connectivity for a job matrix sounds reasonable to us but not necessarily the page.

[maxklenk]

Thanks for sharing how you simulate customer journeys. This definitely creates more insights than checking only the entry page.

If you want to test multiple entry pages it would make sense to add the different scripts to the matrix as well, while for sure you can only add a single script too.