search

WordPress / phpunit-test-reporter

GNU General Public License v3.0
29 stars 21 forks source link

Support multiple PHP versions, improve display #109

Closed swissspidy closed 2 months ago

swissspidy commented 8 months ago

To-do:

javiercasares commented 8 months ago

FYI: https://github.com/WordPress/phpunit-test-runner/pull/212

desrosj commented 2 months ago

I keep getting hung up on the environments concept. I am thinking that we need to remove that for this round and give it some more thought.

For example, environment name. At first added a taxonomy for Environment Type because some suggested values for multi-environment was "vps", "cloud", etc.. That feels more appropriate as a taxonomy. But I think that environment name could also make sense for something that's more like the name of a hosting package, maybe Basic, choice plus, Online Store/e-commerce. This would vary reporter to reporter, but could allow them to create their own groups.

But what is an environment? Does a different PHP version indicate a different "environment" if everything else is the same? Is it only if the host considers it a different environment?

I've opened https://github.com/swissspidy/phpunit-test-reporter/pull/2 that removes the display of environment counts for now, and also has some improvements to how the report outcomes are counted.

swissspidy commented 2 months ago

But I think that environment name could also make sense for something that's more like the name of a hosting package, maybe Basic, choice plus, Online Store/e-commerce. This would vary reporter to reporter, but could allow them to create their own groups.

Yes, I think that's what hosts will use it for. So there can be lots of different values.

Is it only if the host considers it a different environment?

I'd say so. Hosts all have so many different setups and offerings, they know best what to call them and how to differentiate that.

We could also call it "label" or "product", but "environment" is probably fine as is.

desrosj commented 2 months ago

I think this is in a really great spot.

Talking with @swissspidy at WCUS Contributor Day, and we decided to remove author archives for test results for this iteration.

I also added a database version taxonomy in addition to the PHP version one. In combination with environment name, this helps ensure reports are not overwritten unintentionally.

swissspidy commented 2 months ago

@desrosj Let's 🚢 it

Crixu commented 2 months ago

oh nice, thank you for the work!