Metrics to measure results

[AlexDawsonUK]

Provide a mechanism for individuals to test the sustainability benefit of their implementation. Sources: EcoGrader / WebsiteCarbon

Future: Test Suites / Impact Assessments / Metrics (For individual guidelines). Any further ideas are welcome in comments below.

[notmariazoe]

Will add a note to my comment in #9 with guidelines on metrics and criteria.

[AlexDawsonUK]

Adding a note here that Digital carbon ratings for individual guidelines, if scoped correctly, and measured using accurate metrics, could be a user-friendly method to identify quick wins see https://github.com/w3c/sustyweb/issues/12.

[AlexDawsonUK]

Adding a request for this feature from @andy-blum RE https://andy-blum.com/articles/web-sustainability-guidelines/

[AlexDawsonUK]

Note: Connecting this to https://github.com/w3c/sustyweb/issues/21 as impact assessments using existing sources has been requested.

[AlexDawsonUK]

Expansion: Test Suite for WSG's.

Some W3C specs have test suites running alongside their work, if we had something similar - forming a showcase of implementation, potential measurement of results (impact), and guidelines could thereby be tested against (before/after) by machine - this could be beneficial to shifting some of the abstraction to something more technical that would feel more specification-like for our progression towards a Working Group.

• This would help with the metrics issue in terms of measurement as it could potentially offer each guideline something people could test against (impacts).
• This would also help our case for machine readability as the spec would have defined testability for every guideline (many W3C specs have a test suite).

Note: I've built complete test suites (1,000+ case) in the past for my research, and I contributed to W3C CSS 2.1's Test Suite - so working out how to produce something testable will be something I'll examine at some point.

[AlexDawsonUK]

Note: WCAG Techniques could be used as inspiration for implementation.

[AlexDawsonUK]

Note: Work is ongoing for a supplementary document containing human and machine guidance. Similar in scope to WCAG-EM but with WCAG Techniques and a Test Suite, this will offer practical advice to both implementors and tool makers.

As it is in development (a living document), expect constant evolution (additions, changes, removals), updates will be provided once it has reached a stable enough state to be considered "settled" and thus ready for inclusion and release.

[mgifford]

There are a growing list of page evaluation tools. It is probably best to give preference to open source ones like CO2.js. 

I do not know how to best choose which proprietary tools to represent. I definitely like both Ecograder and WebsiteCarbon.com

Do we want to be maintaining a wiki page of systems like the W3C does? I never want to look through giant lists like:
https://www.w3.org/WAI/ER/tools/

[AlexDawsonUK]

In the Introduction to Web Sustainability document, we do have a list of different tools which individuals can use.

The intro document was collated using all of the content of the Wiki (so has anything which didn't naturally fit into the WSGs). As such it contains books, courses, websites, tools, etc all in a handy list people can reference from.

[AlexDawsonUK]

It's been a lot of effort but its finally done! STAR now has a complete set of techniques and a test suite.

Techniques contain a link-back to the specification, a description, multiple examples, and machine reproducible steps.

The test suite is based upon the techniques and naturally won't always reflect real-world implementations but can showcase one example of a PASS or FAIL potential.

The template used for the test suite is based upon the W3C CSS Working Group's Test Suite formatting template. This should help W3C members easily get to grips with our tests and the structure works well with the content we have in our specification and supplements.

Cross-linking to STAR regarding machine testability also now exists (example) as requested in issue #29.

All of these features will be bundled into the next major milestone release.