Make runs comparable - Githubissues

harlan-zw / unlighthouse

Scan your entire site with Google Lighthouse in 2 minutes (on average). Open source, fully configurable with minimal setup.

https://unlighthouse.dev

MIT License

3.82k stars 111 forks source link

Make runs comparable #20

Open LukaHarambasic opened 2 years ago

LukaHarambasic commented 2 years ago

Clear and concise description of the problem

I would love to make runs more easily comparable.

Main use case: My website is built with Nuxt 2, and I want to test Astro and Nuxt 3. As my website isn't that complex and I would like to learn more about Astro, I would completely rewrite the two cases. To than based on the results of the unlighthouse reports, take a better decision on which one to use for production.

Other use case: Testing different hosting providers to make them easily comparable.

I'm aware that Lighthouse score don't tell the whole story, but I think for such simple use cases they can be a big help

Suggested solution

As the reports are already persisted in /.unlighthouse, I would suggest three modes for the Ui, based on the existing data. Implementing a switch which allows you to change the current mode.

Mode 1 - default: You can switch from the default mode (current implementation), where you see one report at a time. You can select which report seeing with a little dropdown menu.
Mode 2 - time-series: Scores of each metric are displayed in a graph to see how it evolves over time.
Mode 3 - compare: A split mode where you can open two different reports to compare them side by side.

Optional: The CI tool could also include a cronjob that runs on a user-defined schedule to create the reports. The user could then define a threshold for each metric to send a message to Slack / Discord / ... when it doesn't meet the requirements.

Alternative

No response

Additional context

PS: I haven't done something in this area, but I would love to implement something like this.

Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guide.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

harlan-zw commented 2 years ago

Hey @LukaHarambasic

Some great ideas in this issue. Some of the functionality you're describing can be accomplished in using LHCI server, however, the documentation to get that setup does not exist (though should be possible).

I'll be starting on v1 of Unlighthouse maybe next month, I've included your issue as part of the plans https://github.com/harlan-zw/unlighthouse/discussions/29

I'll see what is possible once development starts