Setup a playground-lighthouse application to measure our lighthouse score.

[peterp]

https://developers.google.com/web/tools/lighthouse

[renansoares]

hi @peterp, can I have more information about this issue? I'm wondering if I can help with this.

[thedavidprice]

Hi @renansoares! Thanks for jumping in here. I don't know if we actually know more about this. Overall we'd like to be able to do the following:

• benchmark Redwood performance
• understand how Redwood performs differently on various hosting providers (e.g. Netlify and Vercel)
• have awareness + visibility when current performance changes related to the benchmarks (both up and down)

But, to be honest, we're not tracking or pay much attention to this a lot of the time. So unclear about 1) how much time this would take compared to 2) the value it would add.

Currently, we've just been doing this occasionally using projects like the Example Blog deployed on Vercel and Netlify.

Do you have any experience here?

[renansoares]

Hi @thedavidprice, thanks for your response.

I have some experience with Lighthouse and I am looking for something to help next.

I see 3 ways to analyze web core vitals metrics:

1. Lighthouse browser: the one which runs in Google Chrome
2. Lighthouse CI: a library to run lighthouse against a page and generates the report (the same generated by the browser). This could be added to the CI, I have added this to the product I am working on and it's a cool way to track the performance changes. Lighthouse Server is a server that uses the metrics of Lighthouse CI to generate charts and make it easier to track the performance changes. There is also an option to add a Git Action with Lighthouse.
3. Web Vitals: a library that allows getting the web vitals metric in the browser during execution time.

I'm thinking about how we could use these libraries to help users to measure performance, and make Redwood collaborators aware of the framework performance.

1. Create a project using create-react-app (documentation) it is by default is installing the web-vitals library. Is this something that we would like to add to RedwoodJS too and create documentation or hooks to allow users to measure?
2. Add Lighthouse CI in any way to the RedwoodJS project? Maybe to an example project and generate the report for different providers like Vercel and Netlify and make it available online?
3. Add some initial Performance page in the documentation describing the importance of using Lighthouse and the ways to do it.

Let's discuss it. I'm happy to contribute if I can 🎉

[thedavidprice]

Wow, lots of great ideas here. And a "yes" to moving something forward. I'll do my best to identify priority, actionable next steps. But also know I can become a bit of a bottleneck as I'm inconsistent in my focus outside Redwood v1 roadmap priorities. That said, if you feel momentum is slipping, there's nothing that gets momentum flowing again like a draft PR 😆 No permission needed.

High Level Goal

--> What would be our goal for Lighthouse measuring + monitoring?

I'd suggest these:

• establish realistic benchmarks (that are "honest")
• monitor ongoing to prevent degradation of performance from release to release
• establish documentation identifying things we've learned regarding best practices and ways to further improve performance (when applicable)

What I don't want to do is fall into the rabbit-hole of chasing Lighthouse 100s and/or making Lighthouse scores the focus of performance. I think it's a very informative and useful tool. I don't think it should ever become "the point".

This needs more discussion. So what say you all?

"Performance" Documentation

I like this idea a lot and suggest it be the first step in this project. If you agree, how about creating an RFC Issue with an initial outline of what could become redwoodjs.com/docs/performance. There's a lot more to come (and do) by way of performance, which includes the use of Prerender that could be referenced (currently it's own stand-alone doc). But what about SEO, OG data, caching, end-user monitoring and analytics, etc. Definitely not saying we start with all these — just adding ideas to what a Performance doc could become.

For now, I think referencing something like web-vitals (which looks really interesting) with usage instructions in the Performance doc would be the right first start. I don't think we'd be ready to add that to the codebase.

CI + Lighthouse

This is a very interesting idea and could possibly be implemented in two ways:

1. Add to current E2E CI checks
2. Automate Redwood project deployment to supported host providers and provide some kind of reports for performance for each

Add to Current E2E CI

Take a look at our current GitHub Action E2E runner and the corresponding Cypress E2E here.

It spins up a new project installation and runs through steps in the tutorial. Given some code mods, do you think we could also include some kind of benchmarking + output for Lighthouse against this project? The advantage is that this runs against every PR.

Eventually, we'd like to be monitoring an assortment of performance-related stats with each PR. Take a look at how Next.js does it via the example here. Maybe Lighthouse could be the beginning of this kind of automated performance benchmarking for Redwood?

Automate Deployment Testing

I haven't even begun this project yet, but eventually, I'd like to do something like the following:

• have a separate repo with a directory for each supported Redwood Deployment target (currently Netlify, Vercel, and AWS Lambdas with many more on the way)
• each directory would contain an identical, canonical Redwood project with the corresponding deployment target config
  • we'd need to be able to build all these projects (and apply code mods with each new version) and then run the corresponding yarn rw setup deploy ... command
• with each new canary release, the projects would be upgraded to the latest canary version and deployed
• we'd monitor deployment, API endpoints, maybe Authentication, GraphQL security, and Web end-user performance

It's a big project and this is the first time I've sketched out what's in my head. No expectations. But hoping it'll get some feedback and, especially, suggestions about how to improve the overall design + figure out possible next steps.

[renansoares]

You just split up all the topics beautifully. I have some comments about each topic.

High Level Goal

I agree with all points you have mentioned.

The idea is to have benchmarks reflecting how people are using Redwood. Having those benchmarks, the community could refer to as a source to make architectural decisions according to each product's requirements. Also, it helps to track performance degradation.

'Performance' documentation

I agree. I will start an RFC for redwoodjs.com/docs/performance. A good start point might be introducing Lighthouse and the Web Vitals library and refer to the prerendering functionality. There are many (perceived) performance subjects (such as virtualization, pagination, animations, examples of common UI performance bottlenecks) that could be added along the way.

In my experience using Lighthouse, it helped me to improve even SEO and caching. I had the reports telling me that my page was missing SEO tags, that I should enhance accessibility tags, and the API caching policy was not well configurated. So, Lighthouse includes many of these concepts that you could start from and dig in to understand better.

Lighthouse CI

Current CI

Lighthouse can be added like one of those runners that the repository currently has. I'm thinking about the benefits of having Lighthouse running in every PR in the main repository. In my opinion, we will be most interested in the Web Vitals Metrics that Lighthouse gives us. How useful these metrics in every PR is an open question for me. It helps me in my job because my team is aware that it exists, can track and see it in every PR, maybe the same?

Deployment testing

This is great; I like so much this idea of having all of those benchmarks. Those benchmarks should monitor essential things like all you have mentioned and give us important metrics.

This test deployment can also highlight the importance of some features like prerendering that reduces the Largest Contentful Paint (LCP).

Would it be helpful to reflect how users are currently using the framework? Maybe it can guide them in the decision-making process? Like if we have more than one fetch, the performance in those scenarios.

[thedavidprice]

Relevant to site performance and search performance (not sure where this fits with Lighthouse):

• https://web.dev/vitals/

Helpful overview article from Vercel blog: https://vercel.com/blog/core-web-vitals

[Josh-Walker-GM]

I tested this out and added a basic lighthouse ci step to the CI workflow. To take it further and have the data persist so that comparisons can be made between commits or the trend between releases would require a dedicated ci server. This server can then accept the lighthouse results and provides the interface.

@thedavidprice Is this something we would still be interested in the near term given that requirement?