Fine-grained reactivity for real-time report

[nicojs]

I want to introduce fine-grained reactivity to the HTML report.

Current situation

In the current situation, real-time updates are implemented ad hoc. This means that metrics are used to calculate the model. Subsequent updates result in recalculating the whole model. Code is rendered and drawn on the screen. To get acceptable performance, changes that are streaming in are buffered. I think this makes it unusable for big code files (like the one found in the TypeScript project).

Suggested improvement.

I want to create a small signals library right into the metrics package. This would allow devs to update the status of a mutant, resulting in cascading changes in the entire model (so file metrics and parent directory metrics). Next, we could wire up these signals into the "reactive" base class we are currently using, drawing inspirations from @lit-labs/preact-signals.

The result is that changes to the model automatically cascade to the screen, no need for the custom update code we use now.

I want to implement this as a breaking change to keep things simpler.

Alternatives I've considered

• We could create a custom solution inside metrics like this, and keep the rest of the API the same:

  const report = new MutationTestingReport(reportsJson);
  // report.results etc work as they do now.
  report.updateMutant(mutant);
  // report.results etc updated  

  However, more is needed to solve the more significant issue of what to update on the screen.

• We could try to depend on @lit-labs/preact-signals directly from metrics and use that library. However, this would add a dependency to metrics, which also adds that dependency to all dependents, like the dashboard and StrykerJS. Also, if lit-labs ever decided on a different Signals API, it would mean further breaking changes. As stated in their blog, they landed on preact-signals with little consideration.

[xandervedder]

I like the idea of rolling our own, it doesn't seem like much is needed to replicate (integrate) the @lit-labs/preact-signals lib. I'd say go for it 🙂.

[hugo-vrijswijk]

If I understand correctly, the main purpose would be the expensive re-calculation of the metrics on large reports, even though not much has actually changed. I don't really see yet how just signals would help with that (or what the API would look like). Would it be an 'Observable' like object that you call an update on and it then lets Lit know to re-render? Like the Lit devs said in the article, just introducing signals doesn't make much of a difference for performance. Since rendering is already batched, the main focus point would be the metrics calculation? I must be missing something but I don't understand how signals improve that.

We could create a custom solution inside metrics like this, and keep the rest of the API the same:

Mutability?!? Are you trying to give me a heart-attack?

You could do something sort-of similar to what metrics-scala does, which is a tree structure. Each node calculates its own mutation score, and updating a mutant just means replacing the relevant nodes, reusing the non-changed ones.

Another option might be to add a parameter to calculateMetrics or oldMetrics or something similar, that does a 'best guess' of whether the report is completely new, or if it can just update a small amount of data.

Either way I think you still need some sort of change detection in metrics app component that will generate the new metrics

[nicojs]

small signals library right into the metrics package

Yeah, it turned out to be wishful thinking on my part. It cannot be 'very small', as we also need a 'signal aware' Map implementation (for holding on to covered / killed mutants inside tests). Since I've created this issue, the proposal for JS Signals was made and became stage 1 and lit.dev has decided to add this proposal signals support to their experimental @lit-labs packages.

All things considered, using the signals proposal pollyfil would be the better strategy. We can use signal-utils for a SignalMap implementation.

@hugo-vrijswijk @xandervedder thoughts?

[nicojs]

If I understand correctly, the main purpose would be the expensive recalculation of the metrics on large reports, even though not much has actually changed. I don't really see yet how signals would help with that (or what the API would look like). Would it be an 'Observable'- like object that you call an update on, and it then lets Lit know to re-render?

We would be able to get rid of the whole "reactivate" shenanigans, which will make the code easier to understand and reason about.

Like the Lit devs said in the article, just introducing signals doesn't make much of a difference for performance. Since rendering is already batched, the main focus point would be the metrics calculation? I must be missing something but I don't understand how signals improve that.

Hmm interesting. I assumed it would safe on performance as we now rerender all components on every change (including the source code parsing, instrumenting, etc.). This change would make sure only the necessary parts rerender (where the changes occurred).

Would you like me to add a 'performance test'? I was thinking to use the checker.ts file as a performance test.

[xandervedder]

All things considered, using the signals proposal pollyfil would be the better strategy.

Agreed, this simplifies things a lot.

Would you like me to add a 'performance test'? I was thinking to use the checker.ts file as a performance test.

I would like to see the performance increases compared to not using the signals 🙂.