Visual indicators in PR comment

[assignUser]

This is not a high priority but more a fit and finish kind of issue.

As described in the inference vignette changes between branches are statistically significant if the confidence interval does not overlap 0. You can of course check that for each benchmark manually but I think it would be nice to add a visual indicator if there are changes that warrant closer inspection. Something like this:

• define_data: 0.04s -> 0.04s [-3.73%, +4.31%] ✔️
• gen_all_dists: 0.18s -> 0.78s [+332.51%, +337.94%] ❌

or in the front

• ✔️ define_data: 0.04s -> 0.04s [-3.73%, +4.31%]
• ❌ gen_all_dists: 0.18s -> 0.78s [+332.51%, +337.94%]

I think this should also be easy to implement in benchmark_verbalize.

[assignUser]

Seeing it actually rendered as a comment I prefer the second version, as it has all the indicators aligned.

[lorenzwalthert]

Sure. I prefer in the front. Also, it should he explained somewhere (probably in tue linked vignette) what these symbols mean. Maybe we even need three for:

1. significantly better
2. Not significantly different.
3. Significantly worse

maybe You can make a proposal?

[assignUser]

• 🚀
• ☑️ or ➖
• ❗🐌

[lorenzwalthert]

• 🚀
• ☑️
• ❗🐌

I like this :D

[assignUser]

I think it looks pretty good :D, should I open a pr? https://github.com/assignUser/simstudy/pull/2#issuecomment-910391800

[lorenzwalthert]

Yes please.