autoplot type 'cd' error to warning?

[pfistfl]

Sometimes I want to show the CD Diagram and plot it even if there are no significant differences (to show that this is the case). Could we consider raising a warning instead of an error? Or alternatively, should we provide an option that allows to disable this check?

[RaphaelS1]

Sometimes I want to show the CD Diagram and plot it even if there are no significant differences (to show that this is the case).

What do you gain from visualising a non-significant plot? CD plot is already too complicated for many people, I worry this will lead to easy misuse.

Or alternatively, should we provide an option that allows to disable this check?

This is actually already possible but it's a hidden feature. Just set p.value = 1

[berndbischl]

i dunno errors are always not so great here. just doc the hidden feature?

[RaphaelS1]

https://github.com/mlr-org/mlr3benchmark/blob/74a4cfeb69bf4a208e2f00a72dbeb890d7f7a440/R/autoplot.BenchmarkAggr.R#L27-L28

[pfistfl]

It’s a different thing to do p value = 1 and getting non-significant CDs

[RaphaelS1]

Ah true because the critical differences change. Do you want to do PR a fix? Just needs an arg to switch this off https://github.com/mlr-org/mlr3benchmark/blob/main/R/BenchmarkAggr.R#L326

[RaphaelS1]

Closed in #22