Minor issues with tests functions: (1) need "*" significance designation when performing single comparison and (2) y.position doesn't seem to play well when y axis has log scale

[barrel0luck]

Hi! I absolutely love your package and use it regularly in my work. I've bothered you before with feedback and the great thing is you've responded and made improvements in your package, so thanks a lot for that! Here are some more things I noticed: (1) When performing a test with a single comparison, having a p.singif column with representation of significance (that matches the p.adj.signif representation) would be greatly appreciated. Especially since the rules for what should be , **, *** etc is difficult to script quickly such that it matches the rules you use when multiple comparisons are present. So for consistency, it'd be great if it's available in the single comparison as well. (2) y.position doesn't seem to be calculated correctly when the plot is made on the log10 scale. I'm not sure exactly exactly what the problem is, as without the log scale the y.position seems to be correctly calculated. It's really strange. It's probably something really simple that I don't understand.

Here's the code for reproducing the problems:

library(tidyverse)
library(rstatix)

# Generate dataset --------------------------------------------------------

x <- c(1:5)

y1 <- c(1e6, 1e7, 1e8, 1e9, 1e10)
y2 <- c(5e6, 5e7, 5e8, 5e9, 5e10) 

data_to_plot <- data.frame(xlabel = NULL, A = NULL, y = NULL)
for (i in 1:length(x)) {
  z1 <- rnorm(mean = y1[i], sd = 100000, n = 6)
  z2 <- rnorm(mean = y2[i], sd = 100000, n = 6)
  z3 <- data.frame(x_label = x[i],
                   A = z1,
                   B = z2)
  data_to_plot <- rbind(data_to_plot, z3)
}

data_to_plot <- pivot_longer(data_to_plot, cols = A:B, names_to = "treatment", values_to = "response")

# Perform tests -----------------------------------------------------------

t_test_results <- data_to_plot %>% 
  group_by(x_label) %>% 
  t_test(response~treatment) %>% 
  add_y_position(fun = "max", step.increase = 0.02)

# A column containing "*" representation of signficance is missing when performing single comparison
# If I've multiple comparisons, this representation is automatically generated in column p.adj.signif

t_test_results$p.signif <- ifelse(t_test_results$p < 0.05, "*", "")

# And when the plot has a log10 y-axis, the y.position calculation is not as nice somehow
# You can see how for the the x_label = 1 & 2 the position of the stars is further away than it should be...

ggplot(data_to_plot, aes(x = x_label, y = response)) +
  stat_summary(aes(color = treatment, linetype = treatment), fun = "mean", geom = "line") +
  stat_summary(aes(color = treatment), fun.data = "mean_sd", geom = "errorbar", width = 0.2) +
  geom_text(data = t_test_results, 
            mapping = aes(x = x_label, y = y.position, label = p.signif), 
            size = 5, show.legend = F) +
  scale_y_log10()

# On non log scale y.position appears to be correct. So what's going on?
ggplot(data_to_plot, aes(x = x_label, y = response)) +
  stat_summary(aes(color = treatment, linetype = treatment), fun = "mean", geom = "line") +
  stat_summary(aes(color = treatment), fun.data = "mean_sd", geom = "errorbar", width = 0.2) +
  geom_text(data = t_test_results, 
            mapping = aes(x = x_label, y = y.position, label = p.signif), 
            size = 5, show.legend = F)

Here's the plot with log10 y scale:

Here's the plot without log10 y scale:

[kassambara]

Hi,

Thank you for the positive feedback, appreciated!

1/ Adding significance symbols: the function rstatix::add_significance() might help, like this:

t_test_results <- data_to_plot %>% 
  group_by(x_label) %>% 
  t_test(response~treatment) %>% 
  add_significance() %>%
  add_y_position(fun = "mean", step.increase = 0.02)

(Feature request #70 opened for automatic significance symbols)

(Note also that, you're plotting group mean in the ggplot, so consider using fun="mean" instead of fun = "max" in the add_y_position())

2/ For the log scale:

The plot you obtained is what is expected in log scale for this data. The transformation log10(t_test_results$y.position) returns 9.002153, 9.021177, 9.176075, 9.778149, 10.707570 in log scale. The first 3 points are flat (around y = 9).

One possibility is to manually adapt the y.position to create the desired output in log scale; for example:

t_test_results$y.position[1:3] <- c(10^7, 10^8, 10^9)
ggplot(data_to_plot, aes(x = x_label, y = response)) +
  stat_summary(aes(color = treatment, linetype = treatment), fun = "mean", geom = "line") +
  stat_summary(aes(color = treatment), fun.data = "mean_sd", geom = "errorbar", width = 0.2) +
  geom_text(data = t_test_results, 
            mapping = aes(x = x_label, y = y.position, label = p.signif), 
            size = 5, show.legend = F) +
  scale_y_log10()

[barrel0luck]

rstatix::add_significance()

Thanks for this and also for opening #7

Note also that, you're plotting group mean in the ggplot, so consider using fun="mean" instead of fun = "max" in the add_y_position())

Ah, thanks for pointing this out. I think I just copy that everywhere without changing it and that's where it came from.

One possibility is to manually adapt the y.position to create the desired output in log scale; for example:

Yes, I'm doing that currently. I've only given a simple example here where this problem is very easy to fix manually. But with a larger dataset and with multiple comparisons, adjusting the position of stars on the log scale becomes really complex and so I was wondering if there was a more automated fix for this...

[kassambara]

OK, let's this issue open

[barrel0luck]

Thanks and sorry for increasing your work! You must already be really busy with maintaining this package and with your own science.