Should we represent the results from the tree validation datasets (Hoek, Racle, Schelker) independently or combine them?

[grst]

Should we represent the results from the tree validation datasets (Hoek, Racle, Schelker) independently or combine them?

[grst]

@FFinotello suggested to show the performance for each validation dataset independently, as they might have different characteristics. So far, I agree.

I am just not sure what is a sensible way to do so.

In principle, there are three ways how to look at the datasets:

• between and within sample comparisons (only fair for epic, quantiseq, cibersort_abs), e.g. the following image shows quantiseq + hoek dataset
• within sample comparisons (only fair for quantiseq, cibersort, cibersort_abs, epic)
• Between sample comparisons (fair for all except cibersort)

In the supplementary information, all three types of comparisons are shown: https://grst.github.io/immune_deconvolution_benchmark/validation-with-real-data.html

For the figure in the paper, I am currently using the between-sample comparison of all three datasets merged (different symbols represent different datasets):

By combining the datasets I hoped to

• increase the variance of the infiltration
• increase the number of samples used to calculate the correlations

and therefore building more meaningful correlation values.

When calculating the correlations on each dataset/cell type invdividually, the sample size is as small as 3. IMO, it does not make sense to calculate correlations for 3 values.

@FFinotello , what do you suggest?

[grst]

Do both, where enough values are available:

[grst]

Latest version:

(I agree the final layout can be improved!)