definition of grouping optimization score leads to poor results in some cases

First off, great tool to help a newcomer get started with using XCMS, thanks!

Now, I understand that IPO is not actively maintained, but I wanted to leave this here for others, and if/when someone takes up development again.

The underlying issue is that the grouping optimization seeks to optimize norm(GS)+norm(RCS). In my case, this leads the algorithm to seek out a minimum dominated by RCS, while loosing out on >95% of the "good groups" available (see below). I know that optimizing for two parameters (GS and RCS) will probably always encounter such issues in certain edge cases, which my data set appears to be. However, a re-definition of the overall score may still help to reduce these edge cases. For my scenario, the following adaptations helped:

score = GS/max(GS) + (1-ARTS/max(ARTS))

This weighs tiny improvements in ARTS less heavily than the hyperbolic RCS. A definition along those lines also avoids calculating the norm only on the current iteration, which contributes to the algorithm running itself into a corner.

Below, the edited results for a bandwidth optimization (second iteration). Note that IPO selects the first parameter set with GS 0.434635 (!). I've seen this happen with two of my datasets pretty independently of initialization parameters.


[[2]]$response
[[2]]$response[[1]]
[[2]]$response[[1]]$exp_index
[1] 1
[[2]]$response[[1]]$good_groups
[1] 16
[[2]]$response[[1]]$bad_groups
[1] 589
[[2]]$response[[1]]$GS
[1] 0.434635
[[2]]$response[[1]]$RCS
[1] 1561.193
[[2]]$response[[1]]$retcor_done
[1] 1

[[2]]$response[[2]]
[[2]]$response[[2]]$exp_index
[1] 2
[[2]]$response[[2]]$good_groups
[1] 2745
[[2]]$response[[2]]$bad_groups
[1] 2752
[[2]]$response[[2]]$GS
[1] 2738.018
[[2]]$response[[2]]$RCS
[1] 225.8368
[[2]]$response[[2]]$retcor_done
[1] 1

[[2]]$response[[3]]
[[2]]$response[[3]]$exp_index
[1] 3
[[2]]$response[[3]]$good_groups
[1] 2930
[[2]]$response[[3]]$bad_groups
[1] 2751
[[2]]$response[[3]]$GS
[1] 3120.647
[[2]]$response[[3]]$RCS
[1] 201.7932
[[2]]$response[[3]]$retcor_done
[1] 1

[[2]]$response[[4]]
[[2]]$response[[4]]$exp_index
[1] 4
[[2]]$response[[4]]$good_groups
[1] 3081
[[2]]$response[[4]]$bad_groups
[1] 2680
[[2]]$response[[4]]$GS
[1] 3542
[[2]]$response[[4]]$RCS
[1] 176.003
[[2]]$response[[4]]$retcor_done
[1] 1

[[2]]$response[[5]]
[[2]]$response[[5]]$exp_index
[1] 5
[[2]]$response[[5]]$good_groups
[1] 3149
[[2]]$response[[5]]$bad_groups
[1] 2652
[[2]]$response[[5]]$GS
[1] 3739.141
[[2]]$response[[5]]$RCS
[1] 150.6902
[[2]]$response[[5]]$retcor_done
[1] 1

[[2]]$response[[6]]
[[2]]$response[[6]]$exp_index
[1] 6
[[2]]$response[[6]]$good_groups
[1] 3103
[[2]]$response[[6]]$bad_groups
[1] 2709
[[2]]$response[[6]]$GS
[1] 3554.304
[[2]]$response[[6]]$RCS
[1] 134.9769
[[2]]$response[[6]]$retcor_done
[1] 1

[[2]]$response[[7]]
[[2]]$response[[7]]$exp_index
[1] 7
[[2]]$response[[7]]$good_groups
[1] 3029
[[2]]$response[[7]]$bad_groups
[1] 2773
[[2]]$response[[7]]$GS
[1] 3308.634
[[2]]$response[[7]]$RCS
[1] 121.0674
[[2]]$response[[7]]$retcor_done
[1] 1

[[2]]$response[[8]]
[[2]]$response[[8]]$exp_index
[1] 8
[[2]]$response[[8]]$good_groups
[1] 2978
[[2]]$response[[8]]$bad_groups
[1] 2825
[[2]]$response[[8]]$GS
[1] 3139.286
[[2]]$response[[8]]$RCS
[1] 110.479
[[2]]$response[[8]]$retcor_done
[1] 1

[[2]]$response[[9]]
[[2]]$response[[9]]$exp_index
[1] 9
[[2]]$response[[9]]$good_groups
[1] 2925
[[2]]$response[[9]]$bad_groups
[1] 2863
[[2]]$response[[9]]$GS
[1] 2988.343
[[2]]$response[[9]]$RCS
[1] 102.0279
[[2]]$response[[9]]$retcor_done
[1] 1

[[2]]$max_settings
      response x1
[1,] 0.9687696 -1
[[2]]$target_value
[[2]]$target_value$exp_index
[1] 1
[[2]]$target_value$good_groups
[1] 16
[[2]]$target_value$bad_groups
[1] 589
[[2]]$target_value$GS
[1] 0.434635
[[2]]$target_value$RCS
[1] 1561.193
[[2]]$target_value$retcor_done
[1] 1

rietho / IPO

definition of grouping optimization score leads to poor results in some cases #65