Updates 17-04-2018

[arennax]

Results with NSGA-II, Add SA results and runtimes

MRE:

SA:

Runtimes:

Generations Stats:

Remarks:

• Tuned learners perform better than the state-of-the-art (CoGEE)

  sharelatex

  ASE2018

[timm]

what is going on with the MRE, SA results? they are obviously buggy- results are not sorted by median. why are you sending my such obviously buggy results?

[timm]

no generation stats for DE10, NSGA2, COGEE

[timm]

no generation stats for DE10, NSGA2, COGEE

[timm]

i applied the following pruning rules:

• for each data set, find things that are ranked one in MRE and SA
  • if that set is empty, go to rank two (only had to do this once- for myasaki)
• for those things, cull the slowest
  • fastest = abe0, cart atlm
  • slower = de*
  • slowest = all the GAs

this results in:

• 3/9 CART
• 2/9 ATLM
• 1/9 GA100
• 1/9 NSGA-II
• 1/9 COGEE
• 1/9 ABE0

so no clear winner, with one priviso:

• in albrecht, the GA methods did best
  • otherwise:
    • not GAs
    • one slower method is useful (DE* for CHINA)
    • but for the remaining 7/9 datasets, defaults are good

that's kinda messed up. no overall best result. one negative result: GA tuning for ABE not worth it

• time to try tuning CART (Wei's IST paper has ranges of parameters).