vivekaxl / MOLearner

Multi-objective learning for configurations
1 stars 0 forks source link

For Objectives > 2 (POM, XOMO, MONRP) + Flash3 #22

Open vivekaxl opened 7 years ago

vivekaxl commented 7 years ago

Flash = AL2

Flash3:

flash3(data, # some examples
      f,   # function to evaluate; e.g. compile+run makefile
      n0=20,
      n1=256 # say
      ):
  shuffle(data)
  training_independent =  data[:n0]
  iteration_count = 0
  while True:
    # Select data in the window of length n1
    n1_select = data[n0 + iteration_count * n1: n0 + (iteration_count + 1) * n1]

    # Build Tree for each objective
    for k in #objs:
      tree[k] = CART(map(eval(f), some), goal=actual[i]

    # Predicted values of n1_select
    predicted_n1 = predict(n1_select, tree)

    # Perform non dominated sort on predicted_n1 -- Filter 1
    nds_predicted_n1 = NDS(predicted_n1)

    # Only choose points from nds_predicted_n1, 
    # which dominated the poitns in training data -- filter 2
    selected_nds_n1 = select(nds_predicted_n1, training_independent, bdom)

    if len(selected_nds_n1) == 0:
        lives -= 1
    else:
        training_indep += selected_nds_n1

    if lives == 0: break
    iteration_count += 1

return training_indep
vivekaxl commented 7 years ago

POMX (Objectives = 3)

screenshot 2017-04-30 12 25 25

XOMOX (Objectives = 4)

screenshot 2017-04-30 12 26 38

MONRP (Objectives = 3)

screenshot 2017-04-30 12 22 55

vivekaxl commented 7 years ago

Observations:

  1. The effectiveness of FLASH deteriorates as the number of objectives increases (Unconstrained). --- (opportunity) This calls for tuning of the stopping criteria #23
  2. (Expected results): Techniques like FLASHX and SWAY, which start with a oversampled valid population (10000) is not effective in terms of constrained model.
  3. Flash3 is definitely not ideal solution. However the batch based NDS is a good idea to pursue. I am current running Flash4, which is FLASH3 + random sample from selected samples ( from n1)
timm commented 7 years ago

Getting lost on what are current results

Please update paper with "the" results using only one version of flash (I think ur best is flash20). Include runtimes. Express igd and gd normalized 0 to 100 . Zulugua models. Etc