search

vivekaxl / MOLearner

Multi-objective learning for configurations
1 stars 0 forks source link

What is AL3? #12

Open vivekaxl opened 7 years ago

vivekaxl commented 7 years ago

The following is the step by step description of AL3:

  1. Select 20 points at random and evaluate them (objs_real)
  2. objs_real is then transformed into a single objective cdom_obj using cdom as the aggregation function
  3. Decision Tree (Regression) is created using the points from step 2.
  4. Leaf with highest median score is selected and the decision rules leading up to the leaf is calculated
  5. These rules (from step 4) is used to filter data
  6. Another constraint is added to rules from Step 5. Only points which are within the one std in each independent variable is preserved. Catch: if none of the data points survive a particular rule, that rule is discarded.
  7. Filtered data from Step 6 are predicted using CART tree (one for each objective)
  8. A non-dominated sort is performed on the filtered results and only the non-dominated points are evaluated.
  9. Repeat step 2 to 8 till no other point is added to the dominated set (lives = 3)

Notes: The non-dominated portion of this method is really slow. Similar issue with AL1 and AL2. This issue came into light while dealing with large datasets.

timm commented 7 years ago

i would have used cdom. so not N trees.

why rule 6?

i would have replaced 5 with a data generator. no need to filter remaining data. just generate one instance at random within the constraints found by 4

then i would have dumped 8. just evaluated the instance found by the last para to the set of (initially 20) instances. then carted that set.

as to stopping rule... er... not improvement on at least one objective in last 4 rounds?

timm commented 7 years ago

in any case, AL3 is just an optimization (run faster). it should do not better than AL1 or AL2. we can ignore AL3 for now, if u like, especially if the real game is that epal is insanely slow

is epal insanely slow for all models? or just for some? get prelim results on the faster models?

vivekaxl commented 7 years ago

is epal insanely slow for all models?

It is particularly slow for higher dimensional data. For smaller dimensional data, it was not obvious. I need to calculate time.

get prelim results on the faster models?

I have the results for faster models. Please look at #14 and #15