Code the fitness function

[fergunet]

In class GPRuleRefinementFitnessCalculator, complete next method

public Fitness calculateFitness(Individual ind) {
        TreeGenome tg = (TreeGenome) ind.getGenome();
        System.out.println("INDIVIDUAL IS: "+writeTree(tg));
        double theFitness;
        boolean toMaximize = true; //If we are maximizing, false otherwise
        //Write magic here to calculate theFitness value

        return new DoubleFitness(new Double(theFitness), toMaximize);
    }

[unintendedbear]

Closed by https://github.com/fergunet/GPRuleRefinement/commit/e4a998cc627ef530242551df1d4a13ede3ae7b87

[unintendedbear]

Ok @zeinebchelly @JJ and @fergunet , let's discuss the fitness.

What @JJ proposes is to divide the dataset in two parts: A and B.

• A would be devoted to generate, evaluate, and validate the classifier.
• B would be devoted to re-validate the best model obtained from A.

So, for the sake of accuracy, we should run an experiment following a 10-fold cross-validation process in which:

• A contains 90% of the dataset, B 10%
• A contains 80% of the dataset, B 20%
• and so on.

Then, as regard of A, we use another 10-fold cross-validation in which:

• A1 contains 90% of A, A2 10%
• A1 contains 80% of A, A2 20%
• and so on.

Where A1 would be used to calculate the fitness on each generation, and A2 would be devoted to validate the individual with best fitness at the end of each run. This means that at the end of this process we would have 10 individuals, each one with 2 attributes: a fitness and an accuracy value. We will take the one with higher fitness, of course, and in case of a tie, the one with higher accuracy.

After that, the process is repeated for every separation of A and B. This would yield again 10 models, with their re-validation values, and we would finally take the best.

What do you think?

PS: Anyways, I'm also reading this http://link.springer.com/chapter/10.1007/978-3-642-17432-2_25 so in case I find a more suitable fitness, I will suggest it here.

[JJ]

I'm obviously OK with that.

[zeinebchelly]

I agree. On Tue, 19 Apr 2016 at 17:39 Juan Julián Merelo Guervós < notifications@github.com> wrote:

I'm obviously OK with that.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/geneura-papers/2016-PPSN/issues/5#issuecomment-212009635

[fergunet]

So, summarizing. We will have to run 9*9 times the GP?

for i=1 to 9:
  A = i*10 percent of Dataset
  B = 100-A percent of Dataset
  for j=1 to 9:
      A1 = j*10 percent of A
      A2 = 100-A percent of A
      ind = executeGA(A1)
      individuals.put(ind)
      best = individuals.getBestWithValidation(A2)
      besties.add(best)
theCompleteBest = besties.getBestWithValidation(B)

(I've checked it with @unintendedbear and it seems correct) Please, confirm in case we are missing something

[unintendedbear]

Please, confirm in case we are missing something

Yes. Time.

[zeinebchelly]

I personally think that it is correct.

2016-04-19 18:11 GMT+01:00 Paloma de las Cuevas Delgado < notifications@github.com>:

Please, confirm in case we are missing something

Yes. Time.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/geneura-papers/2016-PPSN/issues/5#issuecomment-212022228