Week 10: Finish the Implementation of Regularized Radom Forest Framework, Tune Parameters and if possible implement Pruning of the Trees

azmfaridee / mothur

This is GSoC2012 fork of 'Mothur'. We are trying to implement a number of 'Feature Selection' algorithms for microbial ecology data and incorporate them into mother's main codebase.

https://github.com/mothur/mothur

GNU General Public License v3.0

3 stars 1 forks source link

Week 10: Finish the Implementation of Regularized Radom Forest Framework, Tune Parameters and if possible implement Pruning of the Trees #20

Open azmfaridee opened 12 years ago

azmfaridee commented 12 years ago

Related Issues: #3, #14, #15, #16, #17, #19

As per issue #19 I have been experimenting with outcomes after we have been running the algorithm on some real life data. Some of the new concerns are tree pruning for better performance, dealing with over-fitting data as well as the scheduled implementation of Regularized Random Forest Algorithm.

In this week I'd try to follow up with all these ideas that have been swirling around

End of Week Deliverable

Complete the implementation of Regularized Random Forest and compare performance/accuracy with normal Random Forest
Possible improved Performance with pruned trees
Investigate other ways of performance turing

azmfaridee commented 12 years ago

End of Week Update:

The implementation of Regularized Radom Forest is note 100% complete, there are a lot of classes that needs to be recycled, so I need to be extra careful to not to break existing code.
As for performance tuning, I've been reading the literature again and discovered that we need to use Information Gain Ratio than using basic information gain, that would boost accuracy a bit more.
Optimum number of selected feature in each split can be fine tuned between Log2 and SquareRoot based measure.
As I did not get sufficient learning materials (the books that I mentioned), I could not implement pruning.

azmfaridee commented 12 years ago

@kdiverson I've uploaded a pdf titled Pruning Decision Trees and Lists.pdf in Dropbox in the Pruning Folder. It seems to be a PhD thesis paper, and has some good examples of popular pruning methods. Take a look.