nickgillian
commented
8 years ago There are a few reasons that you might observe this.
One relates to how you might be generating the training/test data for each run. The second relates to the random nature of how the Random Forest algorithm works (i.e., the random bit in the Random Forest).
For the first case (training/test dataset setup), if you want to see more consistent results across multiple runs, make sure you are using the SAME training/test data each time. For example, if you have a single dataset that you are randomly splitting into a separate training/test dataset each time you run the program then there is a good chance you will see variation in the results (regardless of whether you are using Random Forests or other GRT classification algorithms). The reason for this is that there might be some 'noisy' or 'harder' samples in your dataset than others, so the results will be lower for cases where these samples get partitioned into the test dataset (as the probability that the algorithm will get these wrong will be higher, so the accuracy will be lower). You can avoid this problem by segmenting your dataset into a training dataset and test dataset once, then save these new datasets to files and always load the same training/test datasets each time you run the algorithm.
For the second case (randomness of RF algorithm), the variation in the accuracy of the system can change based on many runs on the SAME training/test dataset if you don't tune the RF learning parameters. The reason for this is that the RF algorithm randomly selects a subset of features each time it builds a node in each tree in the forest, and then optimizes the parameters for each node based on the random subset. It also selects a random subset of the training dataset to use for training each tree in the forest. This might not sound like the best way to optimize the decision nodes in each tree, however, if you average these results across multiple trees in a forest (which is what the RF prediction algorithm does for you automatically), you end up with much smoother decision boundaries and much better results than using a single decision tree optimized on all the training data and features.
However, if you only use a very small subset of features at each node (say 5 random features from 1000 possible features) or only use a very small number of trees (say 2), you might see a large variation in the accuracy of the model each time your train a model. The way to make these results more stable is to (1) increase the number of random splits used at each node, and (2) increase the number of trees in the forest. You should be careful that you don't increase these values too high (i.e., use all features or 100000 trees), as the accuracy of the model might not increase (in fact, it could decrease), but the training time could increase significantly.
I hope this helps.
JairoJs
Hello again, sorry to throw questions here, I dont know where else to make them since the forum is down.
Im training a prediction model using random forest, and I just notice that with the same dataset, and the same parameters its giving me diferent accuracy values in a range that I cannot ignore (1-7%) is this a normal behavior? for example I ran the program with an especific configuration and it gave me 91.4896% then I ran it again with out changing any lines and showed 95.8347% then I did the same and have 93.7367%