How good extracted features are:

[amritbhanu]

Experiment:

• Now model is trained based on 7 features (because of regx problems, lack of certain info)
  • e.g. cant access some web traffic info or SSL final state.
• All the phishtank data is defective so to make a data set with some yes, some now, we added in some "no" from the UCI data. so now we have a twoThirds:oneThirds phished:not for testing.
• Our testing examples now contain phishing websites in abundance as dataset 2 source only has phishing sites.
• From dataset 1, 5x5 stratified cross validation to get training of 80% and testing 20%.
• This testing is now combined with examples from dataset 2 which is about 1200 examples.
• Extracted data can be found here

Results previously (on UCI data) when trained on Dataset1 and tested on Dataset1 with 9 features:

Results now (on PhishTank+some UCI) when trained on Dataset1 and tested on Dataset1 combined with Dataset2 with 7 features:

Conclusion

• Good features are extracted.
• Only NB results show different trend. Decrement in precision, and increment in false alarm and recall.

[timm]

Conclusions:

• Not accessing web traffic info or SSL final state costs us about 5 points. No biggie
  • Lesson learned: data extraction can be characterized as a cost-benefit exercise.
    • No need to chase absolutely everything.
• Dont use NB
• As to the others,
  • similar runtimes (so real winner there)
  • J48 might be better to "explainability" (but that is a subjective measure)