A minimal benchmark for scalability, speed and accuracy of commonly used open source implementations (R packages, Python scikit-learn, H2O, xgboost, Spark MLlib etc.) of the top machine learning algorithms for binary classification (random forests, gradient boosted trees, deep neural networks etc.).
I finished what I wanted to focus on for Linear and Random Forests. There are lots of things one might want to do, I listed a few here: https://github.com/szilard/benchm-ml/blob/master/TODO.md
If you are interested in tackling any of those, let me know.