search

szilard / GBM-perf

Performance of various open source GBM implementations
MIT License
213 stars 28 forks source link

XGBoost CPU speed by version #47

Closed szilard closed 3 years ago

szilard commented 3 years ago

m5.4xlarge (16 cores=8+8HT)

1M rows dataset

sudo docker run --rm -ti gbmperf_cpu /bin/bash

ln -s train-1m.csv train.csv
wget https://cran.r-project.org/src/contrib/Archive/xgboost/xgboost_0.71.1.tar.gz
## wget the other versions you want to run 
for i in `ls -1 xgboost_*.tar.gz`; do 
  R CMD INSTALL $i >/dev/null 2>/dev/null 
  egrep "Version:|Date:" /usr/local/lib/R/site-library/xgboost/DESCRIPTION
  R --slave < GBM-perf/cpu/run/2-xgboost.R
done
Version Date time [s] AUC
0.71.1 2018-05-11 13.5 0.7494959
0.81.0.1 2019-01-30 54.8 0.7494959
0.90.0.1 2019-07-25 26.4 0.7494959
1.0.0.1 2020-03-23 6.6 0.7494531
1.1.1.1 2020-06-12 4.9 0.7478858
1.2.0.1 2020-08-28 5.9 0.7478858
1.3.1.1 2020-12-22 6.1 0.7478858

(Versions/Dates from R/CRAN; 0.71 is the first one that has hist)

With github version as of today: 3.9 0.7478858

szilard commented 3 years ago

8 cores (no HT):

for i in `ls -1 xgboost_*.tar.gz`; do 
  R CMD INSTALL $i >/dev/null 2>/dev/null 
  egrep "Version:|Date:" /usr/local/lib/R/site-library/xgboost/DESCRIPTION
  taskset -c 0-7 R --slave < GBM-perf/cpu/run/2-xgboost.R
done

8 cores/1 core:

Version Date time 8c time 1c AUC
0.71.1 2018-05-11 14.8 34.9 0.7494959
0.81.0.1 2019-01-30 46.8 29.6 0.7494959
0.90.0.1 2019-07-25 28.5 26.8 0.7494959
1.0.0.1 2020-03-23 7.9 25.0 0.7494531
1.1.1.1 2020-06-12 5.5 29.6 0.7478858
1.2.0.1 2020-08-28 6.3 31.2 0.7478858
1.3.1.1 2020-12-22 6.4 31.9 0.7478858
github today 4.6 24.5 0.7478858