search

szilard / GBM-perf

Performance of various open source GBM implementations
MIT License
215 stars 28 forks source link

Spark MLlib GBT 4d (for demos) #19

Closed szilard closed 5 years ago

szilard commented 5 years ago

100 trees, depth 10, 100K data, 32 cores (1 node) runs for 17 mins. Ugh!

How can we make Spark MLlib GBT work fast enough for a demo?

Smaller data? Less trees? Less depth? More cores? More nodes? Let's help the Spark fans...

szilard commented 5 years ago

We actually want it fast even on a laptop. So we'll work with m5.xlarge (4 cores, 16GB RAM)

Less data (10K records):

runs 200 sec, AUC=0.670, peak RAM usage 7.8GB - too slow!

Less trees (10 trees)

runs 65 sec, AUC=0.711, RAM 6.5GB - nice, we are getting faster!

Less depth (depth 4)

runs 360 sec, AUC=0.725, RAM 6.4GB - oh no, way too slow!

Even less depth? depth 1 maybe?

runs 290 sec, AUC=0.704, RAM 6.2GB - argh!

We need a combination! Less trees (10) and less depth (4)

runs 40sec, AUC=0.708 - maybe OK for a demo! but can we get even faster?

Maybe 5 trees and depth 3?

runs 20sec, AUC=0.694 - dunno... I guess training sophisticated machine learning algorithms (sorry, I mean AI) just takes time

szilard commented 5 years ago

Oh, yeah, 1 tree and depth 1 is the ultimate solution! Thanks @daroczig for the idea!

runs 6 sec, AUC=0.634

szilard commented 5 years ago

btw lightgbm 10 trees depth 4 take 0.2 sec (vs Spark 40 sec)