Open trendelkampschroer opened 1 year ago
See also #48. Some comments:
I waited for compile step for about 10 minutes and then interrupted a process. Speedup (if there is any) does not worth it if my application would take more than 10 minutes to start. Tested on MacBook Pro M1.
Similar situation here. I'm running lleaves on a TPU with 334GB RAM, but it still takes forever to compile, even 1 hour is not enough (I haven't been able to successfully compile yet). For context I am using LightGBM Classifier with the following parameters:
n_estimators=600
num_leaves=15
max_depth=5
I have tried using finline
as both True and False.
I don't know what is happening on a low level, but I see that RAM usage increases at a steady rate to ~66GB before stagnating, and nothing happens.
I am copying over parts of my comments from the discussion started in #58
Consider the following example in which a random forest with a large number of trees is compiled via
lleaves.Model.compile
One can see that the vast majority of time is spend on the
llvmlite
calls, i.e. it seems that the overhead for parsing the tree and building the IM is relatively small compared to the actual costs for compiling. It would be really cool if the compile time could be further reduced - the compiled artifact is machine specific so that a safe deployment on e.g. a kubernetes cluster potentially requires a costly re-compilation whenever the model is deployed. It would be super helpful if this time could be further reduced.In addition I observe the following timings for different values of
num_trees
:Notably, the compilation step seems to not benefit from multiple cores on my machine, so if there is a way to just throw more CPU resources at it, I'd be happy to explore it.