elephaint
commented
3 years ago Hi,
Thanks for the kind words, much appreciated!
- That's a good remark, and something I was planning on making available too. It's currently not possible to easily extract those, but it should be an easy fix. I'll have a look at it in the coming days and I expect to make it available either through a separate function or by adding a parameter to the predict / predict_dist functions.
- I'm not familiar with the get_dump() from xgboost, but the 'model.save' method saves all the required arrays for the tree. In particular, there are six arrays of importance for the tree structure:
- 'nodes_idx': [n_estimators x max_nodes] int64 array containing indices of nodes. In PGBM, I use a fixed indexing system for node locations in a binary tree. The first (top) node has index one, the first left child node has index two, the first right child node has index three; the left child node of the first left node is four, and so forth. I hope this makes the schema clear. Hence, given any index you can relatively easily understand the position of the node in the binary tree.
- 'nodes_split_feature': [n_estimators x max_nodes] int64 array. Which feature to split on (i.e. '0' represents the first feature, and so forth)
- 'nodes_split_bin': [n_estimators x max_nodes] int64 array. Which bin out of the possible max_bins to split on.
- 'leaves_idx': [n_estimators x max_leaves] int64 array. The index of a leaf. Follows the same indexing procedure as desribed above. Index positions are unique, i.e. indices are either nodes or leaves (or non-existent if the tree has stopped learning for some reason before arriving at that index, for example due to hitting the maximum number of leaves)
- 'leaves_mu': [n_estimators x max_leaves] float32 array (float64 in Numba version). The empirical mean ('the leaf weight')
- 'leaves_var': [n_estimators x max_leaves] float32 array (float64 in Numba version). The empirical variance ('the leaf variance').
Note 1) max_nodes = max_leaves - 1 Note 2) For prediction, also the arrays 'bins' ([n_features x n_bins] float32) and the initial prediction (yhat_0, float32 scalar) are required. These are saved too when calling 'model.save'.
Hope this helps! I'll have a look at implementing 1) in the coming days.
Kind regards,
Olivier
flippercy
Hi:
Thank you for the awesome library! I did some tests with it and have a few questions:
How to pull the parameters, such as mean and standard deviation, of the final fitted distribution for each leaf? Such information is extremely helpful when the result is presented and explained to stakeholders. Currently the model just returns some numbers sampled from the distribution but business users are likely to focus on the distribution itself.
Is there anyway to spit out the model's tree structure to a data frame like what get_dump() does for xgboost?
Thank you!