Performing simple inference

[maxwellflitton]

Thanks for creating such a great library. I went through the math in the README and then created some simple data that would mimic the decision tree in the README and trained a basic DecisionTreeClassifier. After this, I got the state_dict of the model which in the debugger looked like the following:

Does this mean that if I have an input tensor, and I merely loop through the matrices performing matrix multiplications in order to get accurate outcomes? I'm asking this because I am looking into performing inference with as few dependencies as possible. Ideally, I'd like to build a compiled program in Rust that just loads those weights and performs the matrix multiplications to get predictions.

[interesaaat]

Hi! Yes that would work but doing inference with matrix multiplication gives ok performance only with small trees. With deep trees it is better to use the other strategies.

[maxwellflitton]

Thanks for the response, I'm getting matrix dimensions errors. I loop through the state_dict and append the weights to a list with the following structure:

I have the outcome categories under the variable name classification, however, when I perform the following code:

input_list = np.array([1.5, 2, 3.5, 5, 4])

result = np.matmul(input_list, weights[0])
result = np.matmul(result, weights[1])
result = np.matmul(result, weights[2])
result = np.matmul(result, weights[3])
result = np.matmul(result, weights[4])

I get dimension errors for the calculations. I tried the transpose and it gets me one layer in and crashes. I also tried the following configuration:

result = np.matmul(weights[0], input_list)
result = np.matmul(result, weights[1])
result = np.matmul(result, weights[2]) # crashes here
result = np.matmul(weights[3], result)
result = np.matmul(weights[4], result)

but it crashes. What approach can I take to perform the calculations? Also, thank you for the highlight for the deep trees. Do you have any links or search terms for these higher-performance strategies so I can read them? I'm also going to try and attempt random forests afterward. Can I perform calculations on the random forests in the same way I do with the decision trees? I appreciate they are a collection of decision trees. I appreciate the time spent replying to such things, my efforts will be open-sourced to enable people to serve Forrest models without needing runtimes. It's being built in rust but will have python bindings.

[interesaaat]

Here is the paper containing experiments and a description of the other approaches for trees evaluation.

Here is the execution of the tree evaluation using matrix multiplications. You can try to replicate this.