Predict in C/C++ with the compiled model

Waerden001 commented 1 year ago

Thanks for creating lleaves! I'm wondering if it's possible to load the complied lleaves model into C/C++ code directly for prediction? After reading the docs, it seems that we can

compile the model in python with cache=<filepath> specified.
link to the ELF file generated in step 1. in C/C++.

If this is right, what's the API to call for prediction? I find in the c_bench folder a function called forest_root which looks like what we need, but can't find the implementation

If this is not right, is there some other way to use the compiled model in C/C++?

siboehm commented 1 year ago

You're on the right track! The c_bench.cpp & c_bench.h is actually a fully self-contained example, which does exactly what you're looking for. forest_root is the default name for the entry function of the compiled tree, so its implementation is in the ELF file :)

In your Cpp code just call forest_root and pass the pointers, then make sure to tell the linker about the ELF file at cache=<filepath>.

Waerden001 commented 1 year ago

That's great, Thanks!

SunHaoOne commented 1 year ago

Hello, could you please provide me with information on how to interface with C++? Specifically, I would like to interface with a single row of data online. Here are the steps I have taken so far:

1. Generate the cache file by python

import lightgbm
from lleaves import lleaves
import pandas as pd
alldata = pd.read_csv('all_features_data.csv')
X = Xs.iloc[[0]]
llvm_model = lleaves.Model(model_file="data_model_30.txt")
llvm_model.compile(cache='./lleaves.so')
print(llvm_model.predict(X))

Then we will get a lleaves.so file here.

2. Interface with cpp

There are the files in the directory:

xxx@xxx:~/Desktop/LightGBM/src/lleaves$ ls
c_bench.cpp  c_bench.h  interface.cpp  lleaves.so

In the interface.cpp

#include "c_bench.h"
#include <vector>
#include <iostream>

int main()
{
    std::vector<double> data = {8.81351540e+00, -2.74901880e-01, -4.78453119e-02, 2.25956985e+01,
                                -2.75495538e-01, -9.12007856e-02, -4.78453119e-02, 1.88485949e+00,
                                1.88485949e+00, 1.64226175e-03, 1.64226175e-03};

    double result;
    forest_root(data.data(), &result, 0, 1);
    std::cout << "Result: " << result << std::endl;
    return 0;
}

I am modifying the c_bench.c file to eliminate the need for benchmark.h and cnpy, as these dependencies are unnecessary for my purposes.

#include "c_bench.h"
#include <algorithm>
#include <cstdlib>

static void bm_lleaves()
{

  // predict over the whole input array
  forest_root(loaded_data, out, (int)0, (int)n_preds);
}

3. Compile with gcc

xxx@xxx:~/Desktop/LightGBM/src/lleaves$ g++ -o interface interface.cpp c_bench.h -L lleaves.so
/usr/bin/ld: /tmp/ccQnP3nl.o: in function `main':
interface.cpp:(.text+0x8b): undefined reference to `forest_root'
collect2: error: ld returned 1 exit status

It seems I cannot find the forest_root function. Do you have any ideas, thanks!

SunHaoOne commented 1 year ago

You're on the right track! The c_bench.cpp & c_bench.h is actually a fully self-contained example, which does exactly what you're looking for. forest_root is the default name for the entry function of the compiled tree, so its implementation is in the ELF file :)

In your Cpp code just call forest_root and pass the pointers, then make sure to tell the linker about the ELF file at cache=<filepath>.

Thank you for providing the code, it worked great after running it in Python. However, I encountered some difficulties during inference. Below are the steps I took to reproduce the issue. If you have some time, could you please guide me through it? I would be very grateful for your help!