tailored daal4py build - Githubissues

intel / scikit-learn-intelex

Intel(R) Extension for Scikit-learn is a seamless way to speed up your Scikit-learn application

https://intel.github.io/scikit-learn-intelex/

Apache License 2.0

1.23k stars 175 forks source link

tailored daal4py build #28

Closed dr-pain closed 5 years ago

dr-pain commented 5 years ago

Starting from 2019u1.1 DAAL has feature to be built with specified set of algorithms. It is impossible to build daal4py out_of_the_box with such DAAL - will see undefined references. Add the tailored build feature to daal4py please.

fschlimb commented 5 years ago

Most algorithms and code simply adjust to whatever DAAL provides. Exceptions are

getTreeState hard-codes known trees/forests
custom distribution classes are #included in a hard-coded way

The code generator needs to be adjusted to

include only the distributed headers for algorithms that are found in the DAAL headers
generate the getTreeState dispatcher dependent on the available trees/forests/algorithms. Needs a tiny bit of code re-structuring, because the dispatcher is currently a manually written piece (src/gettree.pyx)

fschlimb commented 5 years ago

Both adjustments should be handled in wrapper_gen.py:gen_footers

add an argument to list the filtered algos and pass filtered list when calling gen_footers
add only #include statements for distributed algos that are both a) in the given list b) listed as 'dist_custom' in wrappers.py:has_dist
return the properly formatted cython code in return[1]
- make 'src/getttree.pyx' a jinja2 template
- assuming a list of supported models in the env (like as {{model_list}})
- iterate over list to do the dispatching
- read the template from file or add the entire file as string ti wrapper_gen.py
- setup jinja env with list of also and render the jinja2 template into return[1]

fschlimb commented 5 years ago

done with #43

The logic has been reversed: DAAL algorithms are now excluded in generator/wrappers.py:wrap_algo - they are no longer explicitly listed in a list in generator/gen_daal4py.py.

dr-pain commented 5 years ago

Could you provide details/instruction on daal4py tailored build?

fschlimb commented 5 years ago

Just let wrap_algo(algo_ver) return True for algorithms you want to have in your build, False otherwise. Parameter 'algo' is the DAAL C++ namespace and ver the version of DAAL.

yinghu5 commented 5 years ago

@fshlimb, could you please provide one step by step instructions?

we tried edit daal4py-2019.3/generator/wrappers.py def wrap_algo(algo, ver): … if algo.startswith('algorithms::decision_forest'): return True return False

fschlimb commented 5 years ago

Sounds right. Any issue with this?