search

soda-inria / sklearn-numba-dpex

Experimental plugin for scikit-learn to be able to run (some estimators) on Intel GPUs via numba-dpex.
BSD 3-Clause "New" or "Revised" License
15 stars 4 forks source link

Fix docker build pipeline #109

Closed fcharras closed 1 year ago

fcharras commented 1 year ago

Numba 0.57 compatibility issues are already reported upstream see https://github.com/IntelPython/numba-dpex/issues/1000

fcharras commented 1 year ago

I'm still not building it locally

There's something wrong with dpnp build that requires further debug

fcharras commented 1 year ago

Latest commits both address latest issues and bump to latest releases.

I'd like to bump aggressively to latest commits of master branch for numba_dpex so it includes cache performance fixes, will revert to a fixed tag once it's officially released.

fcharras commented 1 year ago

Fixes https://github.com/soda-inria/sklearn-numba-dpex/issues/94

fcharras commented 1 year ago

Not yet functional, see https://github.com/IntelPython/dpctl/issues/1190

However the conda install works.

fcharras commented 1 year ago

Last push finally contains a Dockerfile for an image that builds and works. But it's not very useable, since it contains a broken dependency tree, pip resolver will try to fix it whevener it can (e.g when building scikit-learn...), and the automated resolution fix just breaks the runtime.

IMO the docker environment can't be updated until https://github.com/IntelPython/dpctl/issues/1190 is fixed.

The solution of pinning everything to 2023.0.0 instead and using older 2023.0.0 oneapi basekit for building is also cumbersome, because some dependencies needs to be force-pinned to 2023.0.0 for this to work (if not the bug currently on main that blocks our weekly image publishing occurs), I'm not interested in doing extra maintaining for a version that is not the latest.

I've checked that the performance for KMeans remains good both locally and on edge cloud with the version combinations 2023.1.0 and that the unit tests pass, so aside from those build problems, the bumps work well.

I need a up-to-date environment without our patches on loading numba_dpex that got in the way of some debug I was doing for the KNN so I'll rebase other branches on this one waiting for things to be fixes upstream.

fcharras commented 1 year ago

Or if the the intel-opencl-rt issue is not fixed upstream it might be the opportunity to try the official numba_dpex docker image but I'm afraid we would loose some flexibility.

fcharras commented 1 year ago

Installers work again with 2023.2.0 release, but there's a regression in latest numba_dpex release that prevent some kernels to run on CPU (e.g. our lloyd fused kernel), see https://github.com/IntelPython/numba-dpex/issues/1068

fcharras commented 1 year ago

Things work on my local environment, I'll merge so that the automated build pipeline can resume, and the conda install instruction are fixed.