search

lesteve / scikit-learn-tests-pyodide

Run scikit-learn test suite inside Pyodide
3 stars 1 forks source link

Use scikit-learn dev wheel from artifact #9

Closed lesteve closed 1 year ago

lesteve commented 1 year ago

I was not able to make the pyodide build with Pyodide dev debug working see this build log

Latest commit with my attempt.

For some reason it seems that Cython code in the neighbors returns 64 bit indices rather than 32bit ... which causes issues like RuntimeError: memory access out of bounds

import numpy as np
from sklearn.metrics._pairwise_distances_reduction import (
    ArgKmin,
    RadiusNeighbors,
)

rng = np.random.RandomState(0)
X = rng.randn(50).reshape(10, 5)
results = RadiusNeighbors.compute(
    X=X,
    Y=X,
    radius=3,
    metric='euclidean',
    # metric_kwargs=nn.effective_metric_params_,
    strategy="auto",
    return_distance=False,
    sort_results=True,
)
print(results[0].dtype)
print(results)

Probably very similar were some issues with dbscan_inner written in Cython that was saying expected np_intp got long long instead.

rth commented 1 year ago

I was not able to make the pyodide build with Pyodide dev debug working see this build log

Ahh, so the issue is at runtime after import rather than at build time?

Probably very similar were some issues with dbscan_inner written in Cython that was saying expected np_intp got long long instead.

Is there a 32 bit linux (or even windows) CI in scikit-learn currently? Sounds like 32bit arch issues.

lesteve commented 1 year ago

Ahh, so the issue is at runtime after import rather than at build time?

Yes runtime.

Is there a 32 bit linux (or even windows) CI in scikit-learn currently? Sounds like 32bit arch issues.

There is a debian 32 bit build, see here for a build log.

rth commented 1 year ago

Hmm, I tried running the test suite on alpine 32bit in docker (which is a bit closer as it uses Musl libc) and they also pass fine.

lesteve commented 1 year ago

I just tried to reproduce the problem I was seeing with doing pyodide build with a development version and I am unable to reproduce :shrug:

Maybe this was more due to the hacks I tried to reuse a debug build dist and add the few things I needed in https://github.com/lesteve/scikit-learn-tests-pyodide/blob/9ec029e49a59f8a51a208694e60a8161825136d7/.github/workflows/run-tests.yml#L44-L49

Edit: the reason I wanted to do that is that because otherwise the artifact is quite big and takes a few minutes to upload and a few minutes to download.