Patching scanpy for xeus

scverse / scanpy

Single-cell analysis in Python. Scales to >1M cells.

https://scanpy.readthedocs.io

BSD 3-Clause "New" or "Revised" License

1.81k stars 584 forks source link

Patching scanpy for xeus #2667

Open VPetukhov opened 9 months ago

VPetukhov commented 9 months ago

What kind of feature would you like to request?

Other?

Please describe your wishes

Hi! I tried running scanpy using JupyterLite for teaching purposes. However, it fails, as only noarch packages are currently supported in xeus. From my error log it seems the only non-noarch dependency is h5py, which is not really needed for teaching.

Would it be possible to make h5py an optional dependency on conda-forge somehow? It would be really helpful for many students to be able to try scanpy without the need to install anything locally.

flying-sheep commented 8 months ago

From my error log it seems the only non-noarch dependency is h5py

That’s surprising! I think numba is our most complex dependency, and umap’s dependency PyNNDescent is also compiled. I think if this isn’t a mistake and it’s really just about h5py, we can think about it.

Trying to install scanpy and following JupyterLite’s debug instructions gives:

ValueError: Can't find a pure Python 3 wheel for: 'umap-learn>=0.3.10', 'session-info', 'numba>=0.41.0'
See: https://pyodide.org/en/stable/usage/faq.html#why-can-t-micropip-find-a-pure-python-wheel-for-a-package

(session-info isn’t a problem, it’s just an old package that doesn’t publish wheels)

ivirshup commented 8 months ago

numba really is the main blocker, see also:

Tracking issue: https://github.com/pyodide/pyodide/issues/621
Potential PR: https://github.com/emscripten-forge/recipes/pull/168
- Unfortunately, the author recently founded prefix.dev so may not have time to complete this 😢

pynndescent also depends on numba.

I am hopeful that numba's new AOT backend may make this easier in the future. Unclear how painful it would be to distribute binaries capable of multithreading though.

I think h5py would also be pretty reasonable to make optional if we could otherwise run in pyodide, since we wouldn't have a filesystem anyways. Though I think pytables runs in pyodide, so it's probably reasonable to get h5py there too.

ivirshup commented 8 months ago