Avoid pip install in every notebook in one project?

[kolibril13]

In order to run a notebook from the ipyreact JupyterLite demo at https://widgetti.github.io/ipyreact, it requires the line %pip install -q ipyreact. This is currently needed in every notebook, and I think it would be great if this line would only be needed in one notebook, or even not at all.
Therefore, my question: Is there yet a way to avoid having this line in every notebook using the pyodide kernel?

[jtpio]

Maybe the simplest way for now would be to use the xeus python kernel? https://github.com/jupyterlite/xeus-python-kernel

Defining dependencies in the xeus python kernel is more natural and familiar (via an environment.yml) than with the pyodide kernel at the moment. Also it should work fine for widgets.

[bollwyvl]

Or we again consider #27, so that a deployer can choose which packages can be auto-installed on import on this kernel.

[kolibril13]

I've tried to use the xeus kernel for ipyreact, but that throws an CI error: https://github.com/kolibril13/ipyreact-testing-xeus/actions/runs/5145098489/jobs/9262258540

error    libmamba Could not solve for environment specs
    The following package could not be installed
    └─ ipyreact   does not exist (perhaps a typo or a missing channel).

[martinRenou]

It looks like ipyreact is not available on conda-forge, could be good to have it though ! @maartenbreddels friendly ping :D

[jtpio]

Ah looks like it's not packaged for conda. We should open a PR on https://github.com/conda-forge/staged-recipes to add it.

[maartenbreddels]

I suffer from conda-forge packaging fatigue :)

[maartenbreddels]

What is the reason conda-forge packages work, but pip packages don't?

[jtpio]

What is the reason conda-forge packages work, but pip packages don't?

They will eventually be supported: https://github.com/jupyterlite/xeus-python-kernel/pull/102

[martinRenou]

I suffer from conda-forge packaging fatigue :)

😅 if it's not a pure Python package, can you please make a PR to emscripten-forge as well? (kidding).

https://github.com/conda-forge/staged-recipes/pull/22993 🚀

[jtpio]

I suffer from conda-forge packaging fatigue :)

Well often you can generate most of the recipe with grayskull: https://github.com/conda/grayskull

[maartenbreddels]

Yes, but it still involves manual steps, especially when requirements/pinning changes. I've streamlined so much of the release process in all my projects, but also supporting conda-forge just sucks the energy out of me for letting me do things manually that feel unnecessary. I like conda/mamba and conda-forge a lot, but the duplicate work that is not automated (but could) just feels very unproductive.

[bollwyvl]

Veering more wildly off-topic, conda-forge.yml has grown some fairly decent python migration powers, but doesn't support all the bells and whistles of the myriad backends. For fairly straightforward build backends (setuptools, flit), the following can be effective:

# /conda-forge.yml
bot:
 automerge: true
 inspection: update-all  # update recipe using all methods

poetry plus a bunch of plugins that mutate the build outside of PEP 517? good luck. hatch plus four dependencies declared somewhere randomly outside of build-system#requires? yeah, naw.

The grayksull+ stuff will not work with the more recent "give me a version now" supported by the conda-forge-admin please update version bot PR command, but with good tests and checks, simple updates can be simple and fast.

But back to this issue about this kernel: some compelling reasons to continue to support this kernel, and adding more package management features that work seamlessly with the upstream pyodide package management system (a la #27), even after another python-based kernel grows micropip support and/or support of binary pyodide packages:

• pyodide is more widely used outside of jupyterlite
  • benefiting from a larger contributor base of folk from more than one organization
• being able to use in-development packages to get per-PR builds
  • without installing or invoking any conda-related shenanigans