Open rth opened 1 year ago
Thanks for the proposal @bollwyvl ! From JupyterLite side I can see why one could be tempted to do something like this, minimally interacting with upstream pyodide-lock.json
and having something that works reasonably fast.
On one side I agree that no one wants to create yet another package manager, on the other I'm concerned that while such a solution should be easy to start with, it might be difficult to maintain or debug over time. Generating a lock file for packages is fairly standard task, and there existing tools we could re-use. While with this approach is fairly new packaging concept and so any issues would be ours. Unless you are aware of any packaging project that does similar things?
With #7 done, to be able to move further to something usable outside pyodide-build
, would probably be to have a public API that can generate a single PackageSpec
from an at-rest .whl
, not yet on PyPI, already in a pyodide-lock.json
, or otherwise uncharacterized.
Some thoughts on API design:
Path
pyodide_lock.spec_from_wheel
as a standalone functionpyodide_lock.PackageSpec.from_wheel
as a class methodOut of scope:
.whl
would incur a lot more complexitypyodide-lock.json
I've got a working, but potentially out-of-date, strawman but I'm not precious about it... if there's another one in pyodide-build
, that's fine, too.
The strawman implementation uses pkginfo
(MIT, 30k .whl
, no deps) and packaging
(BSD, 50k .whl
, no deps. These could be in an extra
dependency, lazily imported inside the implementation, as they would not be required to validate... but then pydantic
is at least 150kb, so maybe who's counting (and [extras]
are... kinda broken).
Added the above as a draft on #18... probably won't have any more time to respond this weekend, but had this kinda working in #17 before descoping, so figured I'd push it up so it could be discussed later.
As proposed by @bollwyvl https://github.com/pyodide/pyodide-lock/issues/4#issuecomment-1608118850
The way the in-flight jupyterlite PR works is by:
extraLockUrls: []
null
(ick!)The concrete things this solves there:
pyodide
stdlibjedi
/parso
(but still downloads them)jedi
work well enough in jupyterlite to use, so would rather replace it with dummy shims to save a couple megabytes on the wireAlternative proposals: https://github.com/pyodide/pyodide-lock/issues/8 https://github.com/pyodide/pyodide-lock/issues/10