search

microsoft / vscode-python-web-wasm

An extension allows to run Python code in a Web browser using WebAssembly technology
MIT License
77 stars 11 forks source link

Develop a way to figure out the maximal list of projects that _may_ be compatible with WASI #42

Open brettcannon opened 1 year ago

brettcannon commented 1 year ago

We are after a list of potential projects that could work with Python under WASI. There's essentially 6 layers to this, each one more specific than the previous, but all accurate for what they claim:

  1. All projects on PyPI
  2. Projects on PyPI that have a pure Python wheel
  3. Projects who only have dependencies that are pure Python wheels
  4. Importation of the top-level project succeeds
  5. Test suite passes
  6. Projects declare their support

We can calculate layers 1 and 2 easily thanks to https://github.com/sethmlarson/pypi-data (although with slightly outdated data). We can calculate layer 2 with sqlite3 pypi.db 'SELECT DISTINCT package_name FROM wheels WHERE python = "py3" OR python = "py2.py3" ORDER BY package_ name ASC' > project_with_pure_wheels.txt.

Layer 3 requires writing a resolver which can resolve for a specific environment and only for specific wheel tags (which helps out with lock files, but it's still work). Layer 4 probably can be automated with introspection inside wheel files and importing module(s) based on what's found in them (luckily WASI is sandboxed, so this should actually be a safe operation). Layer 5 will either have to be some crazy set of heuristics or we accept it as being a per-project thing (plus being able to actually run pytest under WASI). And for layer 6, I have opened https://discuss.python.org/t/do-we-want-classifiers-for-webassembly-on-pypi/22712 to discuss it with the community.

brettcannon commented 1 year ago

https://github.com/pypa/trove-classifiers/pull/129 adds Trove classifiers for declaring WebAssembly support on PyPI. They will eventually show up at https://pypi.org/classifiers/ .