finos / perspective

A data visualization and analytics component, especially well-suited for large and/or streaming datasets.
https://perspective.finos.org/
Apache License 2.0
7.95k stars 1.06k forks source link

Switching between pyodide + regular python builds requires cleaning / rebuilding native artifacts #2420

Open tomjakubowski opened 8 months ago

tomjakubowski commented 8 months ago

Bug Report

little annoyance when switching between pyodide + native python development

Steps to Reproduce:

  1. create an "editable install" of perspective (pip install -e python/perspective)
  2. use the editable install somewhere (e.g. in a jupyter project)
  3. run a pyodide build
  4. editable python install is broken because .so is WASM, not mach-o (or what ever)

ImportError: dlopen(/Users/tom/perspective/perspective/python/perspective/perspective/table/libpsppy.so, 0x0002): tried: '/Users/tom/perspective/perspective/python/perspective/perspective/table/libpsppy.so' (not a mach-o file), '/System/Volumes/Preboot/Cryptexes/OS/Users/tom/perspective/perspective/python/perspective/perspective/table/libpsppy.so' (no such file), '/Users/tom/perspective/perspective/python/perspective/perspective/table/libpsppy.so' (not a mach-o file)

the remedy is to rebuild all the python so a native (mach-o in my case) .so is at that path, instead of an emscripten/wasm .so

Expected Result:

pyodide build + native python build are independent and their compiled artifacts do not interfere with one another

Actual Result:

ImportError: dlopen(/Users/tom/perspective/perspective/python/perspective/perspective/table/libpsppy.so, 0x0002): tried: '/Users/tom/perspective/perspective/python/perspective/perspective/table/libpsppy.so' (not a mach-o file), '/System/Volumes/Preboot/Cryptexes/OS/Users/tom/perspective/perspective/python/perspective/perspective/table/libpsppy.so' (no such file), '/Users/tom/perspective/perspective/python/perspective/perspective/table/libpsppy.so' (not a mach-o file)

Environment:

latest master, macos 13.3.1

Additional Context:

I propose separating C++ build artifacts in our working tree by target architecture

timkpaine commented 8 months ago

Wouldn't this also be true of different python versions?