pyodide build does not respect setuptools exclude_package_data

[gabrielfougeron]

Hi,

The package I'm building includes a GUI. Within the GUI, you can setup the data for computation, and either run the computation in the GUI (thanks to pyodide) or run the computation in the CLI (faster).

The project is becoming more mature and I'm entertaining the idea of releasing wheels on pypa.

I'd like the non-pyodide wheels to include a fully functional GUI. This means packaging a pyodide wheel inside another wheel (as well as the rest of the css / html / javascript).

I wrote these lines in my setup.py:

GUI_data = ['*.js','*.html','assets/**','img/**','python_scripts/**','python_dist/*.whl']

if "PYODIDE" in os.environ: # GUI stuff not needed in Pyodide whl
    exclude_package_data['choreo.GUI'].extend(GUI_data)
else:
    package_data['choreo.GUI'].extend(GUI_data)

However, excluded files are sill making their way inside the pyodide wheel! I'm using pydodide-build version 0.29.0. I think this is a bug. Or am I doing something wrong?

[agriyakhetarpal]

Hi @gabrielfougeron, thanks for the report! pyodide-build provides a build frontend just like pypa/build does, and it only invokes the build backend that builds the wheels, i.e., setuptools.build_meta in this case. If excluding package data doesn't work, I think it should be a bug with setuptools. Could you try excluding this package data via pyproject.toml? I'll take a look, still!

[agriyakhetarpal]

Also, do the files still make their way in the wheels if you build a package for Linux/macOS/Windows, with pip install build && python -m build?

[gabrielfougeron]

Thanks for the super quick answer!

Could you try excluding this package data via pyproject.toml?

I don't know how to do that programmatically (i.e. when building with pyodide but not under other platforms)

Also, do the files still make their way in the wheels if you build a package for Linux/macOS/Windows, with pip install build && python -m build?

Using python -m build under Linux works as expected : Excluded packages get excluded and included ones get included.

I should probably try to build a MWE to reproduce the bug.

[gabrielfougeron]

Snap, doing the same thing in a small new project does not reproduce the problem .... Let me get back to you when I better understand what's going on.

[agriyakhetarpal]

I don't know how to do that programmatically (i.e. when building with pyodide but not under other platforms)

When building a distribution with pyodide-build, it is backend-agnostic, it should respect the same configuration and all the settings that you supply for your build backend. To do it programmatically, you could leave it in pyproject.toml or in setuptools.setup(exclude_package_data={"": ["files, to, exclude"]} if you want to compute it dynamically somewhere earlier in the file.

No worries and no hurry on a reproducer!