NotADirectoryError when installing from source distribution

Discussed in https://github.com/reactive-python/reactpy/discussions/1180

^{Originally posted by **thegamecracks** January 3, 2024} When attempting to install the package specifically using its source distribution rather than the built wheels, the following error occurs: ```py >pip install reactpy --no-binary reactpy Collecting reactpy Using cached reactpy-1.0.2.tar.gz (123 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... error error: subprocess-exited-with-error × Preparing metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [26 lines of output] Traceback (most recent call last): File "..\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in main() File "..\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 152, in prepare_metadata_for_build_wheel whl_basename = backend.build_wheel(metadata_directory, config_settings) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Temp\pip-build-env-qtfs6sj6\overlay\Lib\site-packages\hatchling\build.py", line 58, in build_wheel return os.path.basename(next(builder.build(directory=wheel_directory, versions=['standard']))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Temp\pip-build-env-qtfs6sj6\overlay\Lib\site-packages\hatchling\builders\plugin\interface.py", line 147, in build build_hook.initialize(version, build_data) File "..\AppData\Local\Temp\pip-build-env-qtfs6sj6\overlay\Lib\site-packages\hatch_build_scripts\plugin.py", line 51, in initialize run(cmd, cwd=str(work_dir), check=True, shell=True) # noqa: S602 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 548, in run with Popen(*popenargs, **kwargs) as process: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 1026, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "..\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 1538, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ NotADirectoryError: [WinError 267] The directory name is invalid [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details. ``` After some debugging, it appears that hatch is trying to build the JavaScript client and assumes `src/js/` is present, but in the source distribution that directory is missing. Here is the build configuration in question: https://github.com/reactive-python/reactpy/blob/3a3ad3f706477642133190c08bbe1e43a991bfa3/src/py/reactpy/pyproject.toml#L127-L136 Right now, both the sdist and wheels uploaded on PyPI contain the `reactpy/_static/` files generated by `npm run build` ([inspector](https://inspector.pypi.io/project/reactpy/1.0.2/)), but hatch is configured to build the client even when the source distribution already has the output files. This isn't a major issue since reactpy's wheels aren't dependent on system architecture, but it would be nice to have this part of the build process fixed (or at least documented). The ideal solution would be to include the JS client's source code in the sdist and only build it when installing from source / generating the wheel. I thought of suggesting this: ```diff + [tool.hatch.build.targets.sdist.force-include] + "../../js" = "js" + + [tool.hatch.build.targets.wheel.force-include] + "reactpy/_static" = "reactpy/_static" # bypass .gitignore - [[tool.hatch.build.hooks.build-scripts.scripts]] - work_dir = "../../js" + [[tool.hatch.build.targets.wheel.hooks.build-scripts.scripts]] + work_dir = "js" ``` However it breaks installing from source (`pip install src/py/reactpy`) as pip skips building the sdist and therefore won't copy `js/` to the correct place. I also don't know if there's an alternative to [`force-include`](https://hatch.pypa.io/latest/config/build/#forced-inclusion) which can map paths outside of the project root while respecting .gitignore. Absent that, it would be easy to accidentally pollute the sdist with `node_modules`. In theory a symbolic link to `src/js/` could remedy this along with slightly different changes to pyproject.toml, specifically the use of `only-include`: ```diff + [tool.hatch.build.targets.sdist] + only-include = ["js", "reactpy"] + + [tool.hatch.build.targets.wheel.force-include] + "reactpy/_static" = "reactpy/_static" # bypass .gitignore - [[tool.hatch.build.hooks.build-scripts.scripts]] - work_dir = "../../js" + [[tool.hatch.build.targets.wheel.hooks.build-scripts.scripts]] + work_dir = "js" ``` But while it works when generating the sdist / building from sdist ([`python -m build`](https://build.pypa.io/en/stable/)), the `npm run build` command fails when installing from source or building the wheel from source (`hatch build`/`python -m build --wheel`):

FWIW, in an unrelated package, [sardine-web](https://github.com/sardine-system/sardine-web/tree/v1.1.0/), I structured it so the JS client's source code was inside the package itself, but [setup.py](https://github.com/sardine-system/sardine-web/blob/v1.1.0/setup.py#L56-L67) would build the client and remove the source code afterwards, leaving only the source code in the sdist and its compiled result in the wheel. In hindsight, it probably would have been easier had I left the client source code outside of the package and only included it in the sdist via MANIFEST.in. Given hatch's own mechanisms for controlling distributions, plus the convenience of `hatch-build-scripts`, it might be easier to move `src/js/` beside the `src/py/reactpy/reactpy/` package and follow the [previous suggestion](#include-js-in-sdist) to pyproject.toml, rather than try to find a workaround with the current layout.

reactive-python / reactpy

NotADirectoryError when installing from source distribution #1184

Discussed in https://github.com/reactive-python/reactpy/discussions/1180