Error Setting Up Development Environment: `pip install -e .` in root throws ValueError

arjxn-py commented 1 month ago

Greetings of the day 👋🏻 TL;DR: I encountered an error while trying to set up the development environment for the project. I was following the contribution guide and ran pip install -e .. The error message indicates that the build system is unable to determine which files to include in the wheel package due to missing or incorrectly specified file selection options in the pyproject.toml:

Error logs

× Preparing editable metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [52 lines of output]
    Traceback (most recent call last):
      File "/home/arjxnpy/miniconda3/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 167, in prepare_metadata_for_build_editable
        hook = backend.prepare_metadata_for_build_editable
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    AttributeError: module 'hatchling.build' has no attribute 'prepare_metadata_for_build_editable'   

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/home/arjxnpy/miniconda3/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
        main()
      File "/home/arjxnpy/miniconda3/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/arjxnpy/miniconda3/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 176, in prepare_metadata_for_build_editable
        whl_basename = build_hook(metadata_directory, config_settings)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/tmp/pip-build-env-93bum18r/overlay/lib/python3.12/site-packages/hatchling/build.py", line 83, in build_editable
        return os.path.basename(next(builder.build(directory=wheel_directory, versions=['editable'])))
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
      File "/tmp/pip-build-env-93bum18r/overlay/lib/python3.12/site-packages/hatchling/builders/plugin/interface.py", line 155, in build
        artifact = version_api[version](directory, **build_data)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/tmp/pip-build-env-93bum18r/overlay/lib/python3.12/site-packages/hatchling/builders/wheel.py", line 494, in build_editable
        return self.build_editable_detection(directory, **build_data)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/tmp/pip-build-env-93bum18r/overlay/lib/python3.12/site-packages/hatchling/builders/wheel.py", line 505, in build_editable_detection
        for included_file in self.recurse_selected_project_files():
      File "/tmp/pip-build-env-93bum18r/overlay/lib/python3.12/site-packages/hatchling/builders/plugin/interface.py", line 180, in recurse_selected_project_files
        if self.config.only_include:
           ^^^^^^^^^^^^^^^^^^^^^^^^
      File "/tmp/pip-build-env-93bum18r/overlay/lib/python3.12/site-packages/hatchling/builders/config.py", line 806, in only_include
        only_include = only_include_config.get('only-include', self.default_only_include()) or self.packages
                                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/tmp/pip-build-env-93bum18r/overlay/lib/python3.12/site-packages/hatchling/builders/wheel.py", line 260, in default_only_include
        return self.default_file_selection_options.only_include
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/arjxnpy/miniconda3/lib/python3.12/functools.py", line 995, in __get__
        val = self.func(instance)
              ^^^^^^^^^^^^^^^^^^^
      File "/tmp/pip-build-env-93bum18r/overlay/lib/python3.12/site-packages/hatchling/builders/wheel.py", line 248, in default_file_selection_options
        raise ValueError(message)
    ValueError: Unable to determine which files to ship inside the wheel using the following heuristics: https://hatch.pypa.io/latest/plugins/builder/wheel/#default-file-selection

    The most likely cause of this is that there is no directory that matches the name of your project (jupytercad_root).

    At least one file selection option must be defined in the `tool.hatch.build.targets.wheel` table, see: https://hatch.pypa.io/latest/config/build/

    As an example, if you intend to ship a directory named `foo` that resides within a `src` directory located at the root of your project, you can define the following:

    [tool.hatch.build.targets.wheel]
    packages = ["src/foo"]
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

System Configuration:

Operating System: Tested on both Windows 11 and WSL (Ubuntu 22.04)
Python Version: 3.11

arjxn-py commented 1 month ago

Initially I tried to to some workaround as I wasn't aware about the existence of dev-install.py, I tried defining tool.hatch.build.targets.wheel in the pyproject-

[tool.hatch.build.targets.wheel]
packages = [
    "python/jupytercad",
    "python/jupytercad_app",
    "python/jupytercad_core",
    "python/jupytercad_lab"
]
include = [
    "python/jupytercad/**/*.py",
    "python/jupytercad_app/**/*.py",
    "python/jupytercad_core/**/*.py",
    "python/jupytercad_lab/**/*.py"
]

But project was not being installed it was very quickly finishing up with the following logs:

Obtaining file:///C:/Users/Arjun/Desktop/Arjun/JupyterCAD
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Installing backend dependencies ... done
  Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: jupytercad_root
  Building editable for jupytercad_root (pyproject.toml) ... done
  Created wheel for jupytercad_root: filename=jupytercad_root-2.0.1-py2.py3-none-any.whl size=3633 sha256=5d0154eb3bdb87b13bde89383145d3aff422b80ab7ca342bc8bff58195858baa
  Stored in directory: C:\Users\Arjun\AppData\Local\Temp\pip-ephem-wheel-cache-p_ncnlug\wheels\09\d9\53\5aff060f2802ec5ca276c86b3bf0f84b1d59d9f65bb57cbaee
Successfully built jupytercad_root
Installing collected packages: jupytercad_root
  Attempting uninstall: jupytercad_root
    Found existing installation: jupytercad-root 2.0.1
    Uninstalling jupytercad-root-2.0.1:
      Successfully uninstalled jupytercad-root-2.0.1
Successfully installed jupytercad_root-2.0.1

arjxn-py commented 1 month ago

Later I tried using dev-install.py & I noticed that it needed jupyterlab to be installed already for jlpm & if not installed it was throwing error - No such file or directory: 'jlpm'. I didn't actually need to execute any further command from Contributing.md as jupyter cad worked for me nicely and I was able to locally setup the project successfully. Hence it might be the case that the contribution guide is a little bit outdated if we've moved our dependency from the root pyproject.toml to install-dev.py for development environment setup.

Apologies in advance in case this was not the valid approach to have a follow up on this. I'd be so happy to have a response. Thanks a lot 💐

trungleduc commented 1 month ago

Thank you for testing out JupyterCAD, indeed the contribution guide is outdated. A PR to update this guide is very welcome

arjxn-py commented 1 month ago

I was writing this on #379, but as it is already merged so I'm posting this here 😄

However there are some pain-points that I'd like to highlight which I experienced while trying to make minimal changes to the UI render them in browser with jlpm run watch.

I'm yet not been able to exactly figure out which is the most suitable directory to stably run jlpm run watch i.e. there is root, python/jupytercad_core, python/jupytercad_app, python/jupytercad_lab (haven't tried running in this one yet) & then there is packages/* (I've tried running watch in packages/base as I've made changes in packages/base/src/panelview/objectproperties.tsx)
- The most responsive one I felt was python/jupytercad_core when i watched some changes in python/jupytercad_core/src/jcadplugin/plugins.ts
Re-compilation of the project after any change takes around a minute which is pretty reasonable considering the size of the project. But the issue I faced was that even after compilation I can't see the changes i made in the browser, it takes X amount of time(varying b/w 5-10 minutes) and a number of tries to show up on the browser. I'd like to confirm if it's the same situation on your end as well or if I should be doing something incorrectly.

I'm a little reluctant to open a new issue for this because I'm not sure if it's an issue with me only. I'm currently using WSL2 (Ubuntu 22.04) on Windows 11 as I'm using Docker ce right now, I can fully switch to windows too if you suggest after moving to Docker Desktop.

Sorry for the long text, I got a little carried away ^^

martinRenou commented 1 month ago

Re-compilation of the project after any change takes around a minute

That's usually what I do when working on JupyterCAD, I feel like it takes less than a minute on my side. I've not used the watch command in a while and I'm not sure it's properly setup, maybe @trungleduc knows if it works.

Wait until you want to change the symbols we compile in the OCC WASM build, you'll see that rebuilding takes maybe 10 minutes 😆 And actually I wonder if this could be the reason for you to wait 5/10 minutes with the watch command, could it trigger an OCC rebuild everytime?

martinRenou commented 1 month ago

Our Open Cascade build does not have a watch script https://github.com/jupytercad/JupyterCAD/blob/main/packages/opencascade/package.json#L31 so I don't know how lerna deals with it

trungleduc commented 1 month ago

calling jlpm run watch at the root is the correct way to watch the project. But lerna watch does not work well with WSL

trungleduc commented 1 month ago

But the issue I faced was that even after compilation I can't see the changes i made in the browser

did you disable your browser cache?

arjxn-py commented 1 month ago

could it trigger an OCC rebuild everytime?

I couldn't see much logs to verify this, i guess the verbosity is low by default is there some way that it could be increased? I'll definitely follow up on this further

did you disable your browser cache?

No I didn't, I just confirmed. I'd do it again with disabled browser cache. And I think this is also something we should include in the documentation as a note.

calling jlpm run watch at the root is the correct way to watch the project. But lerna watch does not work well with WSL

Ah I see, I think then I'd rather try migrating/replicating the development environment fully on windows too and then test again.

I've not used the watch command in a while

Is there something else that you use more often? If yes, may I also know 🙂.

Thanks both :)

arjxn-py commented 1 month ago

Hi there, Good News(for me 😅) I have set-up the project fully on windows and It's compiling much faster than WSL.

However, I think there's one issue that hasn't been addressed yet & that's about $LERNA_PACKAGE_NAME. i.e.:

When I watch the build in root & modify any files, it throws an error saying -

lerna notice filter including "$LERNA_PACKAGE_NAME"
lerna ERR! EFILTER No packages remain after filtering [ '$LERNA_PACKAGE_NAME' ]

I feel it's because LERNA_PACKAGE_NAME is not set for me locally but I'm not exactly sure what to set it.

But just to verify that jlpm run watch is working nicely for me, in package.json in temporarily replaced

lerna watch -- lerna run build --scope=\\$LERNA_PACKAGE_NAME --include-dependents with
lerna watch -- lerna run build --scope=@jupytercad/jupytercad-core --include-dependents

& It now works as expected with no more issue of being able to see changes on browser. Really glad, Thanks a lot both 💐

arjxn-py commented 1 month ago

Hi @trungleduc @martinRenou, sorry to ping you.

Just a gentle follow up on this -

I feel it's because LERNA_PACKAGE_NAME is not set for me locally but I'm not exactly sure what to set it.

arjxn-py commented 1 month ago

I feel it's because LERNA_PACKAGE_NAME is not set for me locally but I'm not exactly sure what to set it.

Small Update: I just removed --scope=\\$LERNA_PACKAGE_NAME for now, i.e. lerna watch -- lerna run build --include-dependents & it works for me & compiles all 7 projects at the same time.

However there's a very small issue now, that the project is always re-compiling without me even modifying anything after the first change I make. Fortunately it doesn't prevent me from developing the code :)

trungleduc commented 1 month ago

LERNA_PACKAGE_NAME is defined automatically by lerna, you should not remove it from the command.

https://lerna.js.org/docs/features/workspace-watching#watch-environment-variables

arjxn-py commented 1 month ago

LERNA_PACKAGE_NAME is defined automatically by lerna, you should not remove it from the command.

https://lerna.js.org/docs/features/workspace-watching#watch-environment-variables

Thanks so much for the response 🚀, I think i found the issue with the help of this.
Quoting from lerna doc link you've sent:

If you use Lerna in Windows, you must frame environment variables in '%'. For example:
$ lerna watch -- lerna run build --scope=%LERNA_PACKAGE_NAME% --include-dependents

arjxn-py commented 3 weeks ago

JupyterGIS's Contributing Guide also looks outdated. Would you mind if I fix that there too?

martinRenou commented 3 weeks ago

JupyterGIS's Contributing Guide also looks outdated. Would you mind if I fix that there too?

We don't mind at all! Thanks a lot for this

arjxn-py commented 2 weeks ago

Should be good to close this one now. Thanks a lot both 💐

jupytercad / JupyterCAD

Error Setting Up Development Environment: `pip install -e .` in root throws ValueError #378

Error logs

System Configuration: