[venv] Unable to run make dist due to `No rule to make target`

[chiawchen]

Description

Hi, I'm trying to do some modification on top of main branch, and then make a dist wheel for later pip installation. However, I encounter make failure,

make: *** No rule to make target '.venv/lib/python3.8/site-packages/setuptools/script', needed by 'dist/jupyter_enterprise_gateway-3.1.0.dev0-py3-none-any.whl'.  Stop.

I've ensure running build in a clean virtualenv by doing

python3 -m venv .venv 
source .venv/bin/activate

Reproduce

(.venv) ~/code/enterprise_gateway$ make dist
pre-commit run --all-files
fix end of files.........................................................Passed
check for case conflicts.................................................Passed
check that executables have shebangs.....................................Passed
fix requirements.txt.....................................................Passed
check for added large files..............................................Passed
check for case conflicts.................................................Passed
check toml...............................................................Passed
check yaml...............................................................Passed
debug statements (python)................................................Passed
forbid new submodules................................(no files to check)Skipped
trim trailing whitespace.................................................Passed
black....................................................................Passed
isort....................................................................Passed
prettier.................................................................Passed
pyupgrade................................................................Passed
doc8.....................................................................Passed
flake8...................................................................Passed
Check GitHub Workflows...................................................Passed
make: *** No rule to make target '.venv/lib/python3.8/site-packages/setuptools/script', needed by 'dist/jupyter_enterprise_gateway-3.1.0.dev0-py3-none-any.whl'.  Stop.

Expected behavior

Build succeed without any issues

Context

• Operating System and version: Ubuntu 20.04.4 LTS
• Browser and version: N/A
• Jupyter Server version: N/A
• setuptools version: 65.3.0

Troubleshoot Output

Paste the output from running `jupyter troubleshoot` from the command line here.
You may want to sanitize the paths in the output.

Command Line Output

Paste the output from your command line running `jupyter lab` here, use `--debug` if possible.

Browser Output

Paste the output from your browser Javascript console here, if applicable.

[kevin-bates]

Hi @chiawchen - this is interesting. The next output you should see after the pre-commit phase is the following (I'm also including a portion of the pre-commit phase):

prettier.................................................................Passed
pyupgrade................................................................Passed
doc8.....................................................................Passed
flake8...................................................................Passed
Check GitHub Workflows...................................................Passed
pip install build && python -m build --wheel . \
    && rm -rf *.egg-info && chmod 0755 dist/*.*
Requirement already satisfied: build in /opt/miniconda3/envs/eg-dev/lib/python3.9/site-packages (0.8.0)
Requirement already satisfied: pep517>=0.9.1 in /opt/miniconda3/envs/eg-dev/lib/python3.9/site-packages (from build) (0.12.0)
Requirement already satisfied: tomli>=1.0.0 in /opt/miniconda3/envs/eg-dev/lib/python3.9/site-packages (from build) (2.0.1)
Requirement already satisfied: packaging>=19.0 in /opt/miniconda3/envs/eg-dev/lib/python3.9/site-packages (from build) (21.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/miniconda3/envs/eg-dev/lib/python3.9/site-packages (from packaging>=19.0->build) (3.0.9)
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (hatchling>=0.25)
* Getting dependencies for wheel...
* Building wheel...
Successfully built jupyter_enterprise_gateway-3.1.0.dev0-py3-none-any.whl

So I'm curious if you're able to run pip install build from your venv?

I'm going to create a fresh conda env (I don't use virtual envs) for python 3.8 (mine is using 3.9, but CI runs >= 3.7) to see if I can learn anything during that exercise.

[kevin-bates]

When creating a clean conda (mamba) env for 3.8, I still don't see any issues. My setuptools is 65.3.0 and my pip is 22.2.2, in case that helps. Hopefully running pip install build will provide some clues for you.

[chiawchen]

interesting, I'm able to run successfully with the full command you provide @kevin-bates

(.venv) ~/code/enterprise_gateway$ pip install build && python -m build --wheel .
Requirement already satisfied: build in ./.venv/lib/python3.8/site-packages (0.8.0)
Requirement already satisfied: pep517>=0.9.1 in ./.venv/lib/python3.8/site-packages (from build) (0.13.0)
Requirement already satisfied: packaging>=19.0 in ./.venv/lib/python3.8/site-packages (from build) (21.3)
Requirement already satisfied: tomli>=1.0.0; python_version < "3.11" in ./.venv/lib/python3.8/site-packages (from build) (2.0.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./.venv/lib/python3.8/site-packages (from packaging>=19.0->build) (3.0.9)
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (hatchling>=0.25)
* Getting dependencies for wheel...
* Building wheel...
Successfully built jupyter_enterprise_gateway-3.1.0.dev0-py3-none-any.whl

but not able to run it through make dist, i also upgrade my pip to latest one 22.2.2

[kevin-bates]

Just fyi... I created a venv and reproduce the issue (although it complains about a different file). Running make -n dist is sufficient to see the problem...

$ make -n dist
pip install -q pre-commit
pre-commit run --all-files
gmake: *** No rule to make target '.venv/lib/python3.8/site-packages/setuptools/command/launcher', needed by 'dist/jupyter_enterprise_gateway-3.1.0.dev0-py3-none-any.whl'.  Stop.

Interestingly, if I deactivate my venv, I still see the same issue which makes me think it might be GNU make related because I happened to create my venv in my EG working directory, so the .venv tree was present where I invoke make. If I remove that .venv hierarchy (and activating my previous conda env), I get the expected results:

$ make -n dist
pip install -q pre-commit
pre-commit run --all-files
make VERSION=3.1.0.dev0 TAG=dev SPARK_VERSION=3.2.1 -C  etc kernelspecs_all
make VERSION=3.1.0.dev0 TAG=dev SPARK_VERSION=3.2.1 -C  etc kernelspecs_yarn
make VERSION=3.1.0.dev0 TAG=dev SPARK_VERSION=3.2.1 -C  etc kernelspecs_conductor
make VERSION=3.1.0.dev0 TAG=dev SPARK_VERSION=3.2.1 -C  etc kernelspecs_kubernetes
make VERSION=3.1.0.dev0 TAG=dev SPARK_VERSION=3.2.1 -C  etc kernelspecs_docker
make VERSION=3.1.0.dev0 TAG=dev SPARK_VERSION=3.2.1 -C  etc kernel_image_files
helm lint etc/kubernetes/helm/enterprise-gateway

Also note that if I create a venv elsewhere, like in my home directory, make -n dist produces the expected results. That is, the only way I reproduce this is by creating the virtual env in the EG working directory.

I notice that activation prepends the PATH with the venv/bin directory and adds VIRTUAL_ENV to the environment. But deactivation removes those things, yet the make -n dist still has issues UNTIL the venv directory is removed (when present in the working directory) - very bizarre!

For now, your workaround would be to create your venv in a different directory.

I don't think this is anything we as a project should spend more time on unless you're NOT creating your venv in the working directory - in which case, we probably have more comparisons to perform.

[chiawchen]

totally agree with you, thanks a lot for figuring out this tricky setup with me!