Cannot install python packages on Windows

[esaulenka]

Describe the bug Looks like gradle accidentely replaces slashes in the file path, and it causes fails in the pip install

To Reproduce Steps to reproduce the behavior:

1. use Win10 with MSYS2 unix-like environment (bash, pacman, etc)
2. clone repo
3. gradle fetch dependencies
4. gradle buildGhidra
5. check buildPyPackage step

> Configure project :
Gradle: 8.7
Python3 command:  python3 (version 3.11)
Python3 pip version: 24.0
OS: Windows 10
Arch: amd64 (gradle: x86-64)
Using platform: win_x86_64
Searching for latest Visual Studio and required components...
  -> Installation Directory: C:\Program Files (x86)\Microsoft Visual Studio\2017\Community
  -> VCTools Version (default): 14.16.27023
  -> SDK Directory (default): C:\Program Files (x86)\Windows Kits\10\
  -> SDK Version (default): 10.0.17763.0
  -> SDK Version (override): N/A

> Task :Debugger-agent-dbgeng:buildPyPackage
Looking in links: d:\Work\Ghidra\ghidra\dependencies\Debugger-rmi-trace
Processing /d/Work/Ghidra/ghidra/Ghidra/Debug/Debugger-agent-dbgeng/build/pypkg
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
  error: subprocess-exited-with-error

  ├Ч pip subprocess to install build dependencies did not run successfully.
  тФВ exit code: 1
  тХ░тФА> [4 lines of output]
      Looking in links: d:\Work\Ghidra\ghidra\dependencies\Debugger-rmi-trace
      Processing /D:\Work\Ghidra\ghidra\dependencies\Debugger-rmi-trace/setuptools-68.0.0-py3-none-any.whl
      ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/D:\\Work\\Ghidra\\ghidra\\dependencies\\Debugger-rmi-trace/setupto
ols-68.0.0-py3-none-any.whl'

      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

├Ч pip subprocess to install build dependencies did not run successfully.
тФВ exit code: 1
тХ░тФА> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

> Task :Debugger-agent-dbgeng:buildPyPackage FAILED

In this environment all paths should be in unix-style, i.e. /d/Work/Ghidra/..., but gradle somehow converts part of it into microsoft-style, that cannot be accepted by pip install.

Ghidra version: 2bf16b0c

[astrelsky]

You can skip it using -x buildPyPackage for now if you want the build to proceed without them.

[d-millar]

Hmmm, that's very odd. I just ran the same and got:

Configure project : Gradle: 8.7 Python3 command: py (version 3.12) Python3 pip version: 24.0 OS: Windows Server 2019 Arch: amd64 (gradle: x86-64) Using platform: win_x86_64 Searching for latest Visual Studio and required components... -> Installation Directory: C:\Program Files\Microsoft Visual Studio\2022\Community -> VCTools Version (default): 14.30.30705 -> SDK Directory (default): C:\Program Files (x86)\Windows Kits\10\ -> SDK Version (default): 10.0.22621.0 -> SDK Version (override): N/A

Task :Debugger-agent-dbgeng:buildPyPackage Looking in links: c:\git\ghidra\ghidra\dependencies\Debugger-rmi-trace Processing c:\git\ghidra\ghidra\ghidra\debug\debugger-agent-dbgeng\build\pypkg Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Installing backend dependencies: started Installing backend dependencies: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Building wheels for collected packages: ghidradbg Building wheel for ghidradbg (pyproject.toml): started Building wheel for ghidradbg (pyproject.toml): finished with status 'done' Created wheel for ghidradbg: filename=ghidradbg-11.1-py3-none-any.whl size=63927 sha256=e2abcf74ee b96d7d720f62993da30a1d5145b925fb5ac5bbe45f31963ad76e1f Stored in directory: C:\msys64\tmp\pip-ephem-wheel-cache-p1c1nab_\wheels\c4\3b\59\698cb8b88bfb110c 14b6b072e3eb373d1f9b64a54a5cb31440 Successfully built ghidradbg

...

Task :Debugger-rmi-trace:buildPyPackage Looking in links: c:\git\ghidra\ghidra\dependencies\Debugger-rmi-trace Processing c:\git\ghidra\ghidra\ghidra\debug\debugger-rmi-trace\build\pypkg Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Installing backend dependencies: started Installing backend dependencies: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Building wheels for collected packages: ghidratrace Building wheel for ghidratrace (pyproject.toml): started Building wheel for ghidratrace (pyproject.toml): finished with status 'done' Created wheel for ghidratrace: filename=ghidratrace-11.1-py3-none-any.whl size=16747 sha256=809509 9ac84ae6e14404ad50071e81f52080be542c0c4d014cc3f42b9a22a4b4 Stored in directory: C:\msys64\tmp\pip-ephem-wheel-cache-s1008hvm\wheels\b9\db\a5\8d56569d9dd0f594 8b91a4eff0bfd54a6a506c8446f5e05463 Successfully built ghidratrace

Slightly newer version of python (doubt that matters) and Windows Server vs. Win 10 proper (but again seems unlikely to be the culprit). My msys2 executable is dated 5/14/22 - pulled relatively recently (maybe Jan.?). Must be something else different here, but I'm not seeing it....

[ryanmkurtz]

I can't reproduce this with Gradle 8.7 on Windows 10, from within a Git Bash shell. I used f59f9fbd14a638701a30a620028cd4b1b8605df3.

[ryanmkurtz]

What kind of Python are you using? Official release from python.org? Microsoft Store version? Something else?

[d-millar]

Following up on @ryanmkurtz's comment, my experiment used the version of python that came with the MSYS distribution. My python versions differed between that installed on Windows proper (3.12), the default version in the Mingw64 env (3.10), and the default version in the MSYS2 env (3.11). That said, each build worked correctly using the default version of python for each environment. Am guessing it would be pretty easy to accidentally get one of the non-default versions in the path and executing in each environment. Any chance that's true your case?

[astrelsky]

Following up on @ryanmkurtz's comment, my experiment used the version of python that came with the MSYS distribution. My python versions differed between that installed on Windows proper (3.12), the default version in the Mingw64 env (3.10), and the default version in the MSYS2 env (3.11). That said, each build worked correctly using the default version of python for each environment. Am guessing it would be pretty easy to accidentally get one of the non-default versions in the path and executing in each environment. Any chance that's true your case?

I planned on waiting until tonight to get the exact error I encountered the other day. But for what it's worth, I just use the python distribution from python.org and run gradle from the command prompt.

I assumed this would be easy to reproduce so I never bothered reporting this myself.

[astrelsky]

Following up on @ryanmkurtz's comment, my experiment used the version of python that came with the MSYS distribution. My python versions differed between that installed on Windows proper (3.12), the default version in the Mingw64 env (3.10), and the default version in the MSYS2 env (3.11). That said, each build worked correctly using the default version of python for each environment. Am guessing it would be pretty easy to accidentally get one of the non-default versions in the path and executing in each environment. Any chance that's true your case?

I planned on waiting until tonight to get the exact error I encountered the other day. But for what it's worth, I just use the python distribution from python.org and run gradle from the command prompt.

I assumed this would be easy to reproduce so I never bothered reporting this myself.

Well it turns out, in my case at least, I just needed to run gradle -I gradle/support/fetchDependencies.gradle init again.

[esaulenka]

Sorry for huge delay. Finally I found some weird Python 3.6 in the Add and remove programs menu. I don't know where it was located (it's not present in the %PATH%, and it's not accessible from command line). When I removed it, I successfully compiled full Ghidra on that machine. Why Gradle tried to use it even if it said that it found python 3.11, I have no explanation. Anyway, sorry for false warning.