search

pothosware / PothosSDR

Pothos SDR windows development environment
https://github.com/pothosware/PothosSDR/wiki
313 stars 48 forks source link

Several issues with the Python and dependencies of the GRC #86

Closed snake-4 closed 3 years ago

snake-4 commented 3 years ago

First issue is that the GRC not starting if the Python is installed for the current user instead of all users which results in the registry key for the Python being in the HKCU instead. Second issue is that if I install the Python for all users and run the GRC for the first time without admin privileges, the GRC helper script gives a warning saying that it cannot install the packages to the site-packages directory because it's not ran as admin and installs them in the user site-packages directory instead. Subsequent runs of the GRC result in the helper script saying No module named 'gi' and it tries to install a GTK package just to have pip say that it's already installed. The only solution to this seems to be removing the packages that the helper script installed and running the GRC as admin afterwards. My Python version is 3.9.1 64bit.

guruofquality commented 3 years ago

@SnakePin thanks for the feedback, I think I can probably make these other use cases work

First issue is that the GRC not starting if the Python is installed for the current user instead of all users which results in the registry key for the Python being in the HKCU instead.

For the first issue I think I can support finding a python thats not in the registry.

So it looks like a "local" user only python install is in %LOCALAPPDATA%\Programs\Python39\python.exe

Is that where your python executable was located as well?

Second issue is that if I install the Python for all users and run the GRC for the first time without admin privileges, the GRC helper script gives a warning saying that it cannot install the packages to the site-packages directory because it's not ran as admin and installs them in the user site-packages directory instead. Subsequent runs of the GRC result in the helper script saying No module named 'gi' and it tries to install a GTK package just to have pip say that it's already installed.

For this second issue its probably a problem with the gtk wheel install, but I just tried a "local" install and it seems to have worked for me, can you share more info so I can figure out what went wrong?

So for a simple test, the wheel for gtk can be installed with %LOCALAPPDATA%\Programs\Python39\Scripts\pip.exe install https://downloads.myriadrf.org/binaries/python39_amd64/PothosSDRPyGTK-2021.1.21-cp39-cp39-win_amd64.whl

Typically you should see the module go to %LOCALAPPDATA%\Programs\Python39\Lib\site-packages\gi and the runtime stuff for gtk in %LOCALAPPDATA%\Programs\Python39\Lib\gtk

And your search path should include that site-packages: `%LOCALAPPDATA%\Programs\Python39\python.exe -c "import sys; print(sys.path))"

Can you share if any of those details are different or what error you see when importing on the command line?

And what errors happen when you try to import gi?

    import gi
    gi.require_version('Gtk', '3.0')
    gi.require_version('PangoCairo', '1.0')
    gi.require_foreign('cairo', 'Context')

    from gi.repository import Gtk
    Gtk.init_check()
snake-4 commented 3 years ago

So it looks like a "local" user only python install is in %LOCALAPPDATA%\Programs\Python39\python.exe Is that where your python executable was located as well?

The python binary is located in %LOCALAPPDATA%\Programs\Python\Python39\python.exe to be exact.

The location of the site-packages folder for a local user installation is %LOCALAPPDATA%\Programs\Python\Python39\Lib\site-packages. If I install the PothosSDRPyGTK package here, I can import gi just fine but the GRC won't run.

I'm testing with a system-wide installation because local user python installations result in the GRC throwing an error.

On a system-wide python installation, I can see two site-package locations, the first one being %APPDATA%\Python\Python39\site-packages and the second one %PROGRAMFILES%\Python39\lib\site-packages.

When the helper script is ran as an admin, it install the packages in the second location, if not then they are installed in the first location. Normally this shouldn't be an issue but if the PothosSDRPyGTK package is installed in the first location, I get an error saying ModuleNotFoundError: No module named 'gi' when I try to import the gi module. The GRC helper script also gets the same error and it just tries to install the module again but as the module is already installed, it just does nothing.

To confirm what I've said above, I've tried installing the PothosSDRPyGTK package in the second location and every other package in the first location and I was able to import gi then.

So to sum it up, with a system-wide python installation, if the GRC helper is not ran as admin for the first-time, it's not possible to run GRC again until you manually uninstall the PothosSDRPyGTK package and run the GRC helper as admin or install the module with an admin command prompt.

Test System-wide Python Local user Python
import gi :warning: (only works if installed as admin) :heavy_check_mark:
GRC :heavy_check_mark: :x: (registry key does not exist)
guruofquality commented 3 years ago

So for the failure case of system-wide python installation with no admin privilege, gi goes here: %APPDATA%\Python\Python39\site-packages but where did the gtk runtime directory go? Do you see a %APPDATA%\Python\Python39\Lib\gtk\ or something like that?

guruofquality commented 3 years ago

On second though, that path seems a little suspicious. Are other system wide non admin packages installed to %APPDATA%\Python\Python39\site-packages or is it more like %APPDATA%\Python\Python39\Lib\site-packages?

snake-4 commented 3 years ago

Normal system wide non admin packages install to %APPDATA%\Python\Python39\site-packages GTK runtime installs to %APPDATA%\Python\Lib\gtk and %APPDATA%\Python\Lib\site-packages\gi I think the folder where the gi and the GTK runtime gets installed to is not the actual non admin Python location.

snake-4 commented 3 years ago

Additional note for non-admin on system wide installation: If I move wrong location's contents (%APPDATA%\Python\Lib) to the real location (%APPDATA%\Python\Python39) the import gi works. I don't know how exactly it works but does the PothosSDRPyGTK wheel hardcode these paths somehow?

guruofquality commented 3 years ago

I don't know how exactly it works but does the PothosSDRPyGTK wheel hardcode these paths somehow?

Its not hardcoded, its just a relative path thing

But the fact that you are seeing %APPDATA%\Python\Lib\site-packages instead of %APPDATA%\Python\Python39\site-packages somehow my setup.py isnt right https://github.com/pothosware/PothosSDR/blob/master/pygtk/setup.py

This is just a guess, but I think what happens is, this setup.py isnt registering any python modules just data files. So for this specific type of install, data files have a different structure so its %APPDATA%\Python instead of %APPDATA%\Python\Python39. I think if I fix this, the search for gtk runtime would change relative path just slightly as well.

Anyway, like I said, thats just a guess, there might also be some other setting on the wheel file. I need to do more reading to figure out how this one happens.

guruofquality commented 3 years ago

Suppose both pythons were detected? what do you think the preference should be? Local over global? And for the modules, I can force pip to install to APPDATA with the --user flag, so there is another preference to always use local user install when possible.

snake-4 commented 3 years ago

First of all, thanks for the fix, it seems to be working fine. I'd say that the global over local whenever possible is a better choice for both modules and the used Python version in my opinion.