henon / Python.Included

A Python.NET based framework enabling .NET libraries to call into Python packages without depending on a local Python installation.
MIT License
313 stars 51 forks source link

File.ReadAllLines(pth).Contains("./Lib") throws exception #14

Closed katjoek closed 3 years ago

katjoek commented 3 years ago

When I'm doing InstallWheel for the first time, it always fails. This happens because in Python.Deployment.InstallWheel the final action is to add .\Lib to the python37._pth (in my case), which throws an exception as the file doesn't exists. There is no code that creates it. I only see _pth being mentioned there and in SetupPython() where it is deleted.

On the second run, the package is already installed and the execution won't reach the part where pytohn37._pth is read. So then there is no problem.

I "fixed" it first by creating the file if it didn't exist, just before appending .Lib to it. But this caused an error. Starting python then gives me an error that it cannot find the module 'encodings'.

Skipping the addition of .\Lib to python37._pth if that file doesn't exist seems to work and behave fine.

Is the code related to the _pth file really necessary?

henon commented 3 years ago

I am not sure, but it is added by some installation method in some scenario. Maybe the file is added by InstallPip?

henon commented 3 years ago

epignatelli added a reference to this issue right before the line where the _pth file is created: https://github.com/pypa/pip/issues/4207#issuecomment-281236055

@epignatelli, maybe you can explain what the ultimate purpose of the _pth file is?

katjoek commented 3 years ago

I am not sure, but it is added by some installation method in some scenario. Maybe the file is added by InstallPip?

It is not added when I run in my scenario. Obviously, I downloaded all mentioned wheel files and added them as embedded resources. Program.cs.txt

henon commented 3 years ago

I'll add your test case to our examples and take a look

katjoek commented 3 years ago

I saw I got mentioned somewhere but I don't know where (I'm kind of a github newby). You asked me to run Python\Deployment\EmbeddedResource. I did. It runs fine. That makes sense as my problem only occurs when you do a InstallWheel, which is not done in this application. When I look into the python installation directory, I do not see a python38._pth file.

henon commented 3 years ago

I just found out that the _pth file is deleted in SetupPython in order to support pip

                    // allow pip on embedded python installation
                    // see https://github.com/pypa/pip/issues/4207#issuecomment-281236055
                    var pth = Path.Combine(EmbeddedPythonHome, Source.GetPythonVersion() + "._pth");
                    File.Delete(pth);
katjoek commented 3 years ago

Indeed, that's the other of the two times you'll find _pth in the code

katjoek commented 3 years ago

It is part if the zip file. So the delete actually does delete the file.

henon commented 3 years ago

Yeah, Epignatelli removed that file in his fork because he didn't personally like the idea of installing wheels without using pip. I think the file should only be removed in InstallPip.

katjoek commented 3 years ago

That sounds good. But when I made InstallWheel create the file before adding .\Lib, the application didn't work anymore. I got an error saying that module 'encoding' could not be found. The application didn't work anymore.

Not having the _pth file makes my test application work... But then the addition of .\Lib to the _pth file should be removed or be made conditional, probably depending on whether the file exists or not.

henon commented 3 years ago

You fixed this. Thanks!