henon / Python.Included

A Python.NET based framework enabling .NET libraries to call into Python packages without depending on a local Python installation.
MIT License
321 stars 52 forks source link

InstallPip hangs #44

Closed AdamTheLawyer closed 1 year ago

AdamTheLawyer commented 1 year ago

My install hangs whenever I run InstallPip().

It downloads get-pip.py just fine--I can see it invoking curl in the installer log--but then it just sits forever.

If I go to a console and run get-pip.py manually, though, it seems to work ... the code answers true to IsPipInstalled().

If I try to install a module (Installer.PipInstallModule("eyecite")), the log shows it downloading and installing, and the packages appear in the right spot in the embedded Python install, but the same thing happens ... the code just hangs.

Any insight on what it could be waiting for?

henon commented 1 year ago

can you run it from source and single step to see what happens and where?

nlogozzo commented 1 year ago

I'm experiencing the same issue currently...

henon commented 1 year ago

Would you be willing to debug it from the source code so we can get to the bottom of it?

AdamTheLawyer commented 1 year ago

Absolutely. What exactly do you need me to do?

henon commented 1 year ago

Download the code and run your use-case in visual studio. Debug step by step until the hang occurs. Then try to find out if there is anything that can be changed so that pip doesn't hang (if it does so, what hangs exactly needs to be confirmed).

jonahnm commented 1 year ago

I'm having the same issue!

henon commented 1 year ago

@AdamTheLawyer have you tried to debug it, any news?

jonahnm commented 1 year ago

I managed to fix this by creating my own cmd window, and installing the module with pip in the directory (in code), it’s likely an issue with the way you’re running commands, the run command function might think pip install never exits.

henon commented 1 year ago

@jonahnm thanks for the info, maybe it is easy to fix. Could you compare your way of running pip with the code that runs it in the library?

jonahnm commented 1 year ago

Ah, so I was browsing stack overflow and I found something that may help https://stackoverflow.com/questions/139593/processstartinfo-hanging-on-waitforexit-why

jonahnm commented 1 year ago

2F140D5C-4735-4BF6-99D0-F3057E9C5756

Edit by henon: Again in text form so others will find this via google

from msdn: The code example avoids a deadlock condition by calling p.StandardOutput.ReadToEnd before p.WaitForExit. A deadlock condition can result if the parent process calls p.WaitForExit before p.StandardOutput.ReadToEnd and the child process writes enough text to fill the redirected stream. The parent process would wait indefinitely for the child process to exit. The child process would wait indefinitely for the parent to read from the full StandardOutput stream.

jonahnm commented 1 year ago

E4CBBB56-AEF0-47CC-88DF-19C3EFD6DDC0 You do not appear to be doing this before waiting for it to exit.

henon commented 1 year ago

Thanks, I updated the nuget. Everybody having this issue please test and let me know if that fixes it: https://www.nuget.org/packages/Python.Included/3.11.2 https://www.nuget.org/packages/Python.Deployment/2.0.1

Thanks

BeckerJason commented 1 year ago

Im having this issue as well:

        Installer.SetupPython();
        Installer.TryInstallPip();
        Installer.PipInstallModule("numpy");
        Installer.PipInstallModule("mediapipe");
        Installer.PipInstallModule("opencv-python");
        Installer.PipInstallModule("socket");

The only line that completes is numpy. Each of the other lines will freeze.

henon commented 1 year ago

@BeckerJason Are you using the the latest python included / deployment nuget?

Sadly nobody in this thread ever confirmed that the issue is gone after the fix. @nlogozzo @AdamTheLawyer @jonahnm Was your issue fixed by the update?

BeckerJason commented 1 year ago

@BeckerJason Are you using the the latest python included / deployment nuget?

Sadly nobody in this thread ever confirmed that the issue is gone after the fix. @nlogozzo @AdamTheLawyer @jonahnm Was your issue fixed by the update?

image

nlogozzo commented 1 year ago

Sadly nobody in this thread ever confirmed that the issue is gone after the fix. @nlogozzo @AdamTheLawyer @jonahnm Was your issue fixed by the update?

I haven't tested yet...i've since switched do bundling the whl file with my application and installing that through this library and that hasn't given me any issues...

BeckerJason commented 1 year ago

@henon Any ideas on the cause for this? Am I using the commands in an unintended way?

henon commented 1 year ago

It is probably due to the redirection of stdout from pip to c#, at least that is a known issue that can cause hangs, see https://stackoverflow.com/questions/139593/processstartinfo-hanging-on-waitforexit-why

I changed the code as suggested in that issue and was waiting for the posters in this thread to confirm the fix, didn't try myself so I am not sure that it worked. You could do some tests and maybe comment the stdout redirection out completely and see if it will still hang in process.WaitForExit().

PS I would do it myself, eventually, once I get some free time onto my hands. If you have time and attempt to fix it you might get the fix faster.

BeckerJason commented 1 year ago

@henon Running your latest code included in my project seems to prevent the hang. running the Nuget causes hang

henon commented 1 year ago

That is great news, I'll update the nuget again, maybe I did something wrong there

henon commented 1 year ago

Published again after doing a clean build. Please try again with https://www.nuget.org/packages/Python.Included/3.11.3 https://www.nuget.org/packages/Python.Deployment/2.0.2

BeckerJason commented 1 year ago

The new nuget seems to have fixed the freezing

BeckerJason commented 1 year ago

Ran some tests. All is working well. Thank you very much @henon

henon commented 1 year ago

Note, this change still caused hangs as @BeckerJason found out and reported in #55. Should be fixed by #56 though