Problem with WindowsForms Application

[DrEricEbert]

I have written a .Net 4.7.2 Class Library and installed the Python.Included 3.7.3.13 Nugget with the following code:

using Python.Deployment;
using Python.Runtime;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace PythonRunnnerCL
{
    public class PythonRunnerClass
    {
        public static async Task Runner()
        {
            // This example demonstrates how Python.Included is able to automatically install a minimal Python
            // environment which it includes as an embedded resource in its .NET assembly file
            // Python.Included is currently fixed to Python 3.7.3 amd64 for windows
            // If you need a different Python version or platform check out the Python.Installer examples!

            // install in local directory
            Installer.InstallPath = Path.GetFullPath(".");

            // see what the installer is doing
            Installer.LogMessage += Console.WriteLine;

            // install the embedded python distribution
            Installer.SetupPython().Wait();

            // install pip3 for package installation
            Installer.TryInstallPip();

            // download and install Spacy from the internet
            Installer.PipInstallModule("spacy");
            Installer.PipInstallModule("numpy");
            Installer.PipInstallModule("python-dateutil");
            Installer.PipInstallModule("six");
            Installer.PipInstallModule("pytz");
            // ok, now use pythonnet from that installation
            PythonEngine.Initialize();

            // call Python's sys.version to prove we are executing the right version
            dynamic sys = PythonEngine.ImportModule("sys");
            Console.WriteLine("### Python version:\n\t" + sys.version);

            // call os.getcwd() to prove we are executing the locally installed embedded python distribution
            dynamic os = PythonEngine.ImportModule("os");
            Console.WriteLine("### Current working directory:\n\t" + os.getcwd());
            Console.WriteLine("### PythonPath:\n\t" + PythonEngine.PythonPath);

            // call spacy
            dynamic spacy = Py.Import("spacy");
            Console.WriteLine("### Spacy version:\n\t" + spacy.__version__);

            dynamic np = Py.Import("numpy");

            Console.WriteLine(np.cos(np.pi * 2));

            dynamic sin = np.sin;
            Console.WriteLine(sin(5));

            double c = (double)(np.cos(5) + sin(5));
            Console.WriteLine(c);

            dynamic a = np.array(new List<float> { 1, 2, 3 });
            Console.WriteLine(a.dtype);

            dynamic b = np.array(new List<float> { 6, 5, 4 }, dtype: np.int32);
            Console.WriteLine(b.dtype);

            Console.WriteLine(a * b);

            PythonEngine.Exec(@"
import sys
import math
import numpy as np

print ('Hello world!')
print ('version:' + sys.version)

np.arange(1) # check if numpy is properly loaded

a1 = np.arange(60000).reshape(300, 200)
a2 = np.arange(80000).reshape(200, 400)
result = np.matmul(a1, a2)

print('result: ' + str(result))

# do some dateutils stuff
from dateutil.relativedelta import *
from dateutil.easter import *
from dateutil.rrule import *
from dateutil.parser import *
from datetime import *
now = parse('Sat Oct 11 17:13:46 UTC 2003')
today = now.date()
year = rrule(YEARLY, dtstart = now, bymonth = 8, bymonthday = 13, byweekday = FR)[0].year
rdelta = relativedelta(easter(year), today)
print('Today is: %s (no, it isn\'t)' % today)
print('Year with next Aug 13th on a Friday is: %s' % year)
print('How far is the Easter of that year: %s' % rdelta)
print('And the Easter of that year is: %s' % (today + rdelta))

# do some pytz stuff
import pytz
print ('UTC time zone: %s' % pytz.utc);

from pytz import timezone
print ('Eastern time zone: %s' % timezone('US/Eastern'));
");

            Console.WriteLine("\nDone. Press any key to exit.");
#if NETFRAMEWORK
         //   Console.ReadKey();
#endif
        }
    }
}

The DLL works fine for a Console Application with this Code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace PythonRunnerCMD
{
    class Program
    {
        static void Main(string[] args)
        {
            PythonRunnnerCL.PythonRunnerClass.Runner().Wait();
        }
    }
}

But not for a WindowsForms Application with the following code:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;

namespace PythonRunnerUI
{
    public partial class PythonRunnerUIForm : Form
    {
        public PythonRunnerUIForm()
        {
            InitializeComponent();
        }

        private void beendenToolStripMenuItem_Click(object sender, EventArgs e)
        {
            this.Close();
        }

        private void runToolStripMenuItem_Click(object sender, EventArgs e)
        {
            PythonRunnnerCL.PythonRunnerClass.Runner().Wait();
        }
    }
}

Why actually? It uses the same .Net framework base... The code downloads the python-3.7.3-embed-amd64.zip as expected and stops in Installer.SetupPython().Wait(); after loading the embedded python distribution. Does anyone have an idea?

[henon]

First check the Numpy.NET readme what I wrote about avoiding deadlocks. https://github.com/SciSharp/Numpy.NET#multi-threading-must-read

If that doesn't help I think you need to download the source code and step into it with the debugger to find out why it never returns.

[DrEricEbert]

Ok i've tried out both suggestions.

PythonEngine.BeginAllowThreads()

and

using (Py.GIL())
  {
...
  }

Are working after the Interpreter is Running. I got

System.AggregateException
  HResult=0x80131500
  Nachricht = Mindestens ein Fehler ist aufgetreten.
  Quelle = mscorlib
  Stapelüberwachung:
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at PythonRunnerUI.PythonRunnerUIForm.runToolStripMenuItem_Click(Object sender, EventArgs e) in C:\Data\HyDesign\trunk\Code\PythonNetTester\PythonRunnerUI\PythonRunnerUIForm.cs:line 28
   at System.Windows.Forms.ToolStripItem.RaiseEvent(Object key, EventArgs e)
   at System.Windows.Forms.ToolStripMenuItem.OnClick(EventArgs e)
   at System.Windows.Forms.ToolStripItem.HandleClick(EventArgs e)
   at System.Windows.Forms.ToolStripItem.HandleMouseUp(MouseEventArgs e)
   at System.Windows.Forms.ToolStrip.OnMouseUp(MouseEventArgs mea)
   at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
   at System.Windows.Forms.Control.WndProc(Message& m)
   at System.Windows.Forms.ToolStrip.WndProc(Message& m)
   at System.Windows.Forms.MenuStrip.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
   at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg)
   at System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(IntPtr dwComponentID, Int32 reason, Int32 pvLoopData)
   at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context)
   at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context)
   at PythonRunnerUI.Program.Main() in C:\Data\HyDesign\trunk\Code\PythonNetTester\PythonRunnerUI\Program.cs:line 19

Innere Ausnahme 1:
DllNotFoundException: Die DLL "python37": Das angegebene Modul wurde nicht gefunden. (Ausnahme von HRESULT: 0x8007007E) kann nicht geladen werden.

on PythonEngine.BeginAllowThreads() and

System.AggregateException
  HResult=0x80131500
  Nachricht = Mindestens ein Fehler ist aufgetreten.
  Quelle = mscorlib
  Stapelüberwachung:
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at PythonRunnerUI.PythonRunnerUIForm.runToolStripMenuItem_Click(Object sender, EventArgs e) in C:\Data\HyDesign\trunk\Code\PythonNetTester\PythonRunnerUI\PythonRunnerUIForm.cs:line 28
   at System.Windows.Forms.ToolStripItem.RaiseEvent(Object key, EventArgs e)
   at System.Windows.Forms.ToolStripMenuItem.OnClick(EventArgs e)
   at System.Windows.Forms.ToolStripItem.HandleClick(EventArgs e)
   at System.Windows.Forms.ToolStripItem.HandleMouseUp(MouseEventArgs e)
   at System.Windows.Forms.ToolStrip.OnMouseUp(MouseEventArgs mea)
   at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
   at System.Windows.Forms.Control.WndProc(Message& m)
   at System.Windows.Forms.ToolStrip.WndProc(Message& m)
   at System.Windows.Forms.MenuStrip.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
   at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg)
   at System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(IntPtr dwComponentID, Int32 reason, Int32 pvLoopData)
   at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context)
   at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context)
   at PythonRunnerUI.Program.Main() in C:\Data\HyDesign\trunk\Code\PythonNetTester\PythonRunnerUI\Program.cs:line 19

Innere Ausnahme 1:
DllNotFoundException: Die DLL "python37": Das angegebene Modul wurde nicht gefunden. (Ausnahme von HRESULT: 0x8007007E) kann nicht geladen werden.

on the using Py.GIL() statement... Without it works in console application.

I've stepped into the Libs...

The Code stopped at:

  public static async Task SetupPython(bool force = false)
        {
            Environment.SetEnvironmentVariable("PATH", $"{EmbeddedPythonHome};" + Environment.GetEnvironmentVariable("PATH"));
            if (!force && Directory.Exists(EmbeddedPythonHome) && File.Exists(Path.Combine(EmbeddedPythonHome, "python.exe"))) // python seems installed, so exit
                return;
            var zip = await Source.RetrievePythonZip(InstallPath);

Could this be the reason? https://stackoverflow.com/questions/13140523/await-vs-task-wait-deadlock

[DrEricEbert]

This works:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;

namespace PythonRunnerUI
{
    public partial class PythonRunnerUIForm : Form
    {
        public PythonRunnerUIForm()
        {
            InitializeComponent();
        }

        private void beendenToolStripMenuItem_Click(object sender, EventArgs e)
        {
            this.Close();
        }

        private void runToolStripMenuItem_Click(object sender, EventArgs e)
        {
           Task.Run(()=> 
           {
               PythonRunnnerCL.PythonRunnerClass.Runner(); 
           });
        }
    }
}