SciSharp / Numpy.NET

C#/F# bindings for NumPy - a fundamental library for scientific computing, machine learning and AI
Other
688 stars 97 forks source link

'System.DllNotFoundException' in Python.Runtime.dll & 'System.IO.IOException' in System.Private.CoreLib.dll #58

Closed Xlient closed 2 years ago

Xlient commented 4 years ago

Hi, I have a UWP app that basically digitally scans an image, everytime I try to get the dtype it throws Exception thrown: 'System.DllNotFoundException' in Python.Runtime.dll & then 'System.IO.IOException' in System.Private.CoreLib.dll

I'm not sure what's going on here, I tried the code in the Console app I get a different exception when I reach this point, but it doesn't deadlock. Exception thrown: 'System.IO.FileNotFoundException' in mscorlib.dll Exception thrown: 'Microsoft.CSharp.RuntimeBinder.RuntimeBinderException' in Microsoft.CSharp.dll

Here is the method that is causing the problem

    private NDarray OrderPoints(Point[] points)
        {
            int[,] Pts = new int[,]
            {
              {points[0].X, points[0].Y },
              {points[1].X, points[1].Y },
              {points[2].X, points[2].Y } ,
              {points[3].X, points[3].Y }
            };

            // exception here / deadlock
               Dtype dtype = Pts.GetDtype();

               NDarray pts = np.array(Pts);
               NDarray rectangle = np.zeros(new Shape(4, 2), dtype); 

            NDarray sum = np.sum(pts, 1);
            NDarray differnce = np.diff(pts, axis: 1);

            rectangle[0] = pts[sum.argmin()];
            rectangle[2] = pts[sum.argmax()];
            rectangle[1] = pts[differnce.argmin()];
            rectangle[3] = pts[differnce.argmax()];

            return rectangle;
        }

I'm using NumPy.net 3.7.1.22. OpencvSharp4 & for uwp pythonnet_netstandard_py37_win

henon commented 4 years ago

I pulled your code into a unit test in the Numpy test suite and it runs through without errors.

        [TestMethod]
        public void IssueByXlient()
        {
            var points=new Point[]{ new Point(0,0), new Point(17, 4), new Point(2,22), new   Point(10, 7), };
            int[,] Pts = new int[,]
            {
                {points[0].X, points[0].Y },
                {points[1].X, points[1].Y },
                {points[2].X, points[2].Y } ,
                {points[3].X, points[3].Y }
            };

            // exception here / deadlock
            Dtype dtype = Pts.GetDtype();

            NDarray pts = np.array(Pts);
            NDarray rectangle = np.zeros(new Shape(4, 2), dtype);

            NDarray sum = np.sum(pts, 1);
            NDarray differnce = np.diff(pts, axis: 1);

            rectangle[0] = pts[sum.argmin()];
            rectangle[2] = pts[sum.argmax()];
            rectangle[1] = pts[differnce.argmin()];
            rectangle[3] = pts[differnce.argmax()];
        }

So I think you have a problem with the setup. Is your project set to compile for AnyCPU? Some projects by default set themselves to Prefer 32-bit which can cause problems, so make sure that is off.

henon commented 4 years ago

I have no experience with UWP sadly.

henon commented 4 years ago

with respect to deadlocks read this: https://github.com/SciSharp/Numpy.NET#multi-threading-must-read

henon commented 4 years ago

UWP has restricted file system access. I guess that is what causes your problems. https://daniel-krzyczkowski.github.io/UWP-External-file-system-access-and-environment-variables/

Numpy uses Python.Included which installs an embedded Python interpreter in your user folder. I guess that will be prohibited by UWP. You can check out Python.Included and check out the examples. They show how to install python and libraries (such as numpy) in custom locations (i.e. your App's directory). Then you can use Numpy.Bare instead with the python folder in the PATH environment variable so it will find it.

henon commented 4 years ago

I don't have UWP installed, but for you and all who have a similar problem I added an example how to install numpy in a custom location: https://github.com/SciSharp/Numpy.NET/tree/master/src/Examples/CustomInstallLocationExample

This is the gist of it, using Python.Deployment and Numpy.Bare and attaching python and numby packages as embedded resource:

class Program
    {
        static void Main(string[] args)
        {
            // ================================================
            // This example demonstrates how to install Python and Numpy from the assembly's resources
            // (build action 'Embedded resource') into a custom location (here the local execution directory ".")
            // and then use Numpy.Bare with that installation.
            // ================================================

            // set the installation source to be the embedded python zip from our resources
            Python.Deployment.Installer.Source = new Python.Deployment.Installer.EmbeddedResourceInstallationSource()
            {
                Assembly = typeof(Program).Assembly,
                ResourceName = "python-3.7.3-embed-amd64.zip",
            };

            // install in local directory. if you don't set it will install in local app data of your user account
            Python.Deployment.Installer.InstallPath = Path.GetFullPath(".");

            // see what the installer is doing
            Python.Deployment.Installer.LogMessage += Console.WriteLine;

            // install from the given source
            Python.Deployment.Installer.SetupPython(force: false).Wait();

            Python.Deployment.Installer.InstallWheel(typeof(Program).Assembly,
                "numpy-1.16.3-cp37-cp37m-win_amd64.whl").Wait();

            // if the installation is local, you don't even need to set the path
            //Environment.SetEnvironmentVariable("PATH", Path.GetFullPath(@"./python-3.7.3-embed-amd64"), EnvironmentVariableTarget.Process);

            // Now use Numpy.Bare
            var a = np.arange(10);
            Console.WriteLine("a: "+ a.repr);
            var b = np.arange(10)["::-1"];
            Console.WriteLine("b: " + b.repr);
            var a_x_b = np.matmul(a, b);
            Console.WriteLine("a x b: " + a_x_b.repr);
        }
    }
Xlient commented 4 years ago

Ok so i downloaded Numpy.Bare and Python. Included and implemented your example.

Python.Deployment.Installer.Source = new Python.Deployment.Installer.EmbeddedResourceInstallationSource()
                {
                    Assembly = typeof(Program).Assembly,

                    ResourceName = "python-3.7.3-embed-amd64.zip",
                };

                string localFolder = Windows.Storage.ApplicationData.Current.LocalFolder.Path;
                Python.Deployment.Installer.InstallPath = localFolder;

                Python.Deployment.Installer.SetupPython(force: false).Wait();

                Python.Deployment.Installer.InstallWheel(localFolder +"\\python-3.7.3-embed-amd64\\Lib\\Numpy.Resources.numpy-1.16.3-cp37-cp37m-win_amd64.whl").Wait();

However, I still get the same exceptions, when I try to use NumPy.

the Inner Exception mentions:

{"Unable to load DLL 'python37' or one of its dependencies: The specified module could not be found. (Exception from HRESULT: 0x8007007E)"}

I think its definitely UWP's restricted file access that causing a problem, I don't know a workaround, as this is really my first time working with UWP. so I may just move to WPF.

henon commented 4 years ago

have you checked that python is installed at the location? if so, now you only have to set the PATH environment variable so that it will be found (see my example above, commented out line cause not needed in my case but probably in yours).

Xlient commented 4 years ago

Yes i followed your code example , still the same results unfortunately