google / or-tools

Google's Operations Research tools:
https://developers.google.com/optimization/
Apache License 2.0
11.25k stars 2.13k forks source link

Windows net452 framework is broken #1862

Closed GuyBenhaim closed 4 years ago

GuyBenhaim commented 4 years ago

Details: Version: 7.5 from NuGet, Windows 10, C#, Routing Solver,

I built the project and when calling it (via URL) the code reached the part that initializes the IndexManager, with valid parameters, and gets an immediate exception:

RoutingIndexManager manager = new RoutingIndexManager(number_of_locations, number_of_vehicles, vehicle_starts_, vehicle_ends_);
**<InnerException>**
<Message>An error has occurred.</Message>
<ExceptionMessage>
 'SWIGExceptionHelper'.
</ExceptionMessage>
<ExceptionType>System.TypeInitializationException</ExceptionType>
<StackTrace>
ב- Google.OrTools.ConstraintSolver.operations_research_constraint_solverPINVOKE.SWIGExceptionHelper..ctor() ב- Google.OrTools.ConstraintSolver.operations_research_constraint_solverPINVOKE..cctor()
</StackTrace>
<InnerException>
<Message>An error has occurred.</Message>
<ExceptionMessage>
Cannot load DLL 'google-ortools-native'. Access is denied. (HRESULT: 0x80070005 (E_ACCESSDENIED))
</ExceptionMessage>
<ExceptionType>System.DllNotFoundException</ExceptionType>
<StackTrace>
Google.OrTools.ConstraintSolver.operations_research_constraint_solverPINVOKE.SWIGExceptionHelper.SWIGRegisterExceptionCallbacks_operations_research_constraint_solver(ExceptionDelegate applicationDelegate, ExceptionDelegate arithmeticDelegate, ExceptionDelegate divideByZeroDelegate, ExceptionDelegate indexOutOfRangeDelegate, ExceptionDelegate invalidCastDelegate, ExceptionDelegate invalidOperationDelegate, ExceptionDelegate ioDelegate, ExceptionDelegate nullReferenceDelegate, ExceptionDelegate outOfMemoryDelegate, ExceptionDelegate overflowDelegate, ExceptionDelegate systemExceptionDelegate) ב- Google.OrTools.ConstraintSolver.operations_research_constraint_solverPINVOKE.SWIGExceptionHelper..cctor()
</StackTrace>
</InnerException>
</InnerException>
</Error>
ptr1120 commented 4 years ago

Did you restore nuget packages and run clean build?

GuyBenhaim commented 4 years ago

Solved that one with some package update. Now runs! Returns a solution!

GuyBenhaim commented 4 years ago

Can I just copy my project files into that one?

ptr1120 commented 4 years ago

Do you have installed visual studio or rider in a current version (e.g. 2019). Do you have installed everything needed for netframework and asp.net?

ptr1120 commented 4 years ago

Yes, you can copy your files into that project. Please write wheather everything works after that.

GuyBenhaim commented 4 years ago

Thx. May take some time; complex project. Maybe I will look into your csproj and find something that helps ....

ptr1120 commented 4 years ago

Thanks. So summary, OrTools work as expected on Netstandard, NetCore, Netframework. Nothing broken in "Windows net452 framework" or OrTools.

The problems seem to be caused by not using the default mechanisms given by the IDE to create a project and install a Nuget package. Especially mixing things netframework/netcore, msbuild/dotnet-cli, old csproj Project format and new csproj Project format will lead to unexpected results.

GuyBenhaim commented 4 years ago

Do you believe that after installing with nuget , there could be some property in your latest csproj that would fix this?

E.g. adding lines 248-254 ?

ptr1120 commented 4 years ago

You should not edit the csproj file manually, especially not in a netframework project with old csproj Format, except you exactly know what you do. Use the "Manage nuget package" for (un-)installing, updating a package. Ensure that you use the old csproj format and packages.config file (which are created automatically by the IDE when you create a netframework project).

GuyBenhaim commented 4 years ago

I always used Manage Nuget Packages to install or-tools, but that ended-up with the exception. Is there another way to use it? Other-wise, its either 'playing' with the existing csproj, or starting with your project and re-building my project from there ...

m0ddixx commented 4 years ago

Interesting. Using the default mechanisms, OrTools seems to work also with a netframework Web Project. What I did:

* Create a Asp.Net WebApplication project using VisualStudio (or Rider)

* Use netframework 4.8 as target framework

* Select MVC or WebApi as project type

* Install newest OrTools using "Manage Nuget packages"

* Set the platform target of the project to `x64`

* Add the following code to an MVC controller:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Web;
using System.Web.Mvc;
using Google.OrTools.ConstraintSolver;

namespace WebApplication1.Controllers
{
    internal class DataModel
    {
        public long[,] DistanceMatrix = {
            {0, 548, 776, 696, 582, 274, 502, 194, 308, 194, 536, 502, 388, 354, 468, 776, 662},
            {548, 0, 684, 308, 194, 502, 730, 354, 696, 742, 1084, 594, 480, 674, 1016, 868, 1210},
            {776, 684, 0, 992, 878, 502, 274, 810, 468, 742, 400, 1278, 1164, 1130, 788, 1552, 754},
            {696, 308, 992, 0, 114, 650, 878, 502, 844, 890, 1232, 514, 628, 822, 1164, 560, 1358},
            {582, 194, 878, 114, 0, 536, 764, 388, 730, 776, 1118, 400, 514, 708, 1050, 674, 1244},
            {274, 502, 502, 650, 536, 0, 228, 308, 194, 240, 582, 776, 662, 628, 514, 1050, 708},
            {502, 730, 274, 878, 764, 228, 0, 536, 194, 468, 354, 1004, 890, 856, 514, 1278, 480},
            {194, 354, 810, 502, 388, 308, 536, 0, 342, 388, 730, 468, 354, 320, 662, 742, 856},
            {308, 696, 468, 844, 730, 194, 194, 342, 0, 274, 388, 810, 696, 662, 320, 1084, 514},
            {194, 742, 742, 890, 776, 240, 468, 388, 274, 0, 342, 536, 422, 388, 274, 810, 468},
            {536, 1084, 400, 1232, 1118, 582, 354, 730, 388, 342, 0, 878, 764, 730, 388, 1152, 354},
            {502, 594, 1278, 514, 400, 776, 1004, 468, 810, 536, 878, 0, 114, 308, 650, 274, 844},
            {388, 480, 1164, 628, 514, 662, 890, 354, 696, 422, 764, 114, 0, 194, 536, 388, 730},
            {354, 674, 1130, 822, 708, 628, 856, 320, 662, 388, 730, 308, 194, 0, 342, 422, 536},
            {468, 1016, 788, 1164, 1050, 514, 514, 662, 320, 274, 388, 650, 536, 342, 0, 764, 194},
            {776, 868, 1552, 560, 674, 1050, 1278, 742, 1084, 810, 1152, 274, 388, 422, 764, 0, 798},
            {662, 1210, 754, 1358, 1244, 708, 480, 856, 514, 468, 354, 844, 730, 536, 194, 798, 0}
        };

        public int VehicleNumber = 4;

        // [START starts_ends]
        public int[] Starts = { 1, 2, 15, 16 };

        public int[] Ends = { 0, 0, 0, 0 };
        // [END starts_ends]
    };

    public class HomeController : Controller
    {
        public ActionResult Index()
        {
            ViewBag.Title = "Home Page";
            ViewBag.Solution = Execute();

            return View();
        }

        private static string Execute()
        {
            // Instantiate the data problem.
            // [START data]
            DataModel data = new DataModel();
            // [END data]

            // Create Routing Index Manager
            // [START index_manager]
            RoutingIndexManager manager = new RoutingIndexManager(
                data.DistanceMatrix.GetLength(0),
                data.VehicleNumber,
                data.Starts,
                data.Ends);
            // [END index_manager]

            // Create Routing Model.
            // [START routing_model]
            RoutingModel routing = new RoutingModel(manager);
            // [END routing_model]

            // Create and register a transit callback.
            // [START transit_callback]
            int transitCallbackIndex = routing.RegisterTransitCallback(
              (long fromIndex, long toIndex) =>
              {
                  // Convert from routing variable Index to distance matrix NodeIndex.
                  var fromNode = manager.IndexToNode(fromIndex);
                  var toNode = manager.IndexToNode(toIndex);
                  return data.DistanceMatrix[fromNode, toNode];
              }
            );
            // [END transit_callback]

            // Define cost of each arc.
            // [START arc_cost]
            routing.SetArcCostEvaluatorOfAllVehicles(transitCallbackIndex);
            // [END arc_cost]

            // Add Distance constraint.
            // [START distance_constraint]
            routing.AddDimension(transitCallbackIndex, 0, 2000,
                                 true,  // start cumul to zero
                                 "Distance");
            RoutingDimension distanceDimension = routing.GetMutableDimension("Distance");
            distanceDimension.SetGlobalSpanCostCoefficient(100);
            // [END distance_constraint]

            // Setting first solution heuristic.
            // [START parameters]
            RoutingSearchParameters searchParameters =
              operations_research_constraint_solver.DefaultRoutingSearchParameters();
            searchParameters.FirstSolutionStrategy =
              FirstSolutionStrategy.Types.Value.PathCheapestArc;
            // [END parameters]

            // Solve the problem.
            // [START solve]
            Assignment solution = routing.SolveWithParameters(searchParameters);
            // [END solve]

            // Print solution on console.
            // [START print_solution]
            return PrintSolution(data, routing, manager, solution);
            // [END print_solution]
        }

        private static string PrintSolution(
            in DataModel data,
            in RoutingModel routing,
            in RoutingIndexManager manager,
            in Assignment solution)
        {
            var sb = new StringBuilder();
            // Inspect solution.
            long maxRouteDistance = 0;
            for (int i = 0; i < data.VehicleNumber; ++i)
            {
                sb.AppendLine($"Route for Vehicle {i}:");
                Console.WriteLine();
                long routeDistance = 0;
                var index = routing.Start(i);
                while (routing.IsEnd(index) == false)
                {
                    sb.Append($"{manager.IndexToNode((int)index)} -> ");
                    var previousIndex = index;
                    index = solution.Value(routing.NextVar(index));
                    routeDistance += routing.GetArcCostForVehicle(previousIndex, index, 0);
                }
                sb.AppendLine($"{manager.IndexToNode((int)index)}");
                sb.AppendLine($"Distance of the route: {routeDistance}m");
                maxRouteDistance = Math.Max(routeDistance, maxRouteDistance);
            }
            sb.AppendLine($"Maximum distance of the routes: {maxRouteDistance}m");
            return sb.ToString();
        }
    }
}

Result: Capture

I even did not add the <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies> setting to csproj. The google-ortools-native.dll is added to the output (bin/) directory during build automatically.

I add the project here, please can anybody confirm that this is working on your side. WebApplication1.zip

As a next step, after we have confirmed that default netcore/netframework (also Web) projects work with OrTools 7.5, we can check which of your specific project configuration is breaking the things.

I cannot confirm this working on .Net-Framework 4.6, which is the current runtime we use for production. If the framework version is the disqualifier I need to tell our admins to update asap. Thx for helping

m0ddixx commented 4 years ago

Update: I tried the same with .Net 4.8 still no success.. Either my project nor the one attached by @ptr1120

ptr1120 commented 4 years ago

@m0ddixx : did you try the minimal example web project given at https://github.com/google/or-tools/issues/1862#issuecomment-581012334?

m0ddixx commented 4 years ago

@ptr1120 that was the one I was referring to, yes.. Unfortunately the same exception

GuyBenhaim commented 4 years ago

Enigma

Mizux commented 4 years ago

My 2 cents,

0x80070005
E_ACCESS_DENIED Access denied by DCOM security. The user does not have remote access to the computer through DCOM. Typically, DCOM errors occur when connecting to a remote computer with a different operating system version. Give the user Remote Launch and Remote Activation permissions in dcomcnfg. Right-click My Computer-> Properties Under COM Security, click "Edit Limits" for both sections. Give the user you want remote access, remote launch, and remote activation. Then go to DCOM Config, find "Windows Management Instrumentation", and give the user you want Remote Launch and Remote Activation. For more information, see Connecting Between Different Operating Systems

src: https://docs.microsoft.com/en-us/windows/win32/wmisdk/wmi-troubleshooting?redirectedfrom=MSDN#access-denied

ptr1120 commented 4 years ago

@m0ddixx following questions:

  1. which exception do you experience, this one:
    Unhandled Exception: System.TypeInitializationException: The type initializer for 'Google.OrTools.ConstraintSolver.operations_research_constraint_solverPINVOKE' threw an exception. ---> System.TypeInitializationException: The type initializer for 'SWIGExceptionHelper' threw an exception. ---> System.DllNotFoundException: Unable to l
    oad DLL 'google-ortools-native': The specified module could not be found.
  2. same issue with the non-web project added here https://github.com/google/or-tools/issues/1862#issuecomment-580615929 ?
  3. same issue with a dotnet core project?
  4. same issue with another computer?
m0ddixx commented 4 years ago

Interesting @Mizux , but do you really think the error is DCOM related, since IIS won't propagate the DLLs over the network. But digging further into that error code i found out a missing visual studio c redistributable is also causing that error code (according to stackoverflow). I can confirm on my machine there is no visual c dependency or visual studio workload. I need to get it installed by an admin so the best I can do is report my findings by tomorrow

m0ddixx commented 4 years ago

Also the E_ACCESS_DENIED Flag has HRESULT: 0x80070005 while the DllNotFoundException (mentioned here) has HRESULT: 0x8007007. So there were basically two different causes.

m0ddixx commented 4 years ago

According to the docs you build the binaries using .Net Core SDK 2.1.302 at least and the ms docs say you need Microsoft Visual C++ 2015 Redistributable Update 3 to run the sdk. And to run the compiled binaries you need the exact same runtime. Coincidentally this package is missing on my machine so I will try to get it installed by tomorrow.

lperron commented 4 years ago

The c++ library is built with vs2019

Le mer. 5 févr. 2020 à 17:53, Nico Kranz notifications@github.com a écrit :

According to the docs https://github.com/google/or-tools/tree/stable/ortools/dotnet#pre-requisites you build the binaries using .Net Core SDK 2.1.302 at least and the ms docs say you need Microsoft Visual C++ 2015 Redistributable Update 3 https://docs.microsoft.com/en-us/dotnet/core/install/dependencies?tabs=netcore21&pivots=os-windows to run the sdk. And to run the compiled binaries you need the exact same runtime. Coincidentally this package is missing on my machine so I will try to get it installed by tomorrow.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/google/or-tools/issues/1862?email_source=notifications&email_token=ACUPL3IJ4XLFZJP35ROLVVTRBLVH3A5CNFSM4KNIJV52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK4EYMI#issuecomment-582503473, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUPL3JGPYGLKWGN6JUHMITRBLVH3ANCNFSM4KNIJV5Q .

m0ddixx commented 4 years ago

@GuyBenhaim is there any difference if you switch Enable SSL to true in the project? I assume you're still running it in IIS Express in a development environment.

GuyBenhaim commented 4 years ago

Running in Local IIS (i.e. not in express). Where exactly should I set SSL to Enabled?

m0ddixx commented 4 years ago

Running in Local IIS (i.e. not in express). Where exactly should I set SSL to Enabled?

Highlight the project in the solution explorer and below the solution explorer there is a property window. There you will find the SSL setting.

m0ddixx commented 4 years ago

BTW: installing the latest Visual C++ redistributable for 2019 fixes the HRESULT: 0x8007007 error. I think you should update your docs because in these docs the requirement between visual studio 2017 or 2019 is vaguely described. Having just vs2017 installed only has the 2017 redistributable shipped. I also wonder why the nuget package is not mentioned in the docs because they seem more convenient to me than including the bare binary packages. Hope that helped some facing issues.

GuyBenhaim commented 4 years ago

Nico Kranz, I've build your project (WebApplication1.zip) with Local IIS, and it crashes with the same HRESULT exception when calling the index manager.

I then built my project with IIS Express and it does not case this exception (still not fully running but for other reasons).

So the issue seems to be with the WebServer ....

GuyBenhaim commented 4 years ago

Next, I rebuilt a WebAPI VS project from scratch (VS2017, ORT 7.4 .NetF 4.7.2 ) and only then added my code and packages. When building/running the Web project via Local-IIS I got exceptions immediately when the main web-page opens in IIS , before even accessing or-tools : FileLoadException: HRESULT: 0x80131045) ConfigurationErrorsException: HttpException (0x80004005)

Correction: This refers to 7.4 as installation of 7.5 failed and NuGet removed the packages and reported failure.

GuyBenhaim commented 4 years ago

Also built/ran with IIS express, same: Could not load file or assembly 'Google.OrTools' or one of its dependencies. Strong name signature could not be verified. The assembly may have been tampered with, or it was delay signed but not fully signed with the correct private key. (Exception from HRESULT: 0x80131045)

Repeated with ORT 7.4 and .Net 4.8 > same.

Mizux commented 4 years ago

@GuyBenhaim concerning the strong name signature, AFAIK it has been fixed for the v7.5 IIRC see https://github.com/google/or-tools/issues/1421#issuecomment-555238081

GuyBenhaim commented 4 years ago

Thx @Mizux

Regrettably v7.5 refuses to install and I used v7.4. (I have downloaded the C++ re-dist) and I cannot check this fix for 7.5

See below the output when installing 7.5 which installs and then removes it (see in bold).

Attempting to gather dependency information for package 'Google.OrTools.7.5.7466' with respect to project 'AandRScratch', targeting '.NETFramework,Version=v4.8' Gathering dependency information took 4.45 sec Attempting to resolve dependencies for package 'Google.OrTools.7.5.7466' with DependencyBehavior 'Lowest' Resolving dependency information took 0 ms Resolving actions to install package 'Google.OrTools.7.5.7466' Resolved actions to install package 'Google.OrTools.7.5.7466' Retrieving package 'Google.OrTools 7.5.7466' from 'nuget.org'. Retrieving package 'Google.OrTools.runtime.linux-x64 7.5.7466' from 'nuget.org'. Retrieving package 'Google.OrTools.runtime.osx-x64 7.5.7466' from 'nuget.org'. Retrieving package 'Google.OrTools.runtime.win-x64 7.5.7466' from 'nuget.org'. Retrieving package 'Google.Protobuf 3.11.2' from 'nuget.org'. Retrieving package 'System.Buffers 4.4.0' from 'nuget.org'. Retrieving package 'System.Memory 4.5.2' from 'nuget.org'. Retrieving package 'System.Numerics.Vectors 4.4.0' from 'nuget.org'. Retrieving package 'System.Runtime.CompilerServices.Unsafe 4.5.2' from 'nuget.org'. Removed package 'Google.Protobuf.3.10.0' from 'packages.config' Successfully uninstalled 'Google.Protobuf.3.10.0' from AandRScratch Adding package 'Google.OrTools.runtime.linux-x64.7.5.7466' to folder 'C:\Users\guy\Google Drive\Routes\Projects\Proto\Allocation\AandRScratch\packages' Added package 'Google.OrTools.runtime.linux-x64.7.5.7466' to folder 'C:\AandRScratch\packages' Added package 'Google.OrTools.runtime.linux-x64.7.5.7466' to 'packages.config' Successfully installed 'Google.OrTools.runtime.linux-x64 7.5.7466' to AandRScratch Adding package 'Google.OrTools.runtime.osx-x64.7.5.7466' to folder 'C:\AandRScratch\packages' Added package 'Google.OrTools.runtime.osx-x64.7.5.7466' to folder 'C:\AandRScratch\packages' Overwriting existing file 'orLogo.png'.. Added package 'Google.OrTools.runtime.osx-x64.7.5.7466' to 'packages.config' Successfully installed 'Google.OrTools.runtime.osx-x64 7.5.7466' to AandRScratch Adding package 'Google.OrTools.runtime.win-x64.7.5.7466' to folder 'C:\AandRScratch\packages' Added package 'Google.OrTools.runtime.win-x64.7.5.7466' to folder 'C:\AandRScratch\packages' Install failed. Rolling back... Package 'Google.OrTools.runtime.win-x64.7.5.7466' does not exist in project 'AandRScratch' Removed package 'Google.OrTools.runtime.osx-x64.7.5.7466' from 'packages.config' Removed package 'Google.OrTools.runtime.linux-x64.7.5.7466' from 'packages.config' Package 'Google.Protobuf.3.10.0' already exists in folder 'C:\AandRScratch\packages' Added package 'Google.Protobuf.3.10.0' to 'packages.config' Removing package 'Google.OrTools.runtime.win-x64.7.5.7466' from folder 'C:\AandRScratch\packages' Removed package 'Google.OrTools.runtime.win-x64.7.5.7466' from folder 'C:\AandRScratch\packages' Removing package 'Google.OrTools.runtime.osx-x64.7.5.7466' from folder 'C:\AandRScratch\packages' Removed package 'Google.OrTools.runtime.osx-x64.7.5.7466' from folder 'C:\AandRScratch\packages' Removing package 'Google.OrTools.runtime.linux-x64.7.5.7466' from folder 'C:\AandRScratch\packages' Removed package 'Google.OrTools.runtime.linux-x64.7.5.7466' from folder 'C:\AandRScratch\packages' Executing nuget actions took 38.03 sec Failed to add reference to 'google-ortools-native'. Please make sure that the file is accessible, and that it is a valid assembly or COM component. Time Elapsed: 00:00:43.6456412

GuyBenhaim commented 4 years ago

I found some time for this, resolved the v7.5 install (cleaned/deleted .nuget cache) and built the Web project from scratch. Running/Debug on IIS Express works!

But I'm still getting "cannot load DLL" exception when running/debug on Local IIS immediately when the code accesses RoutingIndexManager manager = new RoutingIndexManager(...); Access is denied (exception from HRESULT: 0x80070005 (E_ACCESSDENIED)) I copied the native.dll (old tip by MIZUX) to /bin and to /lib, but still getting the exception.

Any tips?

GuyBenhaim commented 4 years ago

Ok, just a Windows-Security access permission issue.

moussabderrahim commented 4 years ago

Hi, I get the same problem on Windows 10, and i fixed it by the following 👍

  1. Update the windows 10 version to the latest (1903)
  2. Install Visual Studio 2019 the last version (Instead of 2017)
  3. Install the latest supported Visual C++ (this last action was the most important)

Regards,

GuyBenhaim commented 4 years ago

Is this going to be resolved in 7.6 or a patch? Thx