search

heal-research / HeuristicLab

HeuristicLab - An environment for heuristic and evolutionary optimization
https://dev.heuristiclab.com
GNU General Public License v3.0
38 stars 16 forks source link

Core migration #3186

Open NimZwei opened 5 months ago

NimZwei commented 5 months ago

Known issues:

ToDos:

Shabbafru commented 5 months ago

Further Todos as discussed:

gkronber commented 5 months ago 
* Figure out if we can / should remove Application stuff and starter form

I would tend to say yes. The main reason for the starter is to be able to check plugin descriptions and dependencies before loading the Optimizer. Another reason was to provide the possibility of multiple apps. However, I would say we can just provide them as separate executable instead of via the common starter.

* Figure out if its ok to remove external evaluation problems for matlab and scilab

Ok, can later be re-added as a plugin (outside of HL main repo).

NimZwei commented 4 months ago

Topic for discusion: When extracting all UI parts from the PluginInfrastructure project into a separate project to have a UI-less PluginInfrastructure, the ErrorHandling class does not split well. It provides a ShowErrorDialog method, which obviously requires WinForms, so it goes into the new PluginInfrastructure.Views project. However, there are some cases, where non-UI projects show an error dialog, such as the VRP:

if (cities != ProblemInstance.Coordinates.Rows - 1)
  ErrorHandling.ShowErrorDialog(new Exception("The optimal solution does not seem to correspond with the problem data"));
else {
  VRPSolution solution = new VRPSolution(ProblemInstance, encoding, new DoubleValue(0));
  BestKnownSolutionParameter.Value = solution;
}

Also the SingleObjectiveTestFunctionProblem has similar behavior where there is a warning shown via the error dialog.

I would opt to remove any error dialog related code from non-ui project and throw an exception instead. This would be a breaking change because previously, after displaying the error dialog, the execution would just resume, whereas throwing an exception would stop execution and the exception would need to be handled somewhere else.

NimZwei commented 4 months ago

Another topic for discussion: We are currently thinking about removing the version from projects and assemblies and their corresponding version folders. For example, HeuristicLab.Core-3.3 would simply be named HeuristicLab.Core, and instead of being located in the directory HeuristicLab.Core/3.3/ it would be placed directly into HeuristicLab.Core.

A few things to discuss:

gkronber commented 4 months ago

What do we do with the different Alglib versions? Currently, older Alglib versions are only used in HeuristicLab.Algorithms.DataAnalysis for some obsolete classes for NearesNeighbor, NeuralNetwork and RandomForest which are only there backward-compatible persistence. Since the classes are all marked as obsolete, and the persistence might break anyway due to plugin/assembly merges, I would opt to remove these classes and the older Alglib versions.

Please use nuget to include latest alglib packages. Old versions of alglib can be removed. Two issues must be checked:

  1. First we made a change to alglib some time ago to prevent a race-condition when accessing the alglib-internal PRNG. I think this was fixed in a later version of alglib anyway
  2. We added a custom evaluation method for random forests we used for variable importance or the intersection plots (PDP), I believe.
NimZwei commented 4 months ago

Based on the comment of @gkronber, I have looked all all ExtLibs that are currently used via direct code inclusion and whether we can replace it by Nuget packages:

NimZwei commented 4 months ago

Adding to the ExtLib nugetd discussion: By including the ExtLib sources and compiling the assemblies ourselves, we know that the output after compilation is just a single dll, which we then include as plugin file so that it is properly handled for Hive. However, if we were using Nuget, we would have to look up which files each dependency actually requires, including any transitive dependencies. This will probably be quite hard to do and maintain, and will slow down dependency updates.

A few solutions that come to mind:

NimZwei commented 4 months ago

After discussion with @s-wagner, @Vanmodeus and @obindreiter we decided on the following points:

Shabbafru commented 4 months ago

Topic for discusion: When extracting all UI parts from the PluginInfrastructure project into a separate project to have a UI-less PluginInfrastructure, the ErrorHandling class does not split well. It provides a ShowErrorDialog method, which obviously requires WinForms, so it goes into the new PluginInfrastructure.Views project. However, there are some cases, where non-UI projects show an error dialog, such as the VRP: ...

I'm also not quite sure about this, but I think in general (and in all other places in HL its like this) an exception is thrown in the non-gui code and the gui catches the error and displays the dialog. So I would also go with exceptions.

But, if we really need an alternative, we could also do something like the following in PluginInfrastructure:

public static void ShowError(string message, Exception exception) {
  Type guiErrorHandling = Type.GetType("HeuristicLab.PluginInfrastructure.Views.ErrorHandling");

  if (guiErrorHandling == null) {    
     throw exception;
  } else {        
    guiErrorHandling.GetMethod("ShowErrorDialog", [typeof(string), typeof(Exception)]).Invoke(null, [message, exception]);
  }
}
Shabbafru commented 3 months ago

Please note that cbd3aec breaks reading/writing HL files (and therefore also the StartPage) until https://github.com/heal-research/HEAL.Attic/pull/53 is done

NimZwei commented 2 months ago

We have encountered a performance issue in certain cases when using .NET 8, which is resolved by upgrading to .NET 9. The issue arises due to a bug (?) in the .NET 8 runtime related to reflection on system types with self-referencing generic constraints.

Details:

When loading a Problem that implements IProblemInstanceConsumer<TData>, we instantiate all corresponding IProblemInstanceProvider<TData> to display them in the UI. This is done via the ApplicationManager.Manager.GetInstances(type) method, which iterates over all currently loaded types to find those that are instantiable and implement the specified type.

For non-generic types, this process is straightforward. However, for generic types, we rely on TypeExtensions.BuildType to handle the generic arguments. For example, when querying all instantiable types for IProblemInstanceConsumer<TData>, we must check if TSPData can be used as a valid generic argument for types with a single generic parameter.

Rather than calling type.MakeGenericType and catching potential exceptions, HeuristicLab proactively filters out types that would cause an exception. This is done to avoid the overhead of handling unnecessary exceptions when iterating through the types. For instance, if a generic parameter has a type constraint (where T : SomeType), HeuristicLab discards any types that don't fulfill these constraints.

In .NET 8, several generic interfaces with self-referencing generic argument constraints were introduced, such as public interface IAdditiveIdentity<TSelf, TResult> where TSelf : IAdditiveIdentity<TSelf, TResult>?. The problem is that calling Type.GetGenericParameterConstraints on the TSelf argument returns an empty list, effectively losing the constraint information. This behavior is peculiar because it works as expected on user-defined types (see example below).

Due to the missing self-referencing constraint, HeuristicLab fails to filter out incompatible types before attempting to build a generic type, leading to exceptions. Since .NET 8 introduced many types with self-referencing generic constraints, this results in a significant increase in exceptions.

I could not find any issues, pull requests, or Stack Overflow discussions addressing this behavior, but it appears to be resolved in .NET 9. Therefore, for HeuristicLab, we should either upgrade to .NET 9 as soon as possible or live with the performance regression for such cases.

using System;
using System.Linq;

var tselfCustomType = typeof(ISelfReferencing<>).GetGenericArguments()[0];
var tselfSystemType = typeof(IUtf8SpanParsable<>).GetGenericArguments()[0];

Console.WriteLine(tselfCustomType.GetGenericParameterConstraints().Count()); // 1
Console.WriteLine(tselfSystemType.GetGenericParameterConstraints().Count()); // 0

public interface ISelfReferencing<TSelf> where TSelf : ISelfReferencing<TSelf>? { }
NimZwei commented 1 month ago

It seems that the discussed issues that cause some exceptions for the PluginInfrastucture TypeExtensions (see https://github.com/heal-research/HeuristicLab/pull/3186#issuecomment-2316905999), stems from a runtime bug, not so much a .net 8/9 bug.

When using explicit runtime versions for a self-contained deployment (to be independent from the installed runtimes), it seems that the bug is present in version 8.0.8 and resolved in the runtime version 8.0.10.

This can be reproduced by the following code:

var tselfCustomType = typeof(ISelfReferencing<>).GetGenericArguments()[0];
var tselfSystemType = typeof(IUtf8SpanParsable<>).GetGenericArguments()[0];

Console.WriteLine(Environment.Version);
Console.WriteLine(tselfCustomType.GetGenericParameterConstraints().Count());
Console.WriteLine(tselfSystemType.GetGenericParameterConstraints().Count());

public interface ISelfReferencing<TSelf> where TSelf : ISelfReferencing<TSelf>? { }

global.json

{
  "sdk": {
    "version": "8.0.304", // or 8.0.403
    "rollForward": "disable"
  }
}

When publishing using two different global.json files to configure the runtime, we get to different outputs

dotnet publish '.\HeuristicLab\3.3\HeuristicLab-3.3.csproj' --outputo bin --self-contained --framework net8.0-windows`
sdk runtime output
8.0.304 8.0.8 
8.0.8 
1
0
8.0.403 8.0.10 
8.0.10
1
1