Value of Loss in ReportBestTrial (from class AutoMLMonitor) is always negative of Metric

[dcostea]

System Information (please complete the following information):

• OS & Version: Windows 11
• ML.NET Version: v3.0
• .NET Version: .NET 7.0

Describe the bug The Loss seems to be the negative of Metric.

To Reproduce Steps to reproduce the behavior:

1. Run a sample with AutoML
2. Add a monitoring class (IMonitor)
3. Check result.Loss in ReportBestTrial(TrialResult result)

Expected behavior A correct value for Loss value.

Screenshots, Code, Sample Projects public void ReportBestTrial(TrialResult result) { // check result.Loss below, the values is negative of result.Metric, e.g. if result.Metric is 0.728, Loss is -0.728 _bestTrials.Add(result); }

Additional context None

[michaelgsharp]

Can you share a simple sample that reproes this please?

[ghost]

This issue has been marked needs-author-action and may be missing some important information.

[dcostea]

@michaelgsharp Sorry Michael, I might have added more details, indeed.

You can find it in this repo: https://github.com/dcostea/AutoMLSampleWithMLNET20/tree/main (just to speed up the debugging, you can search for the line MachineLearningServices.cs:87 var experimentResult = await experiment.RunAsync(cts.Token); and inspect it)

The bug reproduces with:

version 2.0

<PackageReference Include="Microsoft.ML" Version="2.0.1" />
<PackageReference Include="Microsoft.ML.AutoML" Version="0.20.1" />
<PackageReference Include="Microsoft.ML.DataView" Version="2.0.1" />
<PackageReference Include="Microsoft.ML.Experimental" Version="0.20.1" />

version 2.0.1

<PackageReference Include="Microsoft.ML" Version="2.0.1" />
<PackageReference Include="Microsoft.ML.AutoML" Version="0.20.1" />
<PackageReference Include="Microsoft.ML.DataView" Version="2.0.1" />
<PackageReference Include="Microsoft.ML.Experimental" Version="0.20.1" />

latest 3.0.0-preview

<PackageReference Include="Microsoft.ML" Version="3.0.0-preview.23106.1" />
<PackageReference Include="Microsoft.ML.AutoML" Version="0.21.0-preview.23106.1" />
<PackageReference Include="Microsoft.ML.DataView" Version="3.0.0-preview.23106.1" />
<PackageReference Include="Microsoft.ML.Experimental" Version="0.21.0-preview.23106.1" />

Edit: I'm adding some data too (please unzip in C:\Temp or some other place, but you have to update the path in the solution): Temp.zip