search

dotnet / machinelearning

ML.NET is an open source and cross-platform machine learning framework for .NET.
https://dot.net/ml
MIT License
8.93k stars 1.86k forks source link

ArgumentOutOfRangeException when running Microsoft.ML.Fairlearn.Tests.GridSearchTest.TestGridSearchTrialRunner2 #6842

Open ericstj opened 9 months ago

ericstj commented 9 months ago

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=419700 Build error leg or test failing: Microsoft.ML.Fairlearn.Tests.WorkItemExecution Pull request: https://github.com/dotnet/machinelearning/pull/6837

Error Message

        [Source=AutoMLExperiment, Kind=Trace] Channel started
        [Source=AutoMLExperiment, Kind=Info] Update Running Trial - Id: 0
        [Source=AutoMLExperiment, Kind=Trace] Microsoft.ML.AutoML.DefaultPerformanceMonitor has been started
        [Source=AutoMLExperiment, Kind=Trace] current CPU: 0, current Memory(mb): 48.51171875
        [Source=AutoMLExperiment, Kind=Trace] maximum memory usage: , PeakMemoryUsage: 48.51171875 trialIsCancelled: False
        [Source=AutoMLExperiment, Kind=Trace] trial setting - {"TrialId":0,"StartedAtUtc":"2023-09-26T20:49:48.6702343Z","EndedAtUtc":null,"Parameter":{"_lambda_search_space":{"2_neg":-4.00442839,"2_pos":0.776618361,"3_neg":-2.88146996,"3_pos":8.18861198},"_pipeline_":{"_SCHEMA_":"e0 * e5","e0":{},"e1":{"NumberOfLeaves":1487,"MinimumExampleCountPerLeaf":23,"NumberOfTrees":61,"MaximumBinCountPerFeature":114,"FeatureFraction":0.4980983951658306,"LearningRate":1.1821894183157802E-07,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight","DiskTranspose":false},"e2":{"NumberOfTrees":5,"NumberOfLeaves":41,"FeatureFraction":0.404329568,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e3":{"NumberOfLeaves":17267,"MinimumExampleCountPerLeaf":63,"LearningRate":5.8468910203332519E-09,"NumberOfTrees":2221,"SubsampleFraction":2.8959196043297387E-08,"MaximumBinCountPerFeature":11,"FeatureFraction":0.88016485093703201,"L1Regularization":4.0092348571131609E-05,"L2Regularization":0.093939385599205991,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e4":{"L1Regularization":9.83808041,"L2Regularization":4.45183802,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e5":{"L1Regularization":80.4055481,"L2Regularization":2602.4187,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"}}}}
        [Source=AutoMLExperiment, Kind=Trace] current CPU: 0.09375, current Memory(mb): 72.10546875
        [Source=AutoMLExperiment, Kind=Trace] maximum memory usage: , PeakMemoryUsage: 72.10546875 trialIsCancelled: False
        [Source=AutoMLExperiment, Kind=Trace] 
        Exception thrown during Trial 0 with configuration {"TrialId":0,"StartedAtUtc":"2023-09-26T20:49:48.6702343Z","EndedAtUtc":null,"Parameter":{"_lambda_search_space":{"2_neg":-4.00442839,"2_pos":0.776618361,"3_neg":-2.88146996,"3_pos":8.18861198},"_pipeline_":{"_SCHEMA_":"e0 * e5","e0":{},"e1":{"NumberOfLeaves":1487,"MinimumExampleCountPerLeaf":23,"NumberOfTrees":61,"MaximumBinCountPerFeature":114,"FeatureFraction":0.4980983951658306,"LearningRate":1.1821894183157802E-07,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight","DiskTranspose":false},"e2":{"NumberOfTrees":5,"NumberOfLeaves":41,"FeatureFraction":0.404329568,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e3":{"NumberOfLeaves":17267,"MinimumExampleCountPerLeaf":63,"LearningRate":5.8468910203332519E-09,"NumberOfTrees":2221,"SubsampleFraction":2.8959196043297387E-08,"MaximumBinCountPerFeature":11,"FeatureFraction":0.88016485093703201,"L1Regularization":4.0092348571131609E-05,"L2Regularization":0.093939385599205991,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e4":{"L1Regularization":9.83808041,"L2Regularization":4.45183802,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e5":{"L1Regularization":80.4055481,"L2Regularization":2602.4187,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"}}}}

        Exception Details: AUC is not defined when there is no negative class in the data
        Parameter name: NegSample

        Abandoning Trial 0 and continue training.

        [Source=AutoMLExperiment, Kind=Trace] AUC is not defined when there is no negative class in the data
        Parameter name: NegSample   at Microsoft.ML.Data.EvaluatorBase`1.AucAggregatorBase`1.ComputeWeightedAuc(Double& unweighted) in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\AucAggregator.cs:line 116
           at Microsoft.ML.Data.BinaryClassifierEvaluator.Aggregator.Finish() in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\BinaryClassifierEvaluator.cs:line 705
           at Microsoft.ML.Data.BinaryClassifierEvaluator.<>c__DisplayClass32_0.<GetAggregatorConsolidationFuncs>b__0(UInt32 stratColKey, ReadOnlyMemory`1 stratColVal, Aggregator agg) in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\BinaryClassifierEvaluator.cs:line 261
           at Microsoft.ML.Data.EvaluatorBase`1.ProcessData(IDataView data, RoleMappedSchema schema, Func`2 activeColsIndices, TAgg aggregator, AggregatorDictionaryBase[] dictionaries) in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\EvaluatorBase.cs:line 183
           at Microsoft.ML.Data.EvaluatorBase`1.Microsoft.ML.Data.IEvaluator.Evaluate(RoleMappedData data) in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\EvaluatorBase.cs:line 33
           at Microsoft.ML.Data.BinaryClassifierEvaluator.Evaluate(IDataView data, String label, String score, String predictedLabel) in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\BinaryClassifierEvaluator.cs:line 951
           at Microsoft.ML.Fairlearn.GridSearchTrailRunner.RunAsync(TrialSettings settings, CancellationToken ct) in D:\a\_work\1\s\src\Microsoft.ML.Fairlearn\Reductions\GridSearchTrialRunner.cs:line 89
           at Microsoft.ML.AutoML.AutoMLExperiment.<RunAsync>d__24.MoveNext() in D:\a\_work\1\s\src\Microsoft.ML.AutoML\AutoMLExperiment\AutoMLExperiment.cs:line 273
        [Source=AutoMLExperiment, Kind=Info] Update Failed Trial - Id: 0 - Pipeline: Unknown=>SdcaLogisticRegressionBinary

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "System.ArgumentOutOfRangeException : AUC is not defined when there is no negative class in the data",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": true
}

Known issue validation

Build: :mag_right: https://dev.azure.com/dnceng-public/public/_build/results?buildId=419700 Error message validated: System.ArgumentOutOfRangeException : AUC is not defined when there is no negative class in the data Result validation: :x: Known issue did not match with the provided build. Validation performed at: 9/27/2023 6:13:57 PM UTC

Report

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 0