dotnet / machinelearning

ML.NET is an open source and cross-platform machine learning framework for .NET.
https://dot.net/ml
MIT License
8.92k stars 1.86k forks source link

[release/3.0] Update dependencies from dotnet/arcade #6938

Closed dotnet-maestro[bot] closed 5 months ago

dotnet-maestro[bot] commented 6 months ago

This pull request updates the following dependencies

From https://github.com/dotnet/arcade

ericstj commented 6 months ago

We need to port changes from https://github.com/dotnet/machinelearning/pull/6703 and https://github.com/dotnet/machinelearning/pull/6935 here.

codecov[bot] commented 6 months ago

Codecov Report

Attention: 44 lines in your changes are missing coverage. Please review.

Comparison is base (d96d7b7) 68.81% compared to head (df6f817) 68.80%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## release/3.0 #6938 +/- ## =============================================== - Coverage 68.81% 68.80% -0.01% =============================================== Files 1240 1240 Lines 249397 249400 +3 Branches 25496 25494 -2 =============================================== - Hits 171614 171610 -4 - Misses 71189 71196 +7 Partials 6594 6594 ``` | [Flag](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet) | Coverage Δ | | |---|---|---| | [Debug](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet) | `68.80% <82.32%> (-0.01%)` | :arrow_down: | | [production](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet) | `63.26% <75.00%> (-0.01%)` | :arrow_down: | | [test](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet) | `88.49% <82.44%> (+<0.01%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet) | Coverage Δ | | |---|---|---| | [...osoft.Data.Analysis.Tests/ArrowIntegrationTests.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-dGVzdC9NaWNyb3NvZnQuRGF0YS5BbmFseXNpcy5UZXN0cy9BcnJvd0ludGVncmF0aW9uVGVzdHMuY3M=) | `100.00% <100.00%> (ø)` | | | [...icrosoft.Data.Analysis.Tests/DataFrameJoinTests.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-dGVzdC9NaWNyb3NvZnQuRGF0YS5BbmFseXNpcy5UZXN0cy9EYXRhRnJhbWVKb2luVGVzdHMuY3M=) | `100.00% <100.00%> (ø)` | | | [.../Microsoft.ML.AutoML.Tests/ColumnInferenceTests.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-dGVzdC9NaWNyb3NvZnQuTUwuQXV0b01MLlRlc3RzL0NvbHVtbkluZmVyZW5jZVRlc3RzLmNz) | `96.29% <100.00%> (ø)` | | | [...t.ML.AutoML.Tests/SuggestedPipelineBuilderTests.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-dGVzdC9NaWNyb3NvZnQuTUwuQXV0b01MLlRlc3RzL1N1Z2dlc3RlZFBpcGVsaW5lQnVpbGRlclRlc3RzLmNz) | `100.00% <100.00%> (ø)` | | | [...crosoft.ML.Core.Tests/UnitTests/TestEntryPoints.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-dGVzdC9NaWNyb3NvZnQuTUwuQ29yZS5UZXN0cy9Vbml0VGVzdHMvVGVzdEVudHJ5UG9pbnRzLmNz) | `98.45% <100.00%> (ø)` | | | [test/Microsoft.ML.Core.Tests/UnitTests/TestLoss.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-dGVzdC9NaWNyb3NvZnQuTUwuQ29yZS5UZXN0cy9Vbml0VGVzdHMvVGVzdExvc3MuY3M=) | `100.00% <100.00%> (ø)` | | | [...t/Microsoft.ML.Core.Tests/UnitTests/TestVBuffer.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-dGVzdC9NaWNyb3NvZnQuTUwuQ29yZS5UZXN0cy9Vbml0VGVzdHMvVGVzdFZCdWZmZXIuY3M=) | `95.31% <100.00%> (ø)` | | | [test/Microsoft.ML.CpuMath.UnitTests/UnitTests.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-dGVzdC9NaWNyb3NvZnQuTUwuQ3B1TWF0aC5Vbml0VGVzdHMvVW5pdFRlc3RzLmNz) | `95.70% <100.00%> (ø)` | | | [test/Microsoft.ML.FSharp.Tests/SmokeTests.fs](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-dGVzdC9NaWNyb3NvZnQuTUwuRlNoYXJwLlRlc3RzL1Ntb2tlVGVzdHMuZnM=) | `87.23% <ø> (ø)` | | | [test/Microsoft.ML.Fairlearn.Tests/MetricTest.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-dGVzdC9NaWNyb3NvZnQuTUwuRmFpcmxlYXJuLlRlc3RzL01ldHJpY1Rlc3QuY3M=) | `100.00% <100.00%> (ø)` | | | ... and [30 more](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet) | | ... and [6 files with indirect coverage changes](https://app.codecov.io/gh/dotnet/machinelearning/pull/6938/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet)
ericstj commented 5 months ago

For testing we also need https://github.com/dotnet/machinelearning/pull/6924 https://github.com/dotnet/machinelearning/pull/6918 https://github.com/dotnet/machinelearning/pull/6923

ericstj commented 5 months ago

We may want to "merge" commit on this one to preserve distinct history. I can rebase to combine all the small changes from the previous arcade PR to clean things up. Will do that after I see the build passing for auditing purposes (making sure I got every relevant commit).

ericstj commented 5 months ago

Hmm, a couple unique failures here. On Ubuntu:

Finished test: Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowTransformInputOutputTypesTest with memory usage 2,374,266,880.00
Starting test: Microsoft.ML.Tests.NerTests.TestSimpleNer
Finished test: Microsoft.ML.Tests.NerTests.TestSimpleNer with memory usage 1,505,148,928.00
Starting test: Microsoft.ML.Tests.ObjectDetectionTests.SimpleObjDetectionTest
total : 171, filtered: 0, filter ratio: 0.000%
Finished test: Microsoft.ML.Tests.ObjectDetectionTests.SimpleObjDetectionTest with memory usage 1,648,238,592.00
Starting test: Microsoft.ML.Tests.QATests.TestSimpleQA
Killed
+ export _commandExitCode=137

Maybe OOM killer? Though memory went down from previous tests, not sure how much is needed for QA.

Coverage build failed SdcaLogisticRegression test with Assert.InRange() Failure: Value not in range\r\nRange: (0 - 0.5)\r\nActual: 0.51672692765628392 https://dev.azure.com/dnceng-public/public/_build/results?buildId=516419&view=ms.vss-test-web.build-test-results-tab&runId=12113782&resultId=101543

Both of these seem unrelated to this change, but curious for your thoughts @michaelgsharp

ericstj commented 5 months ago

Seems like those were flaky. This is ready for review @michaelgsharp

ericstj commented 5 months ago

@michaelgsharp this is still waiting on your review, please 🙏