"5 new packages used not in baseline!" after pulling the latest arcade from .Net 7 Eng channel

[YuliiaKovalova]

• [ X ] This issue is blocking MSBuild servicing is coming, the issue fails the build

After applying recent changes from arcade, source-build step fails with "5 new packages used not in baseline!" please see the details in the build: https://tfsprodwus2su6.visualstudio.com/A011b8bdf-6d56-4f87-be0d-0092136884d9/DevDiv/_build/results?buildId=9289342&view=logs&j=2f0d093c-1064-5c86-fc5b-b7b1eca8e66a&t=52d0a7a6-39c9-5fa2-86e8-78f84e98a3a2

[missymessa]

Did you follow the steps in the error output:

[error].packages/microsoft.dotnet.arcade.sdk/7.0.0-beta.24171.6/tools/SourceBuild/AfterSourceBuild.proj(65,5): error : (NETCORE_ENGINEERING_TELEMETRY=AfterSourceBuild) 5 new packages used not in baseline! See report at /mnt/vss/_work/1/s/artifacts/source-build/self/prebuilt-report/baseline-comparison.xml for more information. Package IDs are:

Microsoft.Build.Tasks.Git.1.2.0-beta-22429-01 Microsoft.DotNet.XliffTasks.1.0.0-beta.22427.1 Microsoft.SourceLink.AzureRepos.Git.1.2.0-beta-22429-01 Microsoft.SourceLink.Common.1.2.0-beta-22429-01 Microsoft.SourceLink.GitHub.1.2.0-beta-22429-01 Prebuilt usages are different from the baseline found at '/mnt/vss/_work/1/s/eng/SourceBuildPrebuiltBaseline.xml'. If it's acceptable to update the baseline, copy the contents of the automatically generated baseline '/mnt/vss/_work/1/s/artifacts/source-build/self/prebuilt-report/generated-new-baseline.xml'.

[JanKrivanek]

@missymessa are those actually comming from arcade update? Shouldn't the arcade update the baseline as well?

[missymessa]

/cc @riarenas @dougbu

[YuliiaKovalova]

If it's acceptable to update the baseline

Nope, I didn't because I don't know: "If it's acceptable to update the baseline" :) Could you tell me when it's ok? It would be great to change this message, because now it looks a bit confusing for a rookie.

[missymessa]

/cc @MichaelSimons because Source Build ^

[MichaelSimons]

/cc @MichaelSimons because Source Build ^

Friendly guidance to use the @dotnet/source-build-internal alias instead of pinging me directly. I am not always available and therefore someone else can respond.

That aside, wow were these changes brought it? It doesn't look like darc was used. The reason I saw that is because it doesn't look like the coherent parent dependencies of arcade were also updated.

    <Dependency Name="Microsoft.SourceLink.GitHub" Version="1.1.0-beta-21480-02" CoherentParentDependency="Microsoft.DotNet.Arcade.Sdk">
      <Uri>https://github.com/dotnet/sourcelink</Uri>
      <Sha>8031e5220baf2acad991e661d8308b783d2acf3e</Sha>
      <SourceBuild RepoName="sourcelink" ManagedOnly="true" />
    </Dependency>
    <Dependency Name="Microsoft.DotNet.XliffTasks" Version="1.0.0-beta.21431.1" CoherentParentDependency="Microsoft.DotNet.Arcade.Sdk">
      <Uri>https://github.com/dotnet/xliff-tasks</Uri>
      <Sha>bc3233146e1fcd393ed471d5005333c83363e0fe</Sha>
      <SourceBuild RepoName="xliff-tasks" ManagedOnly="true" />
    </Dependency>

Looking at the version of arcade you updated to (https://github.com/dotnet/arcade/blob/834d09f0c963da37c88aed7ba3a83acafe582948/eng/Version.Details.xml), these versions should have changed as well. Please use darc when flowing dependencies.

[YuliiaKovalova]

/cc @MichaelSimons because Source Build ^

Friendly guidance to use the @dotnet/source-build-internal alias instead of pinging me directly. I am not always available and therefore someone else can respond.

That aside, wow were these changes brought it? It doesn't look like darc was used. The reason I saw that is because it doesn't look like the coherent parent dependencies of arcade were also updated.

    <Dependency Name="Microsoft.SourceLink.GitHub" Version="1.1.0-beta-21480-02" CoherentParentDependency="Microsoft.DotNet.Arcade.Sdk">
      <Uri>https://github.com/dotnet/sourcelink</Uri>
      <Sha>8031e5220baf2acad991e661d8308b783d2acf3e</Sha>
      <SourceBuild RepoName="sourcelink" ManagedOnly="true" />
    </Dependency>
    <Dependency Name="Microsoft.DotNet.XliffTasks" Version="1.0.0-beta.21431.1" CoherentParentDependency="Microsoft.DotNet.Arcade.Sdk">
      <Uri>https://github.com/dotnet/xliff-tasks</Uri>
      <Sha>bc3233146e1fcd393ed471d5005333c83363e0fe</Sha>
      <SourceBuild RepoName="xliff-tasks" ManagedOnly="true" />
    </Dependency>

Looking at the version of arcade you updated to (https://github.com/dotnet/arcade/blob/834d09f0c963da37c88aed7ba3a83acafe582948/eng/Version.Details.xml), these versions should have changed as well. Please use darc when flowing dependencies.

Hi @MichaelSimons ,

Thank you for looking at it.

I used darc, this command specifically: darc update-dependencies --source-repo dotnet/arcade --channel ".NET 7 Eng" I have just repeated the same action for the branch and there is nothing missed:

remote branch: https://github.com/dotnet/msbuild/compare/exp/ykovalova/1_esTempaltes_vs17.7

[MichaelSimons]

I will defer back to the engineering services team to explain why darc did not update the CoherentParentDependency relationships. That is why the source-build leg is detecting prebuilts.

[JanKrivanek]

^ @dotnet/dnceng

[chcosta]

@mmitche

[missymessa]

Probably wait until Monday for Ricardo to take a look at this, since he handled the 7.0 update and probably has the most context here. (He is oof today)

[mmitche]

I'm unsure why the coherency update didn't happen, but I pushed an update to your branch @YuliiaKovalova

[mmitche]

@YuliiaKovalova FYI the --repo parameter there is not substring compatible. You need the exact repo name https://github.com/dotnet/arcade

[JanKrivanek]

@YuliiaKovalova FYI the --repo parameter there is not substring compatible. You need the exact repo name https://github.com/dotnet/arcade

You mean --source-repo parameter of update-dependencies? It specifically says 'contains' in the description. Based on the screenshot above (https://github.com/dotnet/arcade/issues/14640#issuecomment-2015645685) it seems like it picked the url properly.

[YuliiaKovalova]

I'm unsure why the coherency update didn't happen, but I pushed an update to your branch @YuliiaKovalova

Hi @mmitche,

Could you point to the changes you made? I checked these 2 branches and didn't spot anything: DotNet-msbuild-Trusted - Repos (visualstudio.com) Comparing main...exp/ykovalova/1_esTempaltes_vs17.7 · dotnet/msbuild (github.com)

Also, I attempted to use url for--source-repo param and it didn't pull anything:

Could please you elaborate?

[mmitche]

Ahh it looks like my changes failed to push. Before I do that, can you update your darc client (eng/common/darc-init.ps1) and try the dependency update again? It does the expected thing for me.

Matt

Sent from Outlookhttp://aka.ms/weboutlook

---

From: YuliiaKovalova @.> Sent: Monday, March 25, 2024 3:05 AM To: dotnet/arcade @.> Cc: Mention @.>; Comment @.> Subject: Re: [dotnet/arcade] "5 new packages used not in baseline!" after pulling the latest arcade from .Net 7 Eng channel (Issue #14640)

I'm unsure why the coherency update didn't happen, but I pushed an update to your branch @YuliiaKovalovahttps://github.com/YuliiaKovalova

Hi @mmitchehttps://github.com/mmitche,

Could you point to the changes you made? I checked these 2 branches and didn't spot anything: DotNet-msbuild-Trusted - Repos (visualstudio.com)https://tfsprodwus2su6.visualstudio.com/A011b8bdf-6d56-4f87-be0d-0092136884d9/DevDiv/_git/DotNet-msbuild-Trusted?version=GBexp%2Fykovalova%2F1_esTempaltes_vs17.7&_a=history Comparing main...exp/ykovalova/1_esTempaltes_vs17.7 · dotnet/msbuild (github.com)https://github.com/dotnet/msbuild/compare/exp/ykovalova/1_esTempaltes_vs17.7

Also, I attempted to use url for--source-repo param and it didn't pull anything: image.png (view on web)https://github.com/dotnet/arcade/assets/95473390/78c13d19-8992-4358-bd63-10843a34af48

Could please you elaborate?

— Reply to this email directly, view it on GitHubhttps://github.com/dotnet/arcade/issues/14640#issuecomment-2017636357 or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACCSFMUXSZIGWQRANVSPDPLYZ7ZHLBFKMF2HI4TJMJ2XIZLTSOBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLLDTOVRGUZLDORPXI6LQMWWES43TOVSUG33NNVSW45FGORXXA2LDOOJIFJDUPFYGLKTSMVYG643JORXXE6NFOZQWY5LFVEYTEMJUGQ2DGMRVQKSHI6LQMWSWS43TOVS2K5TBNR2WLKRSGIYDENJSGAYTENVHORZGSZ3HMVZKMY3SMVQXIZI. You are receiving this email because you were mentioned.

Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[YuliiaKovalova]

@mmitche , thank you for recommendation. I was able to update darc and pull updates, but it didn't resolve the main problem: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=9300907&view=results

[mmitche]

Very odd that it didn't update coherency....can you post your output? When I did your command locally I did get an error. I did push an update to the devdiv repo with coherency updates though.

[mmitche]

I'm getting the following:

PS C:\r\msbuild> darc update-dependencies --id 218828
Looking up build with BAR id 218828
Updating 'Microsoft.DotNet.Arcade.Sdk': '7.0.0-beta.24171.6' => '7.0.0-beta.24172.6' (from build '20240322.6' of 'https://github.com/dotnet/arcade')
Updating 'Microsoft.DotNet.XUnitExtensions': '7.0.0-beta.24171.6' => '7.0.0-beta.24172.6' (from build '20240322.6' of 'https://github.com/dotnet/arcade')
Checking for coherency updates...
Updating 'Microsoft.SourceLink.GitHub': '1.1.0-beta-21480-02' => '1.2.0-beta-22429-01' to ensure coherency with Microsoft.DotNet.Arcade.Sdk@7.0.0-beta.24172.6
Updating 'Microsoft.DotNet.XliffTasks': '1.0.0-beta.21431.1' => '1.0.0-beta.22427.1' to ensure coherency with Microsoft.DotNet.Arcade.Sdk@7.0.0-beta.24172.6
fail: Error: Failed to update dependencies.
      Microsoft.DotNet.DarcLib.DependencyFileNotFoundException: Could not find global.json in https://github.com/dotnet/arcade
         at Microsoft.DotNet.DarcLib.LocalGitClient.GetFileContentsAsync(String relativeFilePath, String repoPath, String branch) in /_/src/Microsoft.DotNet.Darc/DarcLib/LocalGitClient.cs:line 46
         at Microsoft.DotNet.DarcLib.DependencyFileManager.ReadGlobalJsonAsync(String repoUri, String branch) in /_/src/Microsoft.DotNet.Darc/DarcLib/Helpers/DependencyFileManager.cs:line 104
         at Microsoft.DotNet.DarcLib.DependencyFileManager.ReadToolsDotnetVersionAsync(String repoUri, String commit) in /_/src/Microsoft.DotNet.Darc/DarcLib/Helpers/DependencyFileManager.cs:line 134
         at Microsoft.DotNet.DarcLib.Local.UpdateDependenciesAsync(List`1 dependencies, IRemoteFactory remoteFactory, IBarApiClient barClient) in /_/src/Microsoft.DotNet.Darc/DarcLib/Actions/Local.cs:line 77
         at Microsoft.DotNet.Darc.Operations.UpdateDependenciesOperation.ExecuteAsync() in /_/src/Microsoft.DotNet.Darc/Darc/Operations/UpdateDependenciesOperation.cs:line 236

[mmitche]

I updated my copy and I can repro @YuliiaKovalova's issue. It looks like it somehow misses coherency when you don't use a build ID.

[mmitche]

https://github.com/dotnet/arcade-services/pull/3448

[YuliiaKovalova]

I updated my copy and I can repro @YuliiaKovalova's issue. It looks like it somehow misses coherency when you don't use a build ID.

@mmitche, thank you for digging deeper in it. Now I don't feel bad for taking so much of your time :)

[riarenas]

with https://github.com/dotnet/arcade-services/issues/3447 as the underlying issue, I believe we can now close this one. Feel free to reopen if there's anything else here that isn't covered by the Arcade-services changes.