search

dotnet / performance

This repo contains benchmarks used for testing the performance of all .NET Runtimes
MIT License
687 stars 267 forks source link

Unable to run performance tests against SDK version that is earlier than in global.json #4355

Open caaavik-msft opened 1 month ago

caaavik-msft commented 1 month ago

We get the version of the SDK to use for our MAUI scenarios runs using https://maui.blob.core.windows.net/metadata/sdks/net9.0.json. At the time of this issue, the SDK version that this JSON reports is 9.0.100-preview.5.24304.3 but the version of the SDK that we have in our global.json is 9.0.100-preview.5.24307.3. Because this version is earlier, when we install the MAUI SDK locally with the dotnet install script, it gets ignored. This surfaces in our build pipeline when we try to extract the "Base Path" from the output of dotnet --info, but that information is missing from the command output. An example failure can be seen here: link

I am not sure what the right solution here is, whether it is just that whenever we get a PR from maestro to change the SDK version that we need to wait for the MAUI version to be updated first, or if there is a safer long term solution that means we don't need to worry about checking for that.

caaavik-msft commented 1 month ago

I realise now this will likely get resolved with #4321 when it is completed, but not sure if that will always guarantee the SDK version used for MAUI runs will be allowed with what we have in global.json

LoopedBard3 commented 1 month ago

I realise now this will likely get resolved with #4321 when it is completed, but not sure if that will always guarantee the SDK version used for MAUI runs will be allowed with what we have in global.json

I think that is true that it will at least be fixed temporarily for that. A potential solution may be to update/remove the global.json when we are grabbing a non-default dotnet version. Not great but that would work if we think we are ever going to be testing with sdk behind main.