[apple][TvOS] Timeout when building aot-instances.dll for System.Numerics.Tensors.Tests

matouskozak commented 3 months ago

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=701353 Build error leg or test failing: System.Numerics.Tensors.Tests Affected CI: tvos_arm64_release_allsubsets_mono (runtime-extra-platforms) Range of commits: https://github.com/dotnet/runtime/compare/a3bf53fc...39ecbe0e

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: :mag_right: Result validation: :warning: Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: :mag_right:" line. Validation performed at: 9/4/2024 4:16:18 PM UTC

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

dotnet-policy-service[bot] commented 3 months ago

Tagging subscribers to 'os-tvos': @vitek-karas, @kotlarmilos, @ivanpovazan, @steveisok, @akoeplinger See info in area-owners.md if you want to be subscribed.

dotnet-policy-service[bot] commented 3 months ago

Tagging subscribers to this area: @directhex, @matouskozak See info in area-owners.md if you want to be subscribed.

matouskozak commented 3 months ago

It looks like it could be related to changes in https://github.com/dotnet/runtime/pull/103005. @akoeplinger do you know what needs to be done so that the test dll can be signed?

matouskozak commented 3 months ago

There is also an ILStrip error which might be the root cause of the issue /tmp/helix/working/B82A0A2B/p/build/apple/AppleBuild.targets(262,5): error : ILStrip failed for /tmp/helix/working/B82A0A2B/w/A9580976/e/publish/System.Numerics.Tensors.dll: Object reference not set to an instance of an object.

vitek-karas commented 2 months ago

@matouskozak Is this still happening- it's has not been hit in the last week per the automated tracking....

matouskozak commented 2 months ago

@matouskozak Is this still happening- it's has not been hit in the last week per the automated tracking....

Yes, still failing, not sure why the failure message changed. Example of recent failing build https://dev.azure.com/dnceng-public/public/_build/results?buildId=732443&view=logs&j=acac6bf3-3edd-5022-4e45-3362a02520b1&t=1de8bf71-8fe5-5d5c-4360-8c2eb159d012. The test is timing out currently ERROR: WORKLOAD TIMED OUT - Killing user command.. thus doesn't have any useful logs to match against.

jeffschwMSFT commented 1 month ago

removing blocking-clean-ci as it has not failed in 30 days

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

matouskozak commented 1 month ago

removing blocking-clean-ci as it has not failed in 30 days

24-Hour Hit Count 7-Day Hit Count 1-Month Count 0 0 0

It's a mobile failure and we don't have a good way to match it for Build Analysis yet. It is still blocking clean extra-platforms CI:

carlossanlop commented 1 month ago

This failure is actually much more widespread than just Tensors. I'll edit the json in the description so all the failures are captured.

Here is an example:

https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-107334-merge-c1d33defa49442c09e/System.Runtime.Tests/1/console.a3d21fde.log?helixlogtype=result

No embedded provisioning profile found at /tmp/helix/working/B06108DF/w/B36209B2/e/System.Runtime.Tests.app/embedded.mobileprovision! Failed to sign the app!

It is affecting at least ios-arm64 and tvos-arm64.

dotnet-policy-service[bot] commented 1 month ago

Tagging subscribers to 'os-ios': @vitek-karas, @kotlarmilos, @ivanpovazan, @steveisok, @akoeplinger See info in area-owners.md if you want to be subscribed.

matouskozak commented 1 month ago

This failure is actually much more widespread than just Tensors. I'll edit the json in the description so all the failures are captured.

Here is an example:

https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-107334-merge-c1d33defa49442c09e/System.Runtime.Tests/1/console.a3d21fde.log?helixlogtype=result
No embedded provisioning profile found at /tmp/helix/working/B06108DF/w/B36209B2/e/System.Runtime.Tests.app/embedded.mobileprovision! Failed to sign the app!
It is affecting at least ios-arm64 and tvos-arm64.

@carlossanlop the embedded.mobileprovision! Failed to sign the app! is actually a generic failure message.

The actual failure actually is error MSB3030: Could not copy the file "/tmp/helix/working/BE410A2F/p/build/microsoft.netcore.app.runtime.ios-arm64/runtimes/ios-arm64/native/System.Runtime.Serialization.Formatters.pdb" because it was not found. and is tracked in https://github.com/dotnet/runtime/issues/107309 (not sure why it didn't get matched).

I'll change this back because we actually have a timeout issue when running dedup on Tensors dll.

dotnet / runtime