search

pytorch / executorch

On-device AI across mobile, embedded and edge for PyTorch
https://pytorch.org/executorch/
Other
2.04k stars 333 forks source link

[Flaky] test-demo-ios / macos-job #2877

Open guangy10 opened 6 months ago

guangy10 commented 6 months ago

I've seen this job being flaky in many PRs. For example, 

2024-04-05T18:47:27.4678510Z    MobileNetClassifierTest.testV3WithCoreMLBackend()
2024-04-05T18:47:27.4678840Z 
2024-04-05T18:47:27.4678950Z ** TEST FAILED **
2024-04-05T18:47:27.4679220Z 
2024-04-05T18:47:27.4679280Z Testing started
2024-04-05T18:47:27.4679840Z Test suite 'MobileNetClassifierTest' started on 'Clone 1 of executorch - xctest (83121)'
2024-04-05T18:47:27.4680680Z Test case 'MobileNetClassifierTest.testV3WithCoreMLBackend()' failed on 'Clone 1 of executorch - xctest (83121)' (5.452 seconds)
2024-04-05T18:47:27.4681790Z Test case 'MobileNetClassifierTest.testV3WithMPSBackend()' passed on 'Clone 1 of executorch - xctest (83121)' (10.237 seconds)
2024-04-05T18:47:27.4682870Z Test case 'MobileNetClassifierTest.testV3WithPortableBackend()' passed on 'Clone 1 of executorch - xctest (83121)' (1.780 seconds)
2024-04-05T18:47:27.4684040Z Test case 'MobileNetClassifierTest.testV3WithXNNPACKBackend()' passed on 'Clone 1 of executorch - xctest (83121)' (0.424 seconds)
2024-04-05T18:47:27.4684680Z 
2024-04-05T18:47:27.4685100Z    ** Deleting Simulator **
2024-04-05T18:47:27.4685470Z 
2024-04-05T18:47:27.4685800Z 
2024-04-05T18:47:27.4686100Z    ** FAILED **
cymbalrush commented 1 month ago

@guangy10 are you still seeing this?