search

pytorch / vision

Datasets, Transforms and Models specific to Computer Vision
https://pytorch.org/vision
BSD 3-Clause "New" or "Revised" License
16.04k stars 6.93k forks source link

Skip flaky test_autocast_cpu test #8580

Closed NicolasHug closed 1 month ago

NicolasHug commented 1 month ago

These tests have been flaky for a while and are failing on 1/5000 expected values. It's not easy to specify the atol/rtol here, so I'm just skipping.

2024-08-10T15:17:12.2632960Z FAILED test/test_ops.py::TestRoIAlign::test_autocast_cpu[rois_dtype0-x_dtype1-True-True] - AssertionError: Tensor-likes are not close!
2024-08-10T15:17:12.2633430Z 
2024-08-10T15:17:12.2633500Z Mismatched elements: 1 / 5000 (0.0%)
2024-08-10T15:17:12.2633880Z Greatest absolute difference: 0.0078125 at index (0, 21, 4, 4) (up to 0.005 allowed)
2024-08-10T15:17:12.2634400Z Greatest relative difference: 0.015380859375 at index (0, 21, 4, 4) (up to 0.005 allowed)
2024-08-10T15:17:12.2635150Z FAILED test/test_ops.py::TestRoIAlign::test_autocast_cpu[rois_dtype0-x_dtype1-False-True] - AssertionError: Tensor-likes are not close!
2024-08-10T15:17:12.2635620Z 
2024-08-10T15:17:12.2635690Z Mismatched elements: 1 / 5000 (0.0%)
2024-08-10T15:17:12.2636060Z Greatest absolute difference: 0.0078125 at index (0, 21, 4, 4) (up to 0.005 allowed)
2024-08-10T15:17:12.2636570Z Greatest relative difference: 0.015380859375 at index (0, 21, 4, 4) (up to 0.005 allowed)
2024-08-10T15:17:12.2637360Z FAILED test/test_ops.py::TestRoIAlign::test_autocast_cpu[rois_dtype1-x_dtype1-True-True] - AssertionError: Tensor-likes are not close!
2024-08-10T15:17:12.2637830Z 
2024-08-10T15:17:12.2637900Z Mismatched elements: 1 / 5000 (0.0%)
2024-08-10T15:17:12.2638270Z Greatest absolute difference: 0.0078125 at index (0, 21, 4, 4) (up to 0.005 allowed)
2024-08-10T15:17:12.2638780Z Greatest relative difference: 0.015380859375 at index (0, 21, 4, 4) (up to 0.005 allowed)
2024-08-10T15:17:12.2639520Z FAILED test/test_ops.py::TestRoIAlign::test_autocast_cpu[rois_dtype1-x_dtype1-False-True] - AssertionError: Tensor-likes are not close!
2024-08-10T15:17:12.2640020Z 
2024-08-10T15:17:12.2640100Z Mismatched elements: 1 / 5000 (0.0%)
2024-08-10T15:17:12.2640460Z Greatest absolute difference: 0.0078125 at index (0, 21, 4, 4) (up to 0.005 allowed)
2024-08-10T15:17:12.2641060Z Greatest relative difference: 0.015380859375 at index (0, 21, 4, 4) (up to 0.005 allowed)
pytorch-bot[bot] commented 1 month ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/8580

Note: Links to docs will display an error until the docs builds have been completed.

:x: 14 New Failures, 4 Unrelated Failures

As of commit 4fc4ce354d321be2de74ae7cc02f4766dd25183b with merge base c4492213c0138cd19ef97527fb649d1df3c9d02d (image):

NEW FAILURES - The following jobs have failed:

* [CMake / macos (macos-m1-stable) / macos-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604555676) ([gh](https://github.com/pytorch/vision/actions/runs/10332854143/job/28604555676)) `The process '/usr/bin/git' failed with exit code 128` * [CMake / windows (windows.4xlarge, cpu) / windows-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604555919) ([gh](https://github.com/pytorch/vision/actions/runs/10332854143/job/28604555919)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128` * [CMake / windows (windows.g5.4xlarge.nvidia.gpu, cuda, 11.8) / windows-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604556543) ([gh](https://github.com/pytorch/vision/actions/runs/10332854143/job/28604556543)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128` * [Tests / unittests-linux (3.8, linux.12xlarge, cpu) / linux-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604556289) ([gh](https://github.com/pytorch/vision/actions/runs/10332854141/job/28604556289)) `test/test_datasets_video_utils.py::TestVideo::test_video_clips` * [Tests / unittests-macos (3.10, macos-m1-stable) / macos-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604556839) ([gh](https://github.com/pytorch/vision/actions/runs/10332854141/job/28604556839)) `The process '/usr/bin/git' failed with exit code 128` * [Tests / unittests-macos (3.11, macos-m1-stable) / macos-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604556933) ([gh](https://github.com/pytorch/vision/actions/runs/10332854141/job/28604556933)) `The process '/usr/bin/git' failed with exit code 128` * [Tests / unittests-macos (3.12, macos-m1-stable) / macos-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604557036) ([gh](https://github.com/pytorch/vision/actions/runs/10332854141/job/28604557036)) `The process '/usr/bin/git' failed with exit code 128` * [Tests / unittests-macos (3.8, macos-m1-stable) / macos-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604557125) ([gh](https://github.com/pytorch/vision/actions/runs/10332854141/job/28604557125)) `The process '/usr/bin/git' failed with exit code 128` * [Tests / unittests-macos (3.9, macos-m1-stable) / macos-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604557225) ([gh](https://github.com/pytorch/vision/actions/runs/10332854141/job/28604557225)) `The process '/usr/bin/git' failed with exit code 128` * [Tests / unittests-windows (3.10, windows.4xlarge, cpu) / windows-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604557290) ([gh](https://github.com/pytorch/vision/actions/runs/10332854141/job/28604557290)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128` * [Tests / unittests-windows (3.11, windows.4xlarge, cpu) / windows-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604557351) ([gh](https://github.com/pytorch/vision/actions/runs/10332854141/job/28604557351)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128` * [Tests / unittests-windows (3.12, windows.4xlarge, cpu) / windows-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604557424) ([gh](https://github.com/pytorch/vision/actions/runs/10332854141/job/28604557424)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128` * [Tests / unittests-windows (3.8, windows.4xlarge, cpu) / windows-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604557509) ([gh](https://github.com/pytorch/vision/actions/runs/10332854141/job/28604557509)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128` * [Tests / unittests-windows (3.9, windows.4xlarge, cpu) / windows-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604557695) ([gh](https://github.com/pytorch/vision/actions/runs/10332854141/job/28604557695)) `The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128`

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

* [CMake / linux (linux.12xlarge, cpu) / linux-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604555305) ([gh](https://github.com/pytorch/vision/actions/runs/10332854143/job/28604555305)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [CMake / linux (linux.g5.4xlarge.nvidia.gpu, cuda, 11.8) / linux-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604555504) ([gh](https://github.com/pytorch/vision/actions/runs/10332854143/job/28604555504)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Lint / python-types / linux-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604555438) ([gh](https://github.com/pytorch/vision/actions/runs/10332854144/job/28604555438)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128` * [Tests / unittests-linux (3.8, linux.g5.4xlarge.nvidia.gpu, cuda, 11.8) / linux-job](https://hud.pytorch.org/pr/pytorch/vision/8580#28604555881) ([gh](https://github.com/pytorch/vision/actions/runs/10332854141/job/28604555881)) (matched **linux** rule in [flaky-rules.json](https://github.com/pytorch/test-infra/blob/generated-stats/stats/flaky-rules.json)) `The process '/usr/bin/git' failed with exit code 128`

This comment was automatically generated by Dr. CI and updates every 15 minutes.