OSX PR builds: OMR Socket Test Failures

babsingh commented 2 years ago

Recently, the following OSX failures have been seen in OMR OSX PR builds:

31: [----------] 19 tests from PortSockTest
31: /Users/runner/work/1/s/fvtest/porttest/omrsockTest.cpp:1209: Failure
31:       Expected: rc
31:       Which is: 1
31: To be equal to: 2
31: [  FAILED  ] PortSockTest.poll_functionality_basic (1111 ms)
31: /Users/runner/work/1/s/fvtest/porttest/omrsockTest.cpp:51: Failure
31:       Expected: privateOmrPortLibrary->sock_bind(privateOmrPortLibrary, *serverSocket, serverSockAddr)
31:       Which is: -506
31: To be equal to: 0
31: /Users/runner/work/1/s/fvtest/porttest/omrsockTest.cpp:1259: Failure
31:       Expected: privateOmrPortLibrary->sock_accept(privateOmrPortLibrary, serverSocket, &connectedServerSockAddr, &connectedServerSocket)
31:       Which is: -20
31: To be equal to: 0
31: [  FAILED  ] PortSockTest.poll_functionality_many_sockets (5 ms)
31: [----------] 19 tests from PortSockTest (1121 ms total)
31: 
31: [==========] 236 tests from 20 test cases ran. (110437 ms total)
31: [  PASSED  ] 234 tests.
31: [  FAILED  ] 2 tests, listed below:
31: [  FAILED  ] PortSockTest.poll_functionality_basic
31: [  FAILED  ] PortSockTest.poll_functionality_many_sockets

PRs, where the above failures are seen:

The above failures are not related to the above PRs.

The above failures happen during sock_bind and sock_accept which may indicate involvement of network issues.

knn-k commented 1 year ago

I see the same socket test failures on AArch64 macOS as I mentioned in #6716.

The error code -506 for bind() is OMRPORT_ERROR_SOCKET_ADDRINUSE. OS-native errno is EADDRINUSE (48).
The error code -20 for accept() is OMRPORT_ERROR_INVALID_ARGUMENTS. OS-native errno is EINVAL (22).

hzongaro commented 8 months ago

As @jmesyou pointed out to me off-line, this test has been failing frequently in PR testing for the past 18 months. Should we consider disabling the test for macOS runs, if it's not something that can be fixed?

babsingh commented 8 months ago

Should we consider disabling the test for macOS runs, if it's not something that can be fixed?

The tests pass on the Jenkins macOS machines. The failures are intermittent and only seen on Azure macOS. It's most likely a machine/network issue. Opened https://github.com/eclipse/omr/pull/7213 to disable the failing tests on Azure macOS.

eclipse / omr

OSX PR builds: OMR Socket Test Failures #6516