Open Torrekie opened 2 years ago
Torrekie
commented
2 years ago Now I can confirm this works on macOS/iOS, but testsuites cannot be compiled since it was using many calls that not exist on Darwin(e.g. pipe2() and accept4()), probably needs to rewrite all tests for Darwin
Torrekie
commented
2 years ago Some tests failed, and it stuck at test 23
TorrekiedeMBP:epoll-shim torrekie$ make -C build/ test
make: Entering directory '/Users/torrekie/proj/epoll-shim/build'
Running tests...
Test project /Users/torrekie/proj/epoll-shim/build
Start 1: epoll-test.epoll__simple
1/130 Test #1: epoll-test.epoll__simple .............................................. Passed 0.03 sec
Start 2: epoll-test.epoll__poll_flags
2/130 Test #2: epoll-test.epoll__poll_flags .......................................... Passed 0.03 sec
Start 3: epoll-test.epoll__leakcheck
3/130 Test #3: epoll-test.epoll__leakcheck ........................................... Passed 0.03 sec
Start 4: epoll-test.epoll__fd_exhaustion
4/130 Test #4: epoll-test.epoll__fd_exhaustion ....................................... Passed 0.03 sec
Start 5: epoll-test.epoll__invalid_op
5/130 Test #5: epoll-test.epoll__invalid_op .......................................... Passed 0.03 sec
Start 6: epoll-test.epoll__invalid_op2
6/130 Test #6: epoll-test.epoll__invalid_op2 ......................................... Passed 0.03 sec
Start 7: epoll-test.epoll__simple_wait
7/130 Test #7: epoll-test.epoll__simple_wait ......................................... Passed 0.03 sec
Start 8: epoll-test.epoll__event_size
8/130 Test #8: epoll-test.epoll__event_size ..........................................***Failed 0.03 sec
Start 9: epoll-test.epoll__recursive_register
9/130 Test #9: epoll-test.epoll__recursive_register .................................. Passed 0.04 sec
Start 10: epoll-test.epoll__simple_epollin
10/130 Test #10: epoll-test.epoll__simple_epollin ...................................... Passed 0.04 sec
Start 11: epoll-test.epoll__sleep_argument
11/130 Test #11: epoll-test.epoll__sleep_argument ...................................... Passed 0.26 sec
Start 12: epoll-test.epoll__remove_nonexistent
12/130 Test #12: epoll-test.epoll__remove_nonexistent .................................. Passed 0.05 sec
Start 13: epoll-test.epoll__add_remove
13/130 Test #13: epoll-test.epoll__add_remove .......................................... Passed 0.03 sec
Start 14: epoll-test.epoll__add_existing
14/130 Test #14: epoll-test.epoll__add_existing ........................................ Passed 0.03 sec
Start 15: epoll-test.epoll__modify_existing
15/130 Test #15: epoll-test.epoll__modify_existing ..................................... Passed 0.09 sec
Start 16: epoll-test.epoll__modify_nonexisting
16/130 Test #16: epoll-test.epoll__modify_nonexisting .................................. Passed 0.02 sec
Start 17: epoll-test.epoll__poll_only_fd
17/130 Test #17: epoll-test.epoll__poll_only_fd ........................................***Failed 0.03 sec
Start 18: epoll-test.epoll__no_epollin_on_closed_empty_pipe
18/130 Test #18: epoll-test.epoll__no_epollin_on_closed_empty_pipe ..................... Passed 0.03 sec
Start 19: epoll-test.epoll__write_to_pipe_until_full
19/130 Test #19: epoll-test.epoll__write_to_pipe_until_full ............................ Passed 0.35 sec
Start 20: epoll-test.epoll__realtime_timer
20/130 Test #20: epoll-test.epoll__realtime_timer ...................................... Passed 1.27 sec
Start 21: epoll-test.epoll__simple_signalfd
21/130 Test #21: epoll-test.epoll__simple_signalfd ..................................... Passed 0.05 sec
Start 22: epoll-test.epoll__signalfd_poll_sigusr1
22/130 Test #22: epoll-test.epoll__signalfd_poll_sigusr1 ...............................***Failed 0.03 sec
Start 23: epoll-test.epoll__signalfd_in_thread
^Cmake: *** [Makefile:71: test] Interrupt: 2The implementations of ppoll and sigtimedwait might not quite working as expected
Torrekie
commented
2 years ago I think I need some help on this, I have made tests compile, but many of them failed, I'm not sure which part causing problems For macOS 11.2.3 arm64:
73% tests passed, 71 tests failed out of 263
Total Test time (real) = 72.77 sec
The following tests did not run:
25 - epoll-test.epoll__epollhup_on_fresh_socket (Skipped)
67 - epoll-test-interpose.epoll__epollhup_on_fresh_socket (Skipped)
85 - timerfd-test.timerfd__many_timers (Skipped)
104 - timerfd-test-interpose.timerfd__many_timers (Skipped)
123 - timerfd-root-test.timerfd_root__zero_read_on_abs_realtime (Skipped)
124 - timerfd-root-test.timerfd_root__read_on_abs_realtime_no_interval (Skipped)
125 - timerfd-root-test.timerfd_root__cancel_on_set (Skipped)
126 - timerfd-root-test.timerfd_root__cancel_on_set_init (Skipped)
127 - timerfd-root-test.timerfd_root__clock_change_notification (Skipped)
128 - timerfd-root-test.timerfd_root__advance_time_no_cancel (Skipped)
129 - timerfd-root-test-interpose.timerfd_root__zero_read_on_abs_realtime (Skipped)
130 - timerfd-root-test-interpose.timerfd_root__read_on_abs_realtime_no_interval (Skipped)
131 - timerfd-root-test-interpose.timerfd_root__cancel_on_set (Skipped)
132 - timerfd-root-test-interpose.timerfd_root__cancel_on_set_init (Skipped)
133 - timerfd-root-test-interpose.timerfd_root__clock_change_notification (Skipped)
134 - timerfd-root-test-interpose.timerfd_root__advance_time_no_cancel (Skipped)
135 - timerfd-mock-test.timerfd_mock__mocked_kevent (Skipped)
136 - timerfd-mock-test-interpose.timerfd_mock__mocked_kevent (Skipped)
145 - signalfd-test.signalfd__sigwait_openbsd (Skipped)
146 - signalfd-test.signalfd__sigchld (Skipped)
156 - signalfd-test-interpose.signalfd__sigwait_openbsd (Skipped)
157 - signalfd-test-interpose.signalfd__sigchld (Skipped)
159 - perf-many-fds.perf_many_fds__perf (Skipped)
160 - perf-many-fds-interpose.perf_many_fds__perf (Skipped)
173 - eventfd-ctx-test.eventfd__pollout (Skipped)
180 - eventfd-ctx-test.eventfd__fork (Skipped)
183 - eventfd-ctx-test.eventfd__epoll (Skipped)
187 - eventfd-ctx-test-interpose.eventfd__pollout (Skipped)
194 - eventfd-ctx-test-interpose.eventfd__fork (Skipped)
197 - eventfd-ctx-test-interpose.eventfd__epoll (Skipped)
200 - pipe-test.pipe__poll_write_end_after_read_end_close (Skipped)
205 - pipe-test.pipe__poll_full_minus_512b_write_end_after_read_end_close (Skipped)
213 - pipe-test.pipe__closed_read_end_of_duplex (Skipped)
218 - pipe-test-interpose.pipe__poll_write_end_after_read_end_close (Skipped)
223 - pipe-test-interpose.pipe__poll_full_minus_512b_write_end_after_read_end_close (Skipped)
231 - pipe-test-interpose.pipe__closed_read_end_of_duplex (Skipped)
The following tests FAILED:
8 - epoll-test.epoll__event_size (Failed)
17 - epoll-test.epoll__poll_only_fd (Failed)
22 - epoll-test.epoll__signalfd_poll_sigusr1 (Failed)
23 - epoll-test.epoll__signalfd_in_thread (Failed)
27 - epoll-test.epoll__epollpri (Failed)
28 - epoll-test.epoll__epollpri_oobinline (Failed)
29 - epoll-test.epoll__epollpri_oobinline_lt (Failed)
40 - epoll-test.epoll__epoll_pwait (Failed)
41 - epoll-test.epoll__cloexec (Failed)
50 - epoll-test-interpose.epoll__event_size (Failed)
59 - epoll-test-interpose.epoll__poll_only_fd (Failed)
64 - epoll-test-interpose.epoll__signalfd_poll_sigusr1 (Failed)
65 - epoll-test-interpose.epoll__signalfd_in_thread (Failed)
69 - epoll-test-interpose.epoll__epollpri (Failed)
70 - epoll-test-interpose.epoll__epollpri_oobinline (Failed)
71 - epoll-test-interpose.epoll__epollpri_oobinline_lt (Failed)
82 - epoll-test-interpose.epoll__epoll_pwait (Failed)
83 - epoll-test-interpose.epoll__cloexec (Failed)
86 - timerfd-test.timerfd__simple_timer (Failed)
87 - timerfd-test.timerfd__simple_periodic_timer (Failed)
88 - timerfd-test.timerfd__complex_periodic_timer (Failed)
89 - timerfd-test.timerfd__reset_periodic_timer (Failed)
90 - timerfd-test.timerfd__reenable_periodic_timer (Failed)
91 - timerfd-test.timerfd__expire_five (Failed)
95 - timerfd-test.timerfd__upgrade_simple_to_complex (Failed)
96 - timerfd-test.timerfd__absolute_timer (Failed)
101 - timerfd-test.timerfd__short_evfilt_timer_timeout (Failed)
102 - timerfd-test.timerfd__unmodified_errno (Failed)
105 - timerfd-test-interpose.timerfd__simple_timer (Failed)
106 - timerfd-test-interpose.timerfd__simple_periodic_timer (Failed)
107 - timerfd-test-interpose.timerfd__complex_periodic_timer (Failed)
108 - timerfd-test-interpose.timerfd__reset_periodic_timer (Failed)
109 - timerfd-test-interpose.timerfd__reenable_periodic_timer (Failed)
110 - timerfd-test-interpose.timerfd__expire_five (Failed)
114 - timerfd-test-interpose.timerfd__upgrade_simple_to_complex (Failed)
115 - timerfd-test-interpose.timerfd__absolute_timer (Failed)
120 - timerfd-test-interpose.timerfd__short_evfilt_timer_timeout (Failed)
121 - timerfd-test-interpose.timerfd__unmodified_errno (Failed)
137 - signalfd-test.signalfd__simple_signalfd (Failed)
140 - signalfd-test.signalfd__multiple_signals (Failed)
144 - signalfd-test.signalfd__sigwaitinfo (Failed)
148 - signalfd-test-interpose.signalfd__simple_signalfd (Failed)
151 - signalfd-test-interpose.signalfd__multiple_signals (Failed)
155 - signalfd-test-interpose.signalfd__sigwaitinfo (Failed)
193 - eventfd-ctx-test-interpose.eventfd__threads_read (Failed)
202 - pipe-test.pipe__poll_full_write_end_after_read_end_close_hup (Failed)
206 - pipe-test.pipe__pipe_event_poll (Failed)
207 - pipe-test.pipe__fifo_writes (Failed)
208 - pipe-test.pipe__fifo_connecting_reader (Failed)
209 - pipe-test.pipe__fifo_reads (Failed)
210 - pipe-test.pipe__fifo_read_eof_wakeups (Failed)
211 - pipe-test.pipe__fifo_read_eof_state_when_reconnecting (Failed)
212 - pipe-test.pipe__closed_read_end (Failed)
214 - pipe-test.pipe__closed_read_end_register_before_close (Failed)
215 - pipe-test.pipe__closed_write_end (Failed)
216 - pipe-test.pipe__closed_write_end_register_before_close (Failed)
220 - pipe-test-interpose.pipe__poll_full_write_end_after_read_end_close_hup (Failed)
224 - pipe-test-interpose.pipe__pipe_event_poll (Failed)
225 - pipe-test-interpose.pipe__fifo_writes (Failed)
226 - pipe-test-interpose.pipe__fifo_connecting_reader (Failed)
227 - pipe-test-interpose.pipe__fifo_reads (Failed)
228 - pipe-test-interpose.pipe__fifo_read_eof_wakeups (Failed)
229 - pipe-test-interpose.pipe__fifo_read_eof_state_when_reconnecting (Failed)
230 - pipe-test-interpose.pipe__closed_read_end (Failed)
232 - pipe-test-interpose.pipe__closed_read_end_register_before_close (Failed)
233 - pipe-test-interpose.pipe__closed_write_end (Failed)
234 - pipe-test-interpose.pipe__closed_write_end_register_before_close (Failed)
236 - socketpair-test.socketpair__simple_edge_triggering (Failed)
240 - socketpair-test-interpose.socketpair__simple_edge_triggering (Failed)
252 - tst-epoll.epollfd_osv__epoll_file (Failed)
255 - tst-epoll-interpose.epollfd_osv__epoll_file (Failed)
Errors while running CTest
jiixyj
commented
2 years ago Thanks for you work on this! I actually tried to port this library to macOS last year, but I was discouraged by macOS' poor support for recent (and not so recent) POSIX features, and the fact that it's a bit hard for me to test this. I noticed I hadn't pushed this work yet (shame on me), but now I rebased my old branch and pushed it into the macos branch. I'll have a look at your branch, and try to merge our work.
Back then I got the EVFILT_EXCEPT stuff working and all the tests, at least on x64 macOS. By "working" I mean I had to skip a lot of them, sadly.
I think we should also look at getting a CI build for macOS running, at least for x64. GitHub Actions should support this.
Torrekie
commented
2 years ago shall I rebase my current darwin branch and do pr on your macos branch?
jiixyj
commented
2 years ago shall I rebase my current darwin branch and do pr on your macos branch?
Yes, that would be great!
Torrekie
commented
2 years ago Some tests skipped due to different behavior with kevent(), then I found the struct kevent was different between Darwin and other BSDs
#ifdef __NetBSD__
struct kevent {
uintptr_t ident; /* identifier for this event */
uint32_t filter; /* filter for event */
uint32_t flags; /* action flags for kqueue */
uint32_t fflags; /* filter flag value */
int64_t data; /* filter data value */
intptr_t udata; /* opaque user data identifier */
};
#elif defined(__APPLE__)
struct kevent {
uintptr_t ident; /* identifier for this event */
int16_t filter; /* filter for event */
uint16_t flags; /* general flags */
uint32_t fflags; /* filter-specific flags */
intptr_t data; /* filter-specific data */
void *udata; /* opaque user data identifier */
};
#endifConsider switch to kevent64 on Darwin
// In <sys/event.h>
struct kevent64_s {
uint64_t ident; /* identifier for this event */
int16_t filter; /* filter for event */
uint16_t flags; /* general flags */
uint32_t fflags; /* filter-specific flags */
int64_t data; /* filter-specific data */
uint64_t udata; /* opaque user data identifier */
uint64_t ext[2]; /* filter-specific extensions */
};
#define EV_SET64(kevp, a, b, c, d, e, f, g, h) do { \
struct kevent64_s *__kevp__ = (kevp); \
__kevp__->ident = (a); \
__kevp__->filter = (b); \
__kevp__->flags = (c); \
__kevp__->fflags = (d); \
__kevp__->data = (e); \
__kevp__->udata = (f); \
__kevp__->ext[0] = (g); \
__kevp__->ext[1] = (h); \
} while(0)
int kevent64(int kq,
const struct kevent64_s *changelist, int nchanges,
struct kevent64_s *eventlist, int nevents,
unsigned int flags,
const struct timespec *timeout);Seems nothing to change for now except some skipped tests
wegank
commented
2 years ago Putting the definition of itimerspec in <sys/timerfd.h> causes build problems in PipeWire. I think it really should be defined elsewhere.
Torrekie
commented
2 years ago then we probably have to create a sys/time.h like that
#ifndef _LIBEPOLL_SHIM_SYS_TIME_H_
#define _LIBEPOLL_SHIM_SYS_TIME_H_
#include_next <sys/time.h>
#if __APPLE__
struct itimerspec {
struct timespec it_interval;
struct timespec it_value;
};
#endif // __APPLE__
#endif // _LIBEPOLL_SHIM_SYS_TIME_H_
kqueue1()since Darwin forced close-on-exec for its file descriptors, I'm not sure if this going to affect epoll functions