Poller::add has platform-specific behavior when trying to insert the same file descriptor multiple times

[elinorbgr]

I encountered this when writing this test for calloop.

I believe this is just the expression of the semantics of the underlying polling systems:

• epoll on Linux returns an error if we try to add the same FD multiple times (fails with EEXIST)
• kqueue on FreeBSD/macOS does not fail and returns a success on the same case (I don't know what it actually does though)

Not sure what would be best to do about it, but I believe it's worth at least documenting it.

[notgull]

This stems from how kqueue works. For other polling systems (e.g. epoll), you register your file descriptor in an associative map that maps your file descriptor to how it wants to be polled. For kqueue, it's a linked list of filters. There's nothing wrong with adding another filter to that list, even if it already exists.

If we want to make the behavior consistent, we could add a HashSet<RawFd> and implement the logic on top of that.

[elinorbgr]

One reason it may be desirable to make the behavior consistent is: "What actually happens if you register the same FD twice with different keys?".

The most likely behavior I suppose is that the new key overwrites the previous one, and following that all events are generated using the second key.

But that may be quite confusing for the user if they did not realize they are registering the same FD twice, and get even worse if users try to rely on the insertion failing, only for that assumption to no longer be valid on other platforms.