Bring over bug fixes and enhancements from the Spirent fork of OSv

[wkozaczuk]

Here is the curated list of bug fixes from Spirent fork (this list and another below do not include most of the IPV6 code on ipv6 branch, nor the commits that have been already merged into OSv master, nor recent patches I have submitted to implement netlink and fix getifaddrs, if_nameindex, if_nametoindex and if_indextoname):

• Don't leak directory entries when processing the init entries - no brainer (has been merged)
• tcp_input: net channel fixes + better state handling - this one is difficult to assess without understanding the TCP stack code and net channels
• libc: add pthread_getname_np - we have pthread_getname_np() on master now but possibly can improve error handling like in the Spirent fork
• loader.py: Fix property name in dump_trace - seems like a no brainer
• this one is not really a bug but a change to the CPU idle polling with some interesting comments
• scripts/check-image-size.sh: Use numeric {u,g}ids instead of text with ls
• tcp_input.cc: Fix a race condition on tcp listen socket teardown - backport from FreeBSD
• tcp_input.cc: Addendum to tcp_input.cc: Fix a race condition on tcp listen socket teardown
• tcp_input.cc: Another addendum to tcp_input.cc: Fix a race condition on tcp listen socket teardown
• various poll/epoll fixes/enhancements
  • epoll: Bump the file reference count while polling - is this truly a bug?
    • the patch was actually sent in 2016 and reviewed by @nyh who had some thoughts about it (https://groups.google.com/g/osv-dev/c/gdpEyn_3yYo/m/f_cI5oj8AAAJ)
  • poll: drop POLLINIGNEOF and fix POLLRDHUP
  • tests/test-epoll: add socket EPOLLRDHUP test
• gdb: fix multiple osv commands
• pthread: add simplisitic {get,set}schedparam - this looks better and possibly more functional than what we have now which I added with this commit d39860d11; on other hand the Spirent version seems to assume the default policy as SCHED_RR which I am not sure is correct
• mmu: improve vma_list searches in fix_hole - this may no longer be applicable as I changed that code to account for linear mappings
• tcp_input.cc: acknowledge SYN 'byte' when connection established
• fix recvmsg() not using flags parameter - possibly on ipv6 branch; also is it really an enhancement
• Fix issue with if_add_ipv4_addr not setting sockaddr family for mask parameter - seems like a no-brainer (maybe)
• fix broken IPv4 default gateway - this seems to matter once we have IPV6 support on master but maybe worth merging now
• Support MTU greater than 1500 bytes. Handle SIOCSIFMTU in the virtio-net driver rather than delegating to ether_ioctl

My sense is that some or possibly all network stack fixes from the list above are on the ipv6 branch of the main repo.

Here is the curated list of enhancements from Spirent fork that map to one or more commits:

• syscalls: allocate local iovec copies on the stack instead of the heap - this seems a no brainer but not have a clue how much difference does it really make (has been merged)
• virtio-blk: Use multiplex strategy for I/O (has been merged)
• rcu: replace double-buffered defer queue with a ring - it would be nice to assess if the ring is truly a better alternative; there is a number of following commits:
  • rcu: Wake the quiescent thread when the ring is half full
  • rcu: Increase the defer ring size
• Add BLKGETSIZE ioctl
• Add POSIX AIO support which actually came through multiple patches so I am just linking the end result:
  • implementation - https://github.com/SpirentOrion/osv/blob/master/core/posix-aio.cc
  • unit test - https://github.com/SpirentOrion/osv/blob/master/tests/tst-posix-aio.cc
• hypervisor "steal" time driver for KVM and VMware; as above came in the form of many commits but the "steal" time functionality does not seem to be plugged in anywhere so possibly unfinished?
  • https://github.com/SpirentOrion/osv/blob/master/include/osv/cpu-steal.hh
  • https://github.com/SpirentOrion/osv/blob/master/drivers/cpu-steal.hh
  • https://github.com/SpirentOrion/osv/blob/master/drivers/cpu-steal.cc
  • https://github.com/SpirentOrion/osv/blob/master/drivers/kvmsteal.cc
  • https://github.com/SpirentOrion/osv/blob/master/drivers/vmw-steal.cc
• VMware tools driver
  • https://github.com/SpirentOrion/osv/blob/master/drivers/vmw-tools.cc
• Add VMware hypervisor detection
• various percpu_xmit changes: needs investigating what exact problems it is trying to solve
  • percpu_xmit: lower per-cpu tx queue size from 4096 to 512 - consider
  • percpu_xmit: synchronization fixes
• bsd: add SCM_TIMESTAMP support to sockets - this one might be on ipv6 branch
• Added support for IP_PKTINFO, IPV6_PKTINFO - how useful is this feature?
• fs: assign unpartioned disk space to a partion on boot
• various route-cache fixes/enhancements:
  • Add storage in the cache for the rt_gateway data from BSD's rtentry objects
  • Set RT_NORTREF to prevent the RO_RTFREE from attempting to free the rtentry object if it came from the cache
  • Handle case if we're send to our non-loopback address and a cache entry gets added for it - fixed by a0251df210a6739830bd37e52211268f5672a27c
• XEN: backport FreeBSD Xen 4.2.1 and related patches; assessing and testing it requires some XEN experience and necessary setup; also considering the latest release of XEN is 4.15, is it worth porting these patches?
  • backport FreeBSD Xen 4.2.1 interface update
  • backport blkfront driver updates
  • BSD malloc: Update behavior of M_XENBLOCKFRONT arena
  • BSD bus DMA: add bounce page support
  • add missing files from the previous commit
  • explicitly disable blkfront indirect segments
  • fix calculation in blkfront driver
  • bus DMA: add deferred bounce pages
  • fix inverted thaw logic in blkfront driver