Strange issue between linking and virtio

This bug has me completely stumped. When we combine the composite rump kernel component, with the rump kernel itself, the ideal process is to take the COS rk component and link it with a RK application that just has a simple stub application (a main that just returns 0). In this sense, we would get a minimal object file that contains the necessary COS code to boot the RK up into a standalone component that does nothing but wait to service incoming invocations. Setting the build system to do this causes qemu itself to crash with the message: qemu-system-i386: Guest moved used index from 0 to 1965 Qemu is crashing, not our system, and there are no error messages that follow. Googling around, people seem to have received similar messages when interacting with virtio, the networking interface we also use when running on qemu.

To get around this problem, for some reason, I can modify the build process to be less "ideal" compared to what I stated above. If link together the composite rk component with the udpserver component prior to linking with the RK stub application, the problem doesn't show up when virtio is interacted with. This is a less than ideal build process because now the rumpkernel object file includes the entire text of the application which is being loaded in the final image as its own component anyway, so we get duplicated text sections and one is never used (the one in the rump kernel).

I can only image that the correlation between the build system change and this qemu bug is that the rumpkernel needs to have networking related functions linked into its object file?? That doesn't make sense to be, but I could test this hypothesis by modifying the stub application to include some socket calls.

I'm at a loss here for how to debug this further, I don't know if this is just a qemu related bug or if the RK build process is doing something complicated I don't understand.

gwsystems / composite

Strange issue between linking and virtio #340