s117
commented
4 years ago It turns out the issue is caused by the incorrect page size setting in the linker:
Anycore system uses an 8K page. However, the new toolchain assumes the target machine uses a 4K page. The difference causes the linker may arranging two program segments to the same 8K virtual page, e.g. a part of the text segment and data segment may alias into the same VM page.
Because the ELF loader in the Proxy Kernel essentially calls mmap() to establish a virtual page to file mapping, as directed by the ELF program header (the actual data loading happens in the page fault handler). A program won't load correctly if two segments share a VM page: the shared page will be mapped twice, and only the second mapping will be preserved.
By default, the loader processes the text segment before the data segment (due to their default order in the ELF program header). When processing the data segment, it will 1. unmap the shared page (previously mapped to the text segment); 2. map the page as a data segment; 3. zero out the region before the data segment, which belongs to the text segment. That's why we observed some instructions is replaced by zeros in memory dump.
The solution is simple, patch the binutils to use 8K page. By commit 3b01b93.
.
Some SPEC2006 benchmarks compiled from the new toolchain failed with an illegal instruction exception. The PC triggering the exception is a legal address, but the memory dump shows codes there are replaced by zeros. If compile the same code using the old toolchain, it runs perfectly.