Resources / Literature review?

[deliciouslytyped]

i'm not well versed in linkers, and I'm having a hard time learning more about them. Pardon me if I commit some faux pas in the next few issues.

I'm familiar with the Linkers and loaders book by John R. Levine, but that's all the domain specific material I know. This book is 20 years old at this point, surely there have been developments in linkers since then? From what little I've looked at linker manual pages, they've been unclear at best. (but perhaps the fault lies in me.)

Do you have any (other) recommended reading material? Are you perhaps able to release the bibliography for your thesis early, if you have anything meaningful there? - or do you already have everything you need in your head and citations are just going to be a formality, if you even need any?

Do you have any other related advice, or is the only way to really figure this stuff out currently, to write your own linker, or at least get neck deep in the code?

It's always nice to have high quality notes about design.

TODO:

• [ ] Do some more of my own literature search, maybe trying to find things that cite the L&L book.

[rui314]

Linker is indeed an area that needs more good documentations. "Linkers & Loaders" gives an overview as to what linker is doing, but it lacks enough details to precisely understand how real Unix linker works. And details matter here, because if you for example forget to implement one feature, it often results in a mysterious crash of a generated executable rather than a friendly error message reported by the loader.

I'm writing a paper about mold, so stay tuned.

I found this is a good read. https://www.repository.cam.ac.uk/bitstream/handle/1810/260771/p607-kell.pdf The paper attempts to formalize the link process. I guess you don't actually want to understand the linker at that level (at least I don't need a formal proof), but still the paper seems a good starting point to understand how actual linker works.

[deliciouslytyped]

Thanks, that looks good, and some of the references at the end might also help. I'll look around to see if the paper has any interesting reverse-dependencies later.

[MattPD]

FWIW, perhaps some of these may be of use: https://github.com/MattPD/cpplinks/blob/master/executables.linking_loading.md

[stefanos82]

I came here to ask more or less the same question, but you people have answered my question(s), especially @MattPD's link came just in time.

@rui314 Do you have any email I could keep in touch with you? I have some questions that I would like to ask in private, if possible.

[deliciouslytyped]

This is exactly why I asked this publicly, and oh wow @MattPD . Super cool when people keep bibliographies. Maybe rui will find a key paper one week before needing to publish the thesis. :P

[stefanos82]

@rui314 regards to the use of fork, I have read an article you might be interested in studying it https://www.microsoft.com/en-us/research/uploads/prod/2019/04/fork-hotos19.pdf

It basically suggests to deprecate fork and choose either posix_spawn() on any POSIX-compliant operating system or CreateProcess() on Windows.

[rui314]

@stefanos82 Here is my email: rui314@gmail.com

As to fork, my linker does not depend on it too much. If necessary, mold can spawn itself as a child process instead of forking. I kind of agree that fork might not be an ideal mechanism to create a new process, and I believe if we redesign Unix, fork would be replaced by something else. But the system call is there for 50 years and will stay here until Unix dies out.

[matu3ba]

It basically suggests to deprecate fork and choose either posix_spawn() on any POSIX-compliant operating system or CreateProcess() on Windows.

posix_spawn has the same class of issues: https://github.com/ziglang/zig/commit/22690efcc2378222503cb8aaad26a6f4a539f5aa, explained here https://github.com/golang/go/issues/22315 with these necessary hacks workarounds: https://github.com/pantsbuild/pants/issues/10507