Linking VKernel into the loader

[minexew]

I don't know why I hadn't realized this sooner. The kernel BIN file can be quite easily converted to an ELF object and appropriate thunks automatically generated to translate between calling conventions!

Since in the TOS calling convention arguments go on the stack, but the callee has to clean them up, the converter will need to be aware of function prototypes, or at least the number of arguments. Slightly annoying, but solvable (pycparser + .h file ?)

In the other direction, well, we really only care about KMain, so we can even hard-code the number of arguments there (at least for now; maybe simulating interrupts by host->guest calls will be useful for interactive mode).

With all of this done, you take the resultant object + thunks and statically link it with the rest of the loader. Another advantage? VKernel can directly import host native functions like time and socket. (Though this doesn't translate so easily to JIT programs.)

In rare occasions, the kernel imports symbols that it itself exports -- the GNU linker might or might not appreciate this. Easy, the converter can just pre-link these. A bit more inconvenient is the fact that the kernel's imports are only fully resolved after the compiler is loaded quite late in the boot process. At that point, the kernel would scan its own binary header and apply the necessary patching. In our case the binary header would be gone, so this would require some adjustments in KLoad.HC / LoadKernel. Alternatively, – call me crazy – we convert the compiler in the exact same way, and link it statically as well.

The VKernel hardcodes its load address in a few places right now, but in general, it's supposed to be fully relocatable.

Opinions?

[minexew]

A few more thoughts:

• LoadKernel is also used to expose kernel functions to the JIT world; this needs to be worked around as well. We can use the same mechanism that would be used to expose Host* functions to JIT: a table of names + addresses to be pre-loaded into the symbol table.
• It is not true that the kernel is fully resolved after loading the compiler. It also imports DocSave, DocUnlock and others from Adam. Eugh. We can probably ignore those for some time.
• Converting to ELF will also mean that we finally get symbols in GDB. Hell yeah!!

[minexew]

Ok, it is certainly possible. The question remains whether it is desirable :)

[tos11]

Having symbols in GDB is definitely desirable so the more we can statically link the better. Maybe don't worry about the linking the compiler now since there is an interrelationship between it and the Kernel? Perhalps instead it would be more interesting to end users to be able to have symbols for both the Kernel and their own HolyC code they wish to run on Linux.

JIT would be nice to have, but really I think the coolest things to do first would be these:

1) Use this to create a HolyC compiler that outputs debuggable statically linked ELF files (which has symbols both for the Kernel and users compiled code).

2) Create some HolyC header files for useful host library functions and extend the compiler from 1 to show it can produce ELFs which can call both TOS Kernel and musl library functions.

I think just achieving 1 would blow people's minds, 2 would be sheer awesomeness.

[minexew]

Indeed, this should open some new possibilities. Do you mean to statically link the user BINs with "VKernel" (really at this point it's becoming more of a TempleOS runtime library) + the current "loader" code that sets up things like Physfs and trap handlers?

Another possible direction would be to resume your work on porting the Kernel functions to C, and use that for a leaner result.

Keep in mind that either way you won't have Adam.

I will definitely push all the new code this week, so you will be able to play around ;)

[tos11]

Yes, that is exactly what I was thinking. Being able to use it as a runtime would be great.

Technically they already can use it as a runtime by just adding their stuff to KMain I guess, but there would be something more elegant in keeping it in a separate BIN file, linking it and having the debugging symbols.

The bonus of having the separate BIN file would be you can show the same BIN file is able to be loaded and run on vanilla Temple OS (assuming is uses no host specific functions).

[minexew]

Merged into master, now referred to as "static mode".

When built on Ubuntu, the program fails to load (crashes before even entering main()). I guess it has something to do with the custom linker script and not-quite-compatible GCC across Fedora & ubuntu.

[minexew]

-Wl,-z,norelro was the fix... I guess this in combination with '-omagic' ensures that all of .text really is writable