Register spilling in OMR

[amitin]

It’s not quite clear how register spilling works in OMR. Does it always use machine stack slots? Is it possible to get registers spilled using a custom base address for spill slots?

[ymanton]

Short answer: yes, always to machine stack slots. If you want to change that you will have to change the code generator(s) you are targeting. If you want to dump some/all registers at specific points (e.g. during a transition out of JIT compiled code and into something else) you can do that sort of thing, we have examples of that downstream. If you want something more elaborate, e.g. the register allocator generating spills to your custom location during its pass, that's probably more involved.

[amitin]

The main goal which we want to achieve is using a machine stack pointer register (%esp on x86) as VM's stack pointer in our JIT-compiled code. This could allow to use push/pop instructions instead of indirect mem access instructions for VM stack operations and get an extra GPR free - this GPR is currently responsible to maintain VM's stack pointer.

But you are correct, in particular we need to dump some registers during transitions out of JIT compiled code. For instance, we need to save VM's 'sp' register (and free its GPR) to VMThread::sp field before returning to interpreter or before a ffi call. Could you please point me to examples you mentioned above?

[ymanton]

The main goal which we want to achieve is using a machine stack pointer register (%esp on x86) as VM's stack pointer in our JIT-compiled code.

OK, so does that mean that you want all other memory accesses unrelated to the program you're running (e.g. register spills) to go elsewhere (i.e. the custom base address you mentioned earlier)? Or do you simply want everything to go elsewhere (i.e. to your VM's stack)?

Having everything go to your VM's stack is easy, the compiler doesn't care what the stack pointer points to, whether it's the system stack or another blob of memory, it will simply use it like a stack. You can have SP point to whatever you want before jumping to code compiled by the OMR compiler and it will use it like a C stack (meaning it will generate a prologue that allocates a stack frame, save preserved regs, etc). If you want to change some or all of that you can define your own linkage, which will allow you to generate your own prologues and epilogues and do whatever setup you want.

If you want "your stuff" to go to a the VM stack (pointed to my the machine SP) and "compiler stuff" (spills and whatnot) go to another stack (either pointed to by another register or accessed indirectly) that would be more involved.

But you are correct, in particular we need to dump some registers during transitions out of JIT compiled code. For instance, we need to save VM's 'sp' register (and free its GPR) to VMThread::sp field before returning to interpreter or before a ffi call. Could you please point me to examples you mentioned above?

Examples of this sort of thing will be downstream in https://github.com/eclipse/openj9 any day now once it goes live (and we actually changed the way this scheme works recently in J9), but I can describe the mechanics. If you want to do something before calls to certain functions (FFs, your interpreter, etc) you define a linkage that will be responsible for generating code to perform those calls. In that code you can dump any/all of the machine registers to wherever you want; you can dump registers blindly or only if they are live, up to you. Alternatively, you can dump nothing and call an assembly routine that can dump whatever you want. In J9 we used to use the latter scheme, the JIT compiler assumed that the register state before/after calls to the VM would be identical (i.e. the VM routine being called would handle all register dumping/restoring), so the JIT compiler wouldn't generate spills around such calls at all. The JIT code generator would simply retrieve the function's linkage object and ask it to generate the call, and the linkage object for VM functions would do what I said above, where as the conventional linkage would save volatile registers before the call and restore them after the call, and so on. If you want to do something special before returning you can do that using a customized epilogue. If you want to do something special on entry into a compiled method you can do that with a customized prologue.

So I think long story short you may want to look into defining some new linkages for yourself; that will allow you to control what happens in prologues, epilogues, and around calls. As far as the compiler is concerned every method, including the one being compiled, has a linkage object associated with it. Prologues and epilogues are generated by the compiled method's linkage, calls are generated by the linkage of the method being called.