Testing for small code model

[Quuxplusone]

Bugzilla Link	PR24159
Status	NEW
Importance	P normal
Reported by	listmail@philipreames.com
Reported on	2015-07-16 15:49:26 -0700
Last modified on	2020-11-17 23:05:54 -0800
Version	trunk
Hardware	PC Linux
CC	lhames@gmail.com, listmail@philipreames.com, llvm-bugs@lists.llvm.org, sanjoy@playingwithpointers.com, stefan.graenitz@gmail.com
Fixed by commit(s)
Attachments
Blocks
Blocked by
See also

In 233722, Lang removed our existing tests for the small code model with MCJIT
because they were entirely non-deterministic.  We should resurrect these tests
to ensure we have test coverage of the small code model as used through
MCJIT/OrcJIT, but do so, we need to fix the non-determinism issue first.

Lang explained the issue in his submission comment as:
"These regression tests are supposed to test small code model support, but have
been XFAIL'd because we don't have an in-tree memory manager that can guarantee
a small-code-model compatible memory layout. Unfortunately, they can
occasionally pass if they get lucky with memory allocation, causing unexpected
passes on the bots. That's not very helpful.

I'm going to remove these until we have the infrastructure (small-code-model
compatible memory manager) to run them properly."

In email, he expanded:
"If we add a simple custom memory manager to llvm-rtdyld that's compatible with
the small memory model (maybe just reserve ~1Mb up front? We'll never need more
than that for regression tests). Then we can add a flag to chose that memory
manager and tests specifically for small-code-model relocations."

Having a pluggable memory manager with a fixed allocation area for small test
cases seems like a generally useful construct.

[Quuxplusone]

This doesn't sound impossible to do. Do we expect any conceptual issues? IIUC
this is mostly a memory management issue. The old tests have been removed with
this commit:
https://reviews.llvm.org/rL233722

I remember reading about OrcV1 incompatibilities with the small code model, but
cannot recall the details. Are there extra show-stoppers in the particular JIT
engines? (MCJIT, OrcV1, OrcV2)

Same question for the linkers/loaders: does this affect RuntimeDyld the same
way as JITLink?

Can we roughly summarize the steps it would take to bring back small code model
tests? Thanks!

[Quuxplusone]

I've got a better handle on this now than I used to. Let me see if I can
summarize the state of things.

Background:

Relocations in small code model can be broken into two groups.

(1) Those that *require* that symbols be allocated within constrained ranges.
For example, the MachO SUBTRACTOR relocations can express the 32-bit
displacement between two global symbols:

A:
  .long B - A

The result of a 32-bit subtractor is only guaranteed to fit the storage if A
and B are guaranteed to be allocated within +/- 2Gb of one another.

(2) Relocations that can be made to work even if the target is allocated at
arbitrary address. For example, a 32-bit branch relocation to an external
symbol: If the target is out of range a jump-stub can be used to reach the
target anywhere in the address space.

Relocations of type (1) are equally "supported" in both RuntimeDyld and
JITLink: They will work if the user-supplied allocator can guarantee that
symbols remain in-range, and may fail otherwise.

Relocations of type (2) *can* be supported in RuntimeDyld, but I would argue
that JITLink is better at it (by design): RuntimeDyld's limited notion of
"stubs" makes it difficult to mix, for instance, GOTs and PLTs. JITLink's
notion of an extensible AtomGraph makes it easy to add both.

So:

In general we want to support (and test) relocations of both kinds. Relocations
of type (2) come up everywhere (and are more or less unavoidable), relocations
of type (1) are rarer, but do come up in use-cases that we care about (e.g.
Swift metadata).

We can write relocation (non-execution) tests (i.e. rtdyld-check, jitlink-check
tests) already: The respective testers simulate allocation starting
contiguously from low memory (>4k) by default. So we can write (and in JITLink,
have already written) tests for each of the small code model relocations for
the target that verify that bits are fixed up as expected.

If we want execution tests too, we have two options, (A) and (B):

(A) Test using object files.

  This has the advantage of allowing us to specify precise, minimal test cases, since we're not relying on codegen to produce the relocations that we are interested in.

  (i) Test using llvm-rtdyld: Davide Italiano already wrote a small-code-model compatible allocator for llvm-rtdyld. One problem we will encounter is that for several formats/architectures (e.g. MachO/x86-64) RuntimeDyld *doesn't* fully support the small code model yet, and would require some work to bring up to speed.

  (ii) Test using llvm-jitlink: Davide's allocator could be ported to JITLink, at which point (at least notionally) we can run arbitrary execution tests, since the small code model is already supported. The downside is that (so far) JITLink is MachO/x86-64 only.

  Neither (i) and (ii) would allow us to turn the LLVM IR test cases back on.

(B) Test using IR.

  If Davide's allocator were ported to lli we could notionally turn the old, small-code-model tests back on. However, lli is using RuntimeDyld by default so we will hit the same limitation described above: RuntimeDyld does not fully support the small code model on all platforms, and getting it to support the small code model may take some work.

Since the old tests must (presumably) have passed at some point, I guess they
must not hit the small code model relocations that RuntimeDyld doesn't not
support. If turning the old tests back on were a priority I would take path (B)
and hope for the best. This approach seems clunky though.

As things stand, my focus is going to be getting JITLink up and running for
more targets, and I think approach (A.ii) is the best one.

[Quuxplusone]

Minor update: llvm-jitlink now has a -slab-allocate to support small code model. The MachO/x86-64 relocation test tests all relocations including those used in the small code model.

I'm going to leave this open for now to track other formats/architectures and re-enabling of the lli small code model tests.

[Quuxplusone]

From what I see in JITLink, small code model should work with the Macho/arm64 backend as well. I just ran an ad-hoc test on TOT that supports the assumption. However, it's quite hacky as I don't have a proper hardware at hand.

@Lang can you confirm that? Anything I should take care of? Thanks

[Quuxplusone]

JITLink supports small code model for both arm64 and x86-64