Incremental takeover

[pgoodman]

By default, Granary operates as a JIT. In theory, Granary can also operate as a load-time binary instrumentation system as well. I think there is a third option, which would be an incremental takeover. This option would only be well-defined for cases where Granary has control of specific program entrpoints that are continuously re-executed (e.g. system calls in the kernel).

Here's the idea:

1. A program entrypoint it enqueued to a worker thread.
2. The worker thread dequeues the instrumentation request, and instruments a trace based on the requested PC.
3. The worker then replaces the entrypoint with the trace pc.
4. The next execution of the entrypoint goes through the trace.
5. Execution eventually reaches a trace exit (lets only handle direct edges for now, JIT the indirect egdes). The direct edge handler enqueues the requested PC and direct edge pointer as a new entrypoint, then replaces the direct edge target pc pointer with the native target.
6. Go to 1.

This would have the interesting property that execution gets laid out nicely, but the overhead of translating would be isolated to a worker thread.

[pgoodman]

I think this is a really insightful email about HHVM's code gen approach, and I think it could apply really well to the proposed incremental takeover: http://article.gmane.org/gmane.comp.compilers.llvm.devel/78971

The idea being: collect profiling data, potentially for multiple targets, and then put their associated code into a one of the hot, cold, or frozen code regions.

[pgoodman]

Another related thing to the hot / cold / frozen code is general placement of fragments. It would be interesting if instrumented fragments could be placed into a colder region than app fragments, at least in the case where those instrumented fragments are predicated. Similarly, it might be interesting to allow for one to add an @unlikely annotation to conditional jumps in inline assembly.