Rework the Java bindings

[nneonneo]

This is a complete rewrite of the Java bindings for Unicorn, focused on speed, correctness and feature parity.

The existing bindings have some significant shortcomings. An incomplete list:

• Hooks don't actually work properly. All hooks are bound to all addresses, causing all hooks to fire at all addresses regardless of the registered bounds. Hook implementation is very slow due to lots of redundant work.
• reg_read and reg_write are very slow for the common use case (i.e. reading long-sized registers or smaller), as they have to box and unbox Longs.
• Bugs! reg_read returns stack garbage in the high bits because the read value isn't zeroed. The entire JVM crashes if you try and read a vector register.
• Context allocation is a raw port of the C API, demanding manual memory management and providing no access to context registers.
• It hasn't been updated since pre-2.0, and lacks many new APIs.

This PR contributes a ground-up rewrite of the bindings. It brings the Java API up to par with Python feature-wise and substantially simplifies the hook implementation, enabling proper bounds-checked hooks.

The new implementation is much faster than the old one. As a point of comparison, the following code hook, executed 10,000,000 times on an x86 loop instruction, takes 9.9s with the old bindings, but just 1.6s in the new bindings - a 6x performance improvement. A similar hook takes 38s to run with the Python bindings.

    public static class MyCodeHook implements CodeHook {
        public long accum;
        public void hook(Unicorn uc, long address, int size, Object user_data) {
            // need to cast to int here b/c old bindings don't properly clear upper bits of value
            accum += (int)(long)uc.reg_read(Unicorn.UC_X86_REG_ECX);
        }
    }

The rewrite strives for compatibility with the previous API, but there are some breaking changes. It is possible to push closer to full backwards compatibility if required, at the cost of reintroducing some of the suboptimal designs. Here are the main points of breakage:

• ReadHook and WriteHook are gone, replaced simply by MemHook. Hooking valid memory accesses now requires a type parameter. This enables fetch and read-after hooks with a unified API and a single callback object.
• mem_read now takes an int, not a long. We are unable to allocate more than 2GB in a single request anyway (Java limitation).
• Instruction hooks now require specifying the instruction explicitly, instead of guessing based on the hook type. This is necessary to distinguish sysenter/syscall and ARM64 mrs/msr/sys/sysl, without excessively bloating the library with redundant hook types. Bounds must also be specified, to support bounds-checked instruction hooks.
• Reading object-type registers (any register larger than 64 bits, or registers with special formats) requires a second argument to reg_read. This allows us to provide a fast reg_read that returns a long for the common cases, while still supporting a more general reg_read for other registers.
• mem_map_ptr is rewritten to take a direct java.nio.Buffer, which enables many more use cases than a simple byte array, and improves performance (a byte array cannot really be used as a mapped buffer without GC-pinning it, which hurts the GC performance).
• Context handling API is redesigned to be safer and more object-oriented.

A lot of bugs are fixed with this implementation:

• Unicorn instances can be properly garbage-collected, instead of hanging around forever in the Unicorn.unicorns table.
• Hooks no longer fire outside of their bounds (#1164), and in fact, hook bounds are properly respected (previously, all hooks were just registered globally to all addresses).
• Hooks are substantially faster, as they are now dispatched directly via a single method call rather than being indirected through invokeCallbacks.
• Loading vector registers works now, rather than crashing the VM (#1539).

Several features are now enabled in the Java implementation:

• All of the current ctl_* calls are implemented.
• mmio_map is implemented.
• New virtual TLB mode is implemented.
• reading/writing Context registers is implemented.
• New hook types are added: TcgOpcodeHook, EdgeGeneratedHook, InvalidInstructionHook, TlbFillHook, and the instruction hooks Arm64SysHook, CpuidHook.
• All known special registers are supported.

Detailed list of backwards incompatible changes

• EventMemHook: added type parameter
• Eliminated ReadHook and WriteHook and replaced both with MemHook. type added to both hooks, value added to read hook.
• Unicorn:
  • Removed public long eng
  • query(int) returns long now, instead of int
  • reg_read(int) returns long now, instead of Object (Object return replaced by reg_read(int, Object))
  • hook_add(ReadHook, long, long, Object) removed
  • hook_add(WriteHook, long, long, Object) removed
  • hook_add(MemHook, long, long, Object) takes a new int type param
  • All hook_add methods now return a long instead of void
  • mem_map_ptr changed to take long, Buffer, int instead of long, long, int, byte[]

[nneonneo]

By the way, if backwards-compatibility will be an issue, it should be possible to restore most of the old APIs. However, the resulting design may be less clean.

[wtdcode]

By the way, if backwards-compatibility will be an issue, it should be possible to restore most of the old APIs. However, the resulting design may be less clean.

Current breaking changes look good to me at least. I was not responding to this PR because of sickness recently. Will do a thorough check after a week or so.

[nneonneo]

OK, at this point I'm fairly happy with the code. It's decently well-tested, the samples match the C code (and the output matches too), and the performance is good. It's ready to be reviewed.

[wtdcode]

OK, at this point I'm fairly happy with the code. It's decently well-tested, the samples match the C code (and the output matches too), and the performance is good. It's ready to be reviewed.

Thanks for your work and I will review it these days as finally I feel a bit better! By the way, could you illustrate your use case?

[nneonneo]

I am planning to integrate Unicorn into some Ghidra-based analysis tools. As Ghidra is written entirely in Java, it is much more efficient to use Java bindings than trying to go through Jython-ctypes.

[wtdcode]

I am planning to integrate Unicorn into some Ghidra-based analysis tools. As Ghidra is written entirely in Java, it is much more efficient to use Java bindings than trying to go through Jython-ctypes.

That's cool! At this moment I don't have too much to say about this PR because I haven't written java for a few months. I need to give it a try before giving more review but thanks again for your brilliant work!

[nneonneo]

Any news on this?

[wtdcode]

Any news on this?

Rushing on a conference deadline ;(

Will have a look once getting it done.

[wtdcode]

All break changes look good to me as Java binding is too old and buggy.

I have no big questions on this PR and thanks for this brilliant work!

[wtdcode]

I will be soon on traveling so please expect another a few days absent. I will get back asap. Thanks for your patience. ;)

[nneonneo]

No worries. I'll aim to get something working with Maven this week, and also look at integrating the (newly merged) reg2 API :)

[nneonneo]

OK, the bindings are migrated to use Maven. The last thing on my TODO list is to switch to the reg2 API, which should be pretty straightforward.

[nneonneo]

OK, that last commit (763d041) adds the reg2 API, which I think is the last issue that needed to be resolved. Should be good for a final review and hopefully merge :)

[wtdcode]

Some last things which probably need your help:

1. I read through this: https://stackoverflow.com/questions/2937406/how-to-bundle-a-native-library-and-a-jni-library-inside-a-jar and https://github.com/adamheinrich/native-utils. Since we plan to push the package to maven central, we would like to bundle dynamic libraries into jar and load it in the static code block of the Unicorn class. We prefer this approach for all bindings for ease of usage.
2. javah seems outdated and could you replace it with javac -h? I'm not 100% sure on this so I'm glad if you could help.
3. I'm thinking of removing Makefile totally so that we don't rely on make, which should enable it to compile on Windows. I see exec plugin allows us to execute any command and thus we can just compile unicorn libraries in pom.xml like other bindings did. This actually relates to my point 1 I believe.

At this moment, I'm experimenting with 1 and 3 to build dynamic libraries within pom.xml and bundle it to jar and load it in unicorn class. Could you have a look at 2?

[nneonneo]

@wtdcode OK, it was easy to switch to javac -h so I have done so. I also made a fix to const_generator to only generate new data if there are changes, which prevents mvn from having to rebuild everything every time.

[nneonneo]

Tests are failing, but I'm not sure it's my fault? The new test test_x86_0xff_lcall fails due to invalid instruction, which makes sense since the patch marks certain instructions as being invalid. Should the test itself be fixed?

[wtdcode]

Ah sorry you are correct, the test should be fixed. xd

[wtdcode]

I have mostly addressed the problem of building the binaries and now I'm investigating how to package everything together.

Meanwhile, I sent a request to Maven Central here:

https://issues.sonatype.org/browse/OSSRH-93144

[nneonneo]

@wtdcode its been a while! Just checking: is there any plan to get this merged? Maybe we can merge it first and then worry about the build issues? It sounded like the build system itself was basically sorted out and we just wanted to get the package on Maven (which I fully support!).

Merry Christmas and a happy new year!

[wtdcode]

@wtdcode its been a while! Just checking: is there any plan to get this merged? Maybe we can merge it first and then worry about the build issues? It sounded like the build system itself was basically sorted out and we just wanted to get the package on Maven (which I fully support!).

Merry Christmas and a happy new year!

Sure, I think that's a good start.

Regarding the build system, the reason why I failed to get it to work is that I lost all my progress (including some other work related to other issues) when I migrated my data, sorry.