Fix GDScript decompilation for Godot versions 2.1, 3.1, and 3.5

Godot versions 2.1 and 3.1 broke binary script compatibility in patch releases (ugh). To make matters worse, they didn't start writing the patch number to the PCK until 3.2. EVEN WORSE, Godot didn't have a version string in the executable with the patch number until version 2.1.3; previous versions just wrote "2.x.stable.\<BUILD>", and there were breakages between 2.1.1 and 2.1.2.

That left the following options for detecting the bytecode revision we needed to decompile these scripts:

We could at least implement binary file parsing for 3.1.x to get the version string from the executable, but we're not always going to be guaranteed to have access to the executable (this is something I want to do eventually anyway for custom build detection, but it's going to take a while)
We could implement dynamic detection via the detect_bytecode_ver.gd script, but that would only work if the game being tested is for the same platform that we're decompiling on. Additionally, there are many situations where we do not want to run the executable for security reasons (like, REing a potentially malicious game).

So, the only real option for these versions was to implement static testing of the bytecode.

This was accomplished by testing the following:

Token changes

We can detect token shifts by checking certain tokens that would be affected by a shift caused by an added token in a subsequent revision of the byte code. We know that for at least some tokens, there are absolute requirements as to which token comes afterwards. This is mostly implemented in the current testing by checking if TK_PARENTHESIS_OPEN follows TK_BUILT_IN_FUNC, but we test the ENUM token for revision 0xed80f45.

Built-in function changes

We know the expected argument count for all revisions. While we don't have a proper token parser to do intelligent parsing of all the arguments, we can at least count the number of arguments by simply counting commas that are not within other enclosures. Bytecode revisions prior to GDScript 2.0 did not allow commas in anything other than enclosures in function arguments; this changed in GDScript 2.0 (lambdas) but that doesn't have a compiled version yet.

If we successfully test a built-in function arg count for a function after the shift occurs, and the other candidate built-in func also has different argument counts, we can say with reasonable certainty that it is the correct revision.

Implementation:

This adds a test_bytecode() function to GDScriptDecomp, but it is only implemented for

0x514a3fb (13, 3.1.1)
0x1a36141 (13, 3.1.0)
0x1ca61a3 (13, 3.1 beta 1-5)
0xed80f45 (10, 2.1.3 - 2.1.6)
0x85585c7 (10, 2.1.2)
0x7124599 (10, 2.1.0 - 2.1.1)

It is not currently implemented on any other revision. It is unlikely that we will run into any of the dev bytecode revisions in the wild, and we have methods of successfully determining the major and minor version of the engine through the PCK header, the APK AndroidManifest.xml, and, as a last resort, checking the headers of all the binary resources in the project.

Because of the above, the pass cases for the implemented tests only take into account the release/beta bytecode revisions within the same major and minor versions. This means that we might have false positives if it's from a dev version.

This could be extended in the future to cover the dev versions, but I'd want to refactor the GDScriptDecomp classes significantly before that so that these tests can be generalized.

This also adds bytecode revision a7aad78 to gdsdecomp for Godot 3.5; there was a hanging PR to add the built-in function deep_equal that was merged in very late in the development cycle. Fortunately, it was added at the END of the built-in functions, so we at least weren't mangling any scripts from 3.5 games.

bruvzg / gdsdecomp