java-deobfuscator / deobfuscator

The real deal
https://javadeobfuscator.com
Apache License 2.0
1.59k stars 295 forks source link

ConstantFolder - Long support #770

Closed analtevs closed 3 years ago

analtevs commented 3 years ago

Needed support for longs in the ConstantFolder transformer (com.javadeobfuscator.deobfuscator.transformers.general.peephole)

This is my suggestion as it worked for me..

this example wasn't handled correctly by the existing code:

LDC 6458377393085008524L
LDC 3104987515852103167L
LOR
BIPUSH 13
LSHL
LDC -236791030383386624L
LXOR
LDC 97L
BIPUSH 21
LUSHR
LDC 8147424378139406440L
LAND
BIPUSH 13
LSHL
BIPUSH 12
LSHL
LDC 100L
LXOR

example code supporting the long variant(s):

                            case LADD:
                            case LAND:
                            case LDIV:
                            case LMUL:
                            case LOR:
                            case LREM:
                            case LSHL:
                            case LSHR:
                            case LSUB:
                            case LUSHR:
                            case LXOR: {
                                List<Frame> frames = result.getFrames().get(ain);
                                if (frames == null) {
                                    break;
                                }
                                Set<Long> results = new HashSet<>();
                                for (Frame frame0 : frames) {
                                    MathFrame frame = (MathFrame) frame0;
                                    if (frame.getTargets().size() != 2) {
                                        throw new RuntimeException("weird: " + frame);
                                    }
                                    Frame top = frame.getTargets().get(0);
                                    Frame bottom = frame.getTargets().get(1);
                                    if (top instanceof LdcFrame && bottom instanceof LdcFrame) {
                                        long bottomValue = ((Number) ((LdcFrame) bottom).getConstant()).longValue();
                                        long topValue = ((Number) ((LdcFrame) top).getConstant()).longValue();
                                        if (ain.getOpcode() == LADD) {
                                            results.add(bottomValue + topValue);
                                        } else if (ain.getOpcode() == LMUL) {
                                            results.add(bottomValue * topValue);
                                        } else if (ain.getOpcode() == LREM) {
                                            results.add(bottomValue % topValue);
                                        } else if (ain.getOpcode() == LSUB) {
                                            results.add(bottomValue - topValue);
                                        } else if (ain.getOpcode() == LDIV) {
                                            results.add(bottomValue / topValue);
                                        } else if (ain.getOpcode() == LSHL) {
                                            results.add(bottomValue << topValue);
                                        } else if (ain.getOpcode() == LSHR) {
                                            results.add(bottomValue >> topValue);
                                        } else if (ain.getOpcode() == LUSHR) {
                                            results.add(bottomValue >>> topValue);
                                        } else if (ain.getOpcode() == LXOR) {
                                            results.add(bottomValue ^ topValue);
                                        } else if (ain.getOpcode() == LOR) {
                                            results.add(bottomValue | topValue);
                                        } else if (ain.getOpcode() == LAND) {
                                            results.add(bottomValue & topValue);
                                        }
                                    } else {
                                        break opcodes;
                                    }
                                }
                                if (results.size() == 1) {
                                    InsnList replacement = new InsnList();
                                    // i think we need two of these since we're dealing with long-type values..
                                    // pop2 will remove two int(s) or one long.  we don't want that behavior.
                                    replacement.add(new InsnNode(POP)); // remove existing args from stack
                                    replacement.add(new InsnNode(POP)); // remove existing args from stack
                                    replacement.add(Utils.getLongInsn(results.iterator().next()));
                                    replacements.put(ain, replacement);
                                    folded.getAndIncrement();
                                }
                                break;
                            }
Janmm14 commented 3 years ago

Would really appreciate if you could just PR this.

However YOU NEED 1 POP2 for popping LONG from the stack instead of 2 POP. A POP may not remove parts of a LONG from the stack. This will cause verify errors and likely the deobfuscator method execution simulation will throw an error as well.

analtevs commented 3 years ago

The struggle with the single pop2 op was an unbalanced stack (at least in testing this was the case) in some cases. I think there were instances where pop2 was removing too much in situations where the final value was smaller than long.

Will try to recover that situation and post the offending byte code. but fwiw I did need to force two single pops .vs the "either or" behavior of pop2.

Janmm14 commented 3 years ago

For the opcodes LSHL, LSHR and LUSHR the second operand (in the code named bottom), actually on top of the stack is not a long, but an int. So you need POP POP2 for LSHL, LSHR and LUSHR and for the other long opcodes you need POP2 POP2

analtevs commented 3 years ago

I'll edit code and write some test cases for this. will submit pr afterwards. :-)

ThisTestUser commented 3 years ago

This has been added in the latest release.