beardypig / ghidra-emotionengine

Ghidra Processor for the Play Station 2's Emotion Engine MIPS based CPU
Apache License 2.0
198 stars 35 forks source link

VU decompilation seems unsound #77

Open chaoticgd opened 2 years ago

chaoticgd commented 2 years ago

Below I will attempt to argue that the current approach to decompiling VU code is detrimental and the P-Code implementation of all VU instructions should be replaced with pcodeop stubs (maybe, see below). Overall I don't find what the decompiler emits currently to be useful, although I admit that my workflow may be different to the workflow of others. As such, I want anyone who's worked with VU code in the past to weigh in below. I want to get as many opinions as I can to get more of a representative sample.

In micro mode:

In macro mode:

On a side note, a lot of the MMI instructions have a similar issue in that the p-code as written produces a lot of unnecessary branches for things like clamping. I think in this case the solution could be to replace clamp operations with pcodeop stubs as well.

I've been using a modified version of ghidra-emotionengine that removes support for micromode, and replaces the implementation of all VU instructions with pcodeop stubs (basically it reverts a bunch of stuff to how it was in PR#23). I expect this would not be a good solution for merging into master as it removes some features, although I find this quite useful. It seems to be more correct and it's certainly closer to what the original code would have looked like.

I suppose the main counterargument to all of this would be that we have the disassembly, so this is at most a minor inconvenience. This is also why it took so long to file this issue, it's not exactly a deal breaker for reversing VU code. Still, my position is that correctness should be favored in this case. I don't see the point of doing all this fancy analysis if there's no guarantee that the result of said analysis is correct.

astrelsky commented 2 years ago

I agree. It is quite broken and I'm not even sure why I submitted it along with the rest of the vu stuff.

I always confuse myself with the vu modes. The vu instructions mixed in on the EE side are working correctly right?

chaoticgd commented 2 years ago

Now I think about it I can't think of an example where pure VU macro instructions on the EE core-side decompile to nothing when there is not a microprogram involved. I was probably thinking of cases where they were trying to transfer data to/from a microprogram, in which case it is broken on the EE core-side and VU macro instructions typically aren't involved (but lqc2/sqc2/qmtc/qmfc/ctc2 are).

As to whether the code for the macro instructions is technically correct I'm not sure. There's a lot of output with p-code op business that I'm not sure about.