Extend the decompiler to produce python code

[gpdaniels]

The ideal end goal for the decompiler would be to make it produce micropython code that could be run in the place of the decompiled mpy file.

[mohankrr]

@gpdaniels @pitust i tried running the decompiler against the latest version of the robot inventor file system files in this repo..

Eg: python decompile.py "d:/GitHub/spike-prime/filesystem/storm - v1.0.06.0034-b0c335b - 2.1.4.13/_api/init.mpy" decompiled/init.py

The output was like this.. is this the expected output? Am i doing something wrong?

<module>:
    129
    str LightMatrix
    tuple 1
    import.nm lightmatrix
    import.from LightMatrix
    st.name LightMatrix
    pop
    129
    str Button
    tuple 1
    import.nm button
    import.from Button
    st.name Button
    pop
    129
    str StatusLight
    tuple 1
    import.nm statuslight
    import.from StatusLight
    st.name StatusLight
    pop
    129
    str ForceSensor
    tuple 1
    import.nm forcesensor
    import.from ForceSensor
    st.name ForceSensor
    pop
    129
    str MotionSensor
    tuple 1
    import.nm motionsensor
    import.from MotionSensor
    st.name MotionSensor
    pop
    129
    str Speaker
    tuple 1
    import.nm speaker
    import.from Speaker
    st.name Speaker
    pop
    129
    str ColorSensor
    tuple 1
    import.nm colorsensor
    import.from ColorSensor
    st.name ColorSensor
    pop
    129
    str App
    tuple 1
    import.nm app
    import.from App
    st.name App
    pop
    129
    str DistanceSensor
    tuple 1
    import.nm distancesensor
    import.from DistanceSensor
    st.name DistanceSensor
    pop
    129
    str Motor
    tuple 1
    import.nm motor
    import.from Motor
    st.name Motor
    pop
    129
    str MotorPair
    tuple 1
    import.nm motorpair
    import.from MotorPair
    st.name MotorPair
    pop
    none
    ret

[gpdaniels]

At the moment yes that is the expected output. Hopefully @pitust will be able to explain more.

[pitust]

Yeah, it doesn't produce python code. Or rather, the currently published version doesn't. I have something that produces something like this:

from lightmatrix import LightMatrix 
from button import Button 
from statuslight import StatusLight 
from forcesensor import ForceSensor 
from motionsensor import MotionSensor 
from speaker import Speaker 
from colorsensor import ColorSensor 
from app import App 
from distancesensor import DistanceSensor 
from motor import Motor 
from motorpair import MotorPair 
return None

which is pretty nice. Unfortunatly, sometimes it produces absolute trash, like this:

# init/app.py
from util.constants import BT_VCP, USB_VCP 
from protocol.ujsonrpc import JSONRPC 
from utime import ticks_ms, ticks_diff 
_NOT_CONNECTED_ERROR = undefined
def App(*args):
    __module__ = __name__
    __qualname__ = "App"
    def __init__(*args):
        $push(BT_VCP.isconnected())
        BROKEN_OP`btrue 13`
        $push(USB_VCP.isconnected())
        BROKEN_OP`btrue 13`
        raise RuntimeError(_NOT_CONNECTED_ERROR)
        if BT_VCP.isconnected():
            if USB_VCP.isconnected():

        else:
        args[0]._json_rpc = JSONRPC(BT_VCP)
        return None
    def _play_sound(*args):
        args[7] = False
        $push(args[7])
        BROKEN_OP`mkclosure 5, 1`
        args[4] = __pop
        $push(args[0]._json_rpc.call)
        $push(undefined)
        BROKEN_OP`map 5`
        $push(args[1])
        $push("sound")
        BROKEN_OP`st.map`
        $push(min(128,, max(0, args[2])))
        $push("volume")
        BROKEN_OP`st.map`
        $push(0)
        $push("pitch")
        BROKEN_OP`st.map`
        $push(0)
        $push("pan")
        BROKEN_OP`st.map`
        $push(args[3])
        $push("wait")
        BROKEN_OP`st.map`
        args[5] = __pop(__pop, __pop, args[4])
        args[6] = ticks_ms()
        while args[7]:

        $push(args[7])
        $push(ticks_diff(ticks_ms(), args[6]) < 129,)
        BROKEN_OP`btrue 39`
        $push(args[7])
        BROKEN_OP`btrue 63`
        args[0]._json_rpc.cancel_call(args[5])
        raise RuntimeError(_NOT_CONNECTED_ERROR)
        return None
    null
    def play_sound(*args):
        $push(isinstance(args[1], str))
        BROKEN_OP`btrue 10`
        raise TypeError(undefined)
        $push(isinstance(args[2], int))
        BROKEN_OP`btrue 20`
        raise TypeError(undefined)
        $push(USB_VCP.isconnected())
        BROKEN_OP`btrue 33`
        $push(BT_VCP.isconnected())
        BROKEN_OP`btrue 33`
        raise RuntimeError(_NOT_CONNECTED_ERROR)
        args[0]._play_sound(args[1], args[2], True)
        return None
    play_sound = play_sound
    null
    def start_sound(*args):
        $push(isinstance(args[1], str))
        BROKEN_OP`btrue 10`
        raise TypeError(undefined)
        $push(isinstance(args[2], int))
        BROKEN_OP`btrue 20`
        raise TypeError(undefined)
        $push(USB_VCP.isconnected())
        BROKEN_OP`btrue 33`
        $push(BT_VCP.isconnected())
        BROKEN_OP`btrue 33`
        raise RuntimeError(_NOT_CONNECTED_ERROR)
        args[0]._play_sound(args[1], args[2], False)
        return None
    start_sound = start_sound
    return None
App = __build_class__(App, "App")
return None

The main point of this decompiler is to make code analysis easier, not to make complete python code from the binaries.

[mohankrr]

thanks.. i get the point that this is just for code analysis..

On another note, i tried running the typescript version of the decompiler.. decompile.ts ... getting unable to compile typescript with.; decompile.ts:454:7 - error TS2550: Property 'replaceAll' does not exist on type 'string'

what version of ECMAScript is this ts file targetting?

[pitust]

Haha. It's targetting ESNext, just because i can really. Run it with -T, transpileOnly (i know there is no errors in my typescript). My tsconfig is actually somewhere else and i forgot to copy it over.

[pitust]

Notice the undefined that's all over the place? That's a decompiler bug i haven't quite tracked down yet. Tha, and the BROKEN_OP things prevent me from publishing the whole thing. Even the disassembler online is out-of-date compared to my local version, but i will merge this once i'm positive it can produce nice python code.

[pitust]

Additionally there are actually argument names that can be recoverd which imho would be awesome for the decompiler.

[gpdaniels]

@pitust did you ever manage to track down all your decompiler bugs and have it produce somewhat readable python code?

[gpdaniels]

@pitust I've been trying to port the decompiler to python and I'm hitting some odd issues.

Above you mentioned:

Even the disassembler online is out-of-date compared to my local version, but i will merge this once i'm positive it can produce nice python code.

Was it designed to work with a different version of the disassember?

[gpdaniels]

First version of the python port is done: https://github.com/gpdaniels/spike-prime/commit/b6458da46a65ce3b56c851170415967eb51be9e5