SWIG revolución

[doegox]

Issue to track ideas, progress and tasks to

• define a unified user API, starting with sth like open/close/sendcmd
• get a C libpm3 (used also by proxmark3 client via that user API to not duplicate bins)
• expose the unified API internally to embedded Python and Lua (one day, that will replace current Lua API)
• expose the unified API externally to Python scripts

First PoC tests on https://github.com/RfidResearchGroup/proxmark3/tree/libpm3_experiments Current compilation paths to keep an eye on:

• make
• cmake
• cmake /lib (test of C libpm3)
• cmake /libpy (test of py wrapper around C libpm3)
• cmake /client_with_swig (test of "libpm3" API accessible to embedded py & lua)

Next steps:

• [x] make client cmake independent of build location (${PM3_ROOT}...)
• [x] start with decent minimal API
• [x] embedded py & lua
• [x] extend API to context to avoid global vars in shared lib ?
• [ ] Make Lua and Python looking for the real libpm3rrg_rdv4.so name and make sure the lib is in their respective path when installed / when in repo. If possible.
• [ ] convert Lua scripts to use the unified API ? provide a sed script to automate script conversion?
• [ ] extend API to be able to divide client => light client + static libpm3
• [ ] move to dynamic libpm3, ! make install)
• [ ] bring same cmake logic to Makefile
• [ ] once we got dynamic libpm3 separated from client, get python module "published" (make install & for local dev) for external python
• [ ] find ways to return output instead of direct print, #dbg msgs will probably need a callback
• [ ] get libpm3 and scripts running on PS/WSL/OSX

[doegox]

unified API is maintained in client/include/ SWIG will not be required for ppl to compile the repo. We'll take care that when updating API, we generate the SWIG artefacts and push them on the repo (currently: client/src/libpm3_luawrap.c , client/src/libpm3_pywrap.c and client/src/libpm3.py)

[iceman1001]

Yes,

We will be able to use "printandlog" fcts to catch most of pm3 output and return to the external API calls. I suppose setting a flag, we enable the "copy output" while running. Or the py / lua needs to redirect stdout,stderror to catch it.

I think we start with three simple ftcs to expose from our API to start with. No need to over do it. Lua wrap doesn't have more than a dozen api fcts hooked up. Some crc/mac/crypto. The next fourth fcts would be "sending usbcommand / receive". That should be many happy for a long time. The rest can come later if community makes an effort.

The exposed API fcts should be in a mature state. ie, not prone to change.
Even if SWIG makes it easy, I don't want to change the API all the time.

[doegox]

yes good point on API stability.

[doegox]

Thinking a bit ahead, we should have also:

• a global read-only flag to tell to the script if it's running embedded or not (easy to do)
• a way to control several proxmarks from the same script i.e. being able to call OpenProxmark several times, each having its comm thread etc. This one is more complex and will impact the API, so better to take it into account soon.

[iceman1001]

Indeed, the openproxmark, could take a string with "/dev/acm01" etc, which would give the possibility to connect to several proxmarks. Nevertheless, I think this is too far off yet. I tend to take bites of a problem and make that work.

[slurdge]

proxmark3+jupyter=💖💖💖

[iceman1001]

...looks like we got someones attention.... Where is that lz4 ;)

[doegox]

Indeed, the openproxmark, could take a string with "/dev/acm01" etc, which would give the possibility to connect to several proxmarks. Nevertheless, I think this is too far off yet. I tend to take bites of a problem and make that work.

I was not thinking of the client itself, rather the lib and API. Such that from a script you could do

import pm3
a=pm3.open("/dev/ttyACM0")
b=pm3.open("/dev/ttyACM1")
a.cmd("hw status")
b.cmd("hw status")

etc

[iceman1001]

yes, that is what I ment.

[doegox]

Here is what we have in the current experimental branch:

Embedded Lua:

local pm3 = require("pm3")
p=pm3.get_current_dev()
pm3.console(p, "hw status")

Embedded Python:

import pm3
p=pm3.get_current_dev()
pm3.console(p, "hw status")

External Lua using libpm3:

local pm3 = require("pm3")
p=pm3.open("/dev/ttyACM0")
pm3.console(p, "hw status")
pm3.close(p)

External Python using libpm3:

import pm3
p=pm3.open("/dev/ttyACM0")
pm3.console(p, "hw status")
pm3.close(p)

External C using libpm3:

#include "pm3.h"

int main(int argc, char *argv[]) {
    pm3_device *p;
    p = pm3_open("/dev/ttyACM0");
    pm3_console(p, "hw status");
    pm3_close(p);
}

Proxmark3 client and libpm3 are still two separated entities that will need to converge into a lightweight client using libpm3.

Note: it doesn't support several devices for real yet but at least the API is ready for it.

[iceman1001]

ok, after context everywhere, I give in, session -> context.

[slurdge]

For Python & LUA, shouldn't the API give an "object" that is a proxmark instance? Like this:

import pm3
with pm3.open("/dev/ttyACM0") as p:
    p.console("hw status")

I guess this is what @doegox has in mind. Also, do you want each lib (python, lua etc.) create the higher level wrappers themselves? I would think of something along the lines:

with pm3.open("/dev/ttyACM0") as p:
   p.hw.status() #gives you a string
   p.hf.search()

But it would mean SWIG is able to create objects across scripting languages... not sure about it. Its easy to create the overall wrapper but it would mean having it in sync with underlying API.

[doegox]

yes the Python wrapper would be more pythonesque the way you propose. For me the questions are:

• how far python API usage should be different from other languages?
• how to achieve it without duplicating code to maintain? It should be achievable with some swig macro, we don want to update interface file every time a command is changed or added.

Note that the console("hw status") is just the start, to have quick access to the existing fcts, in the future we should get more real functions APIs, e.g. int hw_status(void); called by CLI parser and by swig API.

[iceman1001]

Lets keep things simple. We take what SWIG offers out of the box. That keeps it easy to maintain. If pattern is the same across engine api, ppl feel acustomed. Looks the same, behavies the same.

As @doegox mentions, in later stages, exposing more and more fcts. Right now thats not the purpose of this exercise.

[doegox]

After some rework thanks to @slurdge here is what we have in the current experimental branch:

Embedded Lua:

local pm3 = require("pm3")
p=pm3.pm3()
p:console("hw status")
print(p.name)

Embedded Python:

import pm3
p=pm3.pm3()
p.console("hw status")
print("Device:", p.name)

External Lua using libpm3:

local pm3 = require("pm3")
p=pm3.pm3("/dev/ttyACM0")
p:console("hw status")
print(p.name)

External Python using libpm3:

import pm3
p=pm3.pm3("/dev/ttyACM0")
p.console("hw status")
print("Device:", p.name)

External C using libpm3:

#include "pm3.h"

int main(int argc, char *argv[]) {
    pm3 *p;
    p = pm3_open("/dev/ttyACM0");
    pm3_console(p, "hw status");
    pm3_close(p);
}

[iceman1001]

Its some nice progress.

[snowsign]

Any updates on this? It's been a few years and there still isn't a consistent Python/Lua API. Python scripts run through the client currently need to rely on the serial port, which isn't possible when running through the client due to locking.

[iceman1001]

No active work is going on. Feel free to pick up and complete it!