search

omarandlorraine / strop

Stochastically generates machine code
MIT License
95 stars 1 forks source link

PyO3, let's be able to import strop from python. #20

Closed omarandlorraine closed 2 years ago

omarandlorraine commented 2 years ago

This pull request logs the thoughts on using PyO3 and maturin to create some kind of module that's importable from Python.

omarandlorraine commented 2 years ago

There is a job that checks that pylint passes on the script python_tests.py. It'd be worth sticking that on the README as well.

omarandlorraine commented 2 years ago

Turns out PyO3 doesn't work nicely with rust Enums (fair enough; there is no real Python equivalent), I may redo Machine as a struct. This will affect all those match statements of course.

LoganDark commented 2 years ago

What's the utility of this? Do you plan on using Python for low-level compiling and binary stuff?

omarandlorraine commented 2 years ago

What's the utility of this? Do you plan on using Python for low-level compiling and binary stuff?

Okay so at the moment there are two ways of specifying which function you want: either use one of the built-in ones, or put the whole table in a massive JSON.

It would be better to be able to say, "here is a function I've written, give me the equivalent assembly". I absolutely don't envisage entire Python programs to be converted to native binaries this way, just individual functions. But I haven't yet decided how important it is

LoganDark commented 2 years ago

It would be better to be able to say, "here is a function I've written, give me the equivalent assembly".

Python (and GIL) makes this inefficient, I'd much prefer specifying a Rust closure and saying "match the behavior of this" :)

I'm not envisioning Python programs being compiled by strop, but Python programs being used to assemble binaries using strop, which gives me flashbacks to avbtool (the only tool for signing Android boot information) and how difficult that was to get working because it depended on Python.

Could this be a separate, optional crate so if I want to use strop, I don't have to pull in Python stuff? This would also ensure strop's API design is good enough to be used by downstream crates in that way, rather than requiring actual first-party integration with Python stuff.

omarandlorraine commented 2 years ago

Could this be a separate, optional crate so if I want to use strop, I don't have to pull in Python stuff?

Uh, probably. I've heard of that kind of thing been done before, and it's a good idea. Can you figure it out?

One problem I know is I don't have a stable API yet. And I also don't really know what it would look like yet.

LoganDark commented 2 years ago

Could this be a separate, optional crate so if I want to use strop, I don't have to pull in Python stuff?

Uh, probably. I've heard of that kind of thing been done before, and it's a good idea. Can you figure it out?

One problem I know is I don't have a stable API yet.

Use a cargo workspace - store all crates in this one repository so that you can cargo check / cargo test across all members at once, and IDE refactoring also rewrites the references from other crates. So you'll still work on one codebase, but instead of just separate modules it's separate crates.

And I also don't really know what it would look like yet.

I'd recommend a name like strop-pyo3 that contains all the Python stuff, and just depends on strop. i.e. strop itself does not depend on Python or have any special APIs for only Python use.

This encourages you to design strop's API in such a way that anyone can make their own crate like strop-mlua or strop-qjs or strop-rhai that bring strop bindings to other scripting languages.

omarandlorraine commented 2 years ago

Yeah I think what I'm doing here is the wrong approach. I'm just going to close this off.

omarandlorraine commented 2 years ago

LoganDark if we're forgetting about any kind of python frontend for this project, and having a separate crate instead, (and separate frontends for lisp, C, etc if we get around to it), then it also kind of makes sense to remove the JSON stuff as well. What do you think?

LoganDark commented 2 years ago

LoganDark if we're forgetting about any kind of python frontend for this project, and having a separate crate instead, (and separate frontends for lisp, C, etc if we get around to it), then it also kind of makes sense to remove the JSON stuff as well. What do you think?

I think the JSON stuff can stay for the command-line binary, but yeah, in the core library you should just have generic stuff. i.e. pass a closure for tests

strop & strop-cli for example

omarandlorraine commented 2 years ago

pass a closure for tests

Okay. There appears to be a variadic_closure crate that looks promising.

LoganDark commented 2 years ago

pass a closure for tests

Okay. There appears to be a variadic_closure crate that looks promising.

What no. You don't need that... Why do you think you need that? You just have a closure that gets passed the generated program, and runs any tests on it - anything (equality, don't care bits, floating point etc...)

omarandlorraine commented 2 years ago

Oh, now I see which closure you mean!

I was thinking that because the closure we're "compiling" can take any number of inputs and outputs, so I misunderstood what you meant.

LoganDark commented 2 years ago

Oh, now I see which closure you mean!

I was thinking that because the closure we're "compiling" can take any number of inputs and outputs, so I misunderstood what you meant.

Sorry if my answer was rude/blunt, my keyboard is currently broken so I won't be able to offer much guidance/help for the next few weeks at least :/