search

rust-lang / miri

An interpreter for Rust's mid-level intermediate representation
Apache License 2.0
4.33k stars 329 forks source link

Support native FFI calls via libffi #11

Open DemiMarie opened 8 years ago

DemiMarie commented 8 years ago

It would be nice to use libffi to allow for external calls.

eddyb commented 8 years ago

Previously suggested on IRC based on @cmr's mentions in a different context. The closure example presents what I think is one of the most valuable features of this library: the ability to JIT a C function which hardcodes extra owned values, a whole closure masquerading in a function pointer.

solson commented 8 years ago

@drbo We've definitely been considering that.

For a bit of context, there are two versions of Miri we'll have in the future. If the Rust teams decide to use a MIR-based constant evaluator, some version of Miri will get incorporated into rustc, and that version will never use libffi because you want to ban arbitrary FFI calls in constant evaluation.

But it would be a lot of fun to add libffi to this external version of Miri. I would probably wait until after any Miri → rustc merging happens, though.

BatmanAoD commented 6 years ago

Now that Miri is indeed in rustc (wooooo!), any further thoughts on this?

If I understand correctly, FFI would be necessary for us to use Miri as the basis for a Jupyter kernel.

eddyb commented 6 years ago

@BatmanAoD I'd advise against using miri for such a purpose - have you looked at webassembly?

BatmanAoD commented 6 years ago

@eddyb I don't actually know what the requirements are for a Jupyter kernel; I was basing my understanding on an old discussion about using rusti instead of Miri for the basis of the kernel.

I can't actually find documentation on how the kernel API works, though (I probably just haven't spent enough time looking). It may be that communication between JupyterLab and the kernel is entirely over HTTP.

In any case: I'm not sure how WebAssembly would solve the problem. The Jupyter kernel is a REPL for a given language; the goal of a Rust kernel would be to permit people to create Jupyter notebooks that would use the Rust language instead of IPython (or any of the other languages that are already supported via kernels). So the need is for a Rust REPL that supports whatever interface JupyterLab uses.

eddyb commented 6 years ago

@BatmanAoD Oops, I misunderstood "kernel" as "OS kernel", not an IPython-like thing. Then miri is likely the way to go, and I agree libffi would be needed.

BatmanAoD commented 6 years ago

@eddyb Yeah, I think the terminology is a bit confusing!

oli-obk commented 6 years ago

This project is almost ready (as in compiling and testing on Travis) again, I have a PR against rustc ready that should fix all the tests. After that we can accept new PRs. So if anyone wants to hack up a prototype using libffi to execute some c functions that we're currently having horrible fake support for (like syscall), that would be totally awesome. Things like mallocare a little harder though, but with a new allocation kind totally doable without touching rustc.

eddyb commented 6 years ago

@oli-obk I think pointers in general are hard. Various syscalls take pointers (e.g. read and write) while others return them (e.g. mmap) - so malloc is not unique in this difficulty. It's easier to detect miri pointers being passed to FFI, and do something about it, than it is to handle the other way - I'm thinking maybe miri could be self-aware (e.g. via /proc/self/maps) and allow access via integer pointers to memory regions that would surely not segfault (that would, amusingly, allow corrupting miri itself - which C code could do anyway, so it's not introducing a new issue).

BatmanAoD commented 6 years ago

@eddyb If it were possible to guarantee that improper writes would segfault, wouldn't that be preferable to corrupting the Miri process without terminating it?

eddyb commented 6 years ago

@BatmanAoD My point was that integer pointers could be used to corrupt miri itself - regardless of how you handle segfaults, the values for which corruption could happen are generally hard to detect, and pointless anyway, because you're also calling into C, which can do anything anyway.

oli-obk commented 6 years ago

Yea.... I don't think we should care too much about users doing horrible things. If you do care, don't enable the ffi feature ^^

elichai commented 5 years ago

Any news on this? Would love to use miri on some projects that have a lot of FFI calls.

Ericson2314 commented 5 years ago

libffi is due for a new release. (GHC and probably other projects have had to vendor it which is a pain.) If you do end up using it, please lobby upstream so we can all get a working release!

emarteca commented 2 years ago

Hey! I've started working on this issue -- here is a design doc of our proposed plan so far: link here. We'd love your feedback/comments on the proposed approach! @oli-obk @RalfJung

Thanks! :)

RalfJung commented 2 years ago

@emarteca that's a very welcome surprise. :-) Let's make a new issue to specifically discuss your project: https://github.com/rust-lang/miri/issues/2365