Possible next steps

[lqd]

Some thoughts/braindump on possible next steps:

1. a gameplan would be great. Since there's a lot of moving parts between the rust compiler, MIR, binaryen, webassembly itself, and psosibly other helpful tools such as miri and emscripten, it's hard to have a precise vision of what to do immediately.
2. we could work on supporting more rust types, and more of the wasm types. Right now we only emit i32s out of the 4 wasm types for instance. (trans_operand panics if a value is not a Rust 32 bits isize, the binaryops emit the hardcoded i32 versions of those instructions, etc)
3. related to types and variables, is memory management, which is the point of issue #11 - what would be the best way to proceed there and so on. (Meanwhile we could prototype the manual version @kripken described there)
4. we could work on better operator support, both more rust operators and also wasm ones. I was trying out the PartialOrd ones, and this requires many traits and impls, Options, #[derive]. Related to point 11.
5. we could work on supporting more of Rust control flow and constructs, like match statements. (I was wondering whether br_table existed in the binaryen API ? -- since it looked interesting for this use case but I didn't investigate it very much)
6. updating to a newer nightly (even though #8 mentioned apparently that there might be some issues to do that). For instance there have been recent simplifications regarding locals (with AFAIK some code very similar to the ones that exists in the backend in trans_lval)
7. infrastructure-wise, tests like mentioned in #15 with validation, scaffolding generation, execution, etc would be definitely helpful
8. how to handle wasm specific features, like imports and exports, prints, or their intrinsics. Right now we emit say Add::add code even though it's not used by BinaryOps since wasm has i32.add for instance.
9. when would we have to deal with wasm32/wasm64 and how would that impact us (probably not much apart from maybe memory management)
10. at some point maybe documenting some Rust/MIR constructs and how they should map to wasm: it could document what exists and works, and explain the things that are left to do and how to do them
11. maybe some goals ? what kind of features would be expect to support and "when" even just for dev. So an idea here would be interesting: would we expect mir2wasm to compile some/all of the Rust std/core library directly and package that to the browser, or support something limited like no_core ?
12. after speaking with @eddyb and @solson Miri could be very useful, it has some utilities all backends need (and could even be a layer between us and the rustc_driver which could make upgrading to recent nightlies easier). e.g globals/constants/statics https://botbot.me/mozilla/rustc/2016-06-22/?msg=68388125&page=1

update: typo

[kripken]

4) br_table is supported in the binaryen C API, but I didn't hook up the relooper support for it yet. So for now we can't use it here. But practically speaking it's just an optimization, as we can do everything it does semantically (just without an efficient jump table). However, if there are reasons we need it more urgently, please let me know and I can prioritize it.

8) Probably best to ignore wasm64 for now. It's not even specced yet, there is just a general plan for it at this point.

[eholk]

10) I think it'd be awesome if we could pass rustc's test suite, or at least the run-pass and run-fail tests.

[eddyb]

@eholk That's a long ways off, although to get things semi-working you could somehow ignore all of the intrinsics and MIR Rvalues/types and compile a broken libcore/libstd that tests can link to.

[eholk]

Yeah, definitely. I think of it as passing the test suite is how we could know we are done, and it also gives a concrete set of test cases to start with. I imagine at least a few of them have no dependencies on libcore/libstd, so we might be able to start with those.

Of course, maybe the WebAssembly platform is sufficiently different that we'll want to have a libwebstd instead, or maybe libwebcore at least.

[eddyb]

@eholk The platform-specific stuff would result in a different libstd, but libcore is fundamental to Rust and has no OS dependencies, while liballoc and libcollections only require a memory allocator.

[eholk]

@eddyb Thanks! I knew there was some very small amount of stuff that Rust needs on each platform, I just didn't remember how it all breaks down. Anyway, it seems like it should be quite possible to run most Rust code on WebAssembly.

[brson]

Of course, maybe the WebAssembly platform is sufficiently different that we'll want to have a libwebstd instead, or maybe libwebcore at least.

When using emscripten as the runtime we can get pretty good POSIX fidelity, so I'm confident we can make std mostly work.

[lqd]

Here's a recently-updated list of things/roadmap of sorts 😄

• handle more types than i32s
• handle more of wasm's BinaryOps
• fix the fun_names map to not use the Sig but the Substs otherwise we'll end up with duplicate translated fns ( + with the same name) when more complex cases with traits/generics/monomorphizations are tested
• monomorphization tests to exercize more of the use cases (including the previous mentioned known issue)
• activate CI on the repo (it's been a couple times things related to isize have been broken either on x86 or x64; maybe automatically running rustfmt could be nice if that was possible)
• basic assert/panic for testing purposes — a first pass is in #36
• have a list of milestones of language features: useful for us as well as possible entry points for new contributors
• tidy up the things @eholk mentioned during the last big PR review
• optimize the calling convention esp re: functions returning structs (I started working on this already but it's not finished) (ie writing the return value to the calling fn's stack frame, to not require copies, esp. since the 2 MIR locals optimization projects have not yet landed — both of which I'm very much looking forward to)
• optimize the prologue/epilogue away when the stack pointer hasn't moved during the fn (already have done this with the previous item's work but it's not finished yet)
• update mir2wasm to a recent nightly: some things were added to help us backends out, and there have been MIR improvements and fixes (and more are coming) as I mentioned
• investigate wasm fn exports (with some similar FFI to wasm fn imports) to be called from js. Possibly only supporting extern on fns will be enough, but with some restrictions on return and arg types being the primitive wasm types
• implement wasm fn imports like we described in the github issue #33
• prepare tests (and eventually benches) to be run in a real wasm environment, browsers and js engines: there's some html/js scaffolding; some of which I already have from when I ran some of our mir2wasm output in chrome, here: https://twitter.com/lqd/status/752945875383705600
• clean-up the unit type representation, the ret_var which can be removed, have the functions return void and not expect a return value, etc
• implement the todos in the code that can be implemented
• modify build.rs to check for CMakeCache.txt in binaryen, if it exists the build will fail, so notify the user if it happens
• use miri for globals/statics/promoted constants etc (and possibly put those values inside wasm globals)
• more tests in general, this is almost always good :)
• implement the SwitchInt terminator to finish (I think) supporting the match construct (I started working on this already but it's just the beginning and I haven't done enough investigation to know whether BinaryenSwitch or the relooper functions are more appropriate, or having to manually roll comparisons/bit tests/binary search tree/jump table, etc)
• there's a todo in the code but this one begs a comment: use a cli parsing library in the mir2wasm bin. I thought I'd talk about it with you guys before adding a crate to the compiler, but I just noticed that getopts is already used. So we'll just need to switch from the ad-hoc code to use the getopts matches!
• add tests about the use of the linear memory; alignment has not been dealt with (and I have no experience with it or the problems it can create)
• add another test run pass where the tests are run after asking binaryen to optimize the wasm module
• related to exports: @eholk and I just had an interesting experience where v8 didn't launch the start function on module load if it wasn't exported, so we'd check the spec/examples/other impls to see if that's normal. Binaryen does execute it regardless, however. We might need to bring back the export for the start/main fn feature.
• references to locals of the primitive types (migrating the local from the wasm stack to the linear memory stack)
• think about inlining ? (but as for perf/benchmarks etc it'll be best to wait until the wast/wasm we produce is executed in real vms to get realistic numbers)
• investigate what exactly we need to compile libcore (besides the previous points, eg more types is an obvious need, as I found out when making progress on compiling core::cmp) — a quick investigation resulted in the following list (non-exhaustive):
  1. a story for libcore::intrinsics
  2. float constants (and the nightly we have is old enough to not have ConstFloat, so upgrading to a recent nightly is also a priority)
  3. other signed and unsigned numeric types
  4. all rust binops (BitAnd, Shr, Shl, BitOr, Rem, BitXor)
  5. unary ops (Not, already implemented in the assert PR)
  6. SwitchInt terminator
  7. promoted literals (probably requiring upgrading binaryen to use wasm globals, and also possibly requiring the use of miri)
  8. ConstVal::Str (probably represented as pointers to a data section of the linear memory)
  9. supporting more general/complex derefs like tmp0 = &mut (*((*var0).5: &'a mut fmt::Write + 'a)) found in fmt::Formatter::write_str
  10. a traits::resolve_trait_method that can fulfill_obligation for traits::VtableObject, found in `fmt::Formatter::write_str``, (unimplemented in miri as well)
  11. closure assignments: AggregateKind::Closure, found in num::<impl i8>::saturating_mul
  12. trying to compile the library showed a couple crashes in some areas related around monomorphizations or type substitutions (like getting the type layout in assignments)
  13. Drop terminator
  14. (that's where my current testing stopped — also hindered by the fact that trying to compile libcore on osx fails with a stack overflow, making wonder at the same time if there shouldn't be an item on this list to investigate changing the way we translate fns, ie not immediately when we see them — but there will be more, surely more around closures, probably boxes, etc — I'll try to continue these tests when I get back from the holidays)

[brson]

Thanks so much @lqd! I try to get easy issues into TWiR's call to participation each week, and I've added mir2wasm to my weekly triage list. I'll endeavor to break some of these out and try to stir up some more contributors.