search

oils-for-unix / oils

Oils is our upgrade path from bash to a better language and runtime. It's also for Python and JavaScript users who avoid shell!
http://www.oilshell.org/
Other
2.85k stars 159 forks source link

add mycpp-souffle translator #2076

Closed melvinw closed 2 months ago

melvinw commented 2 months ago

This commit adds a new translator called mycpp-souffle. It translates files using mycpp with optimizations enabled.

You can build the translation examples with souffle optimizations enabled with the same targets you normally would, but with mycpp replaced by mycpp-souffle in the suffix. For example, you build the intermediate C++ and the cxx-asan binary for mycpp/examples/gc_stack_roots.py respectively by running the following commands:

ninja _gen/mycpp/examples/gc_stack_roots.mycpp-souffle.cc
ninja _bin/cxx-asan/mycpp/examples/gc_stack_roots.mycpp-souffle

You can also build OSH and YSH with this new variant. Their targets are named _bin/<variant>/mycpp-souffle/osh and _bin/<variant>/mycpp-soufle/ysh respectively. Currently, only the cxx-opt, cxx-asan, and cxx-asan+gcalways variants are enabled for the new translator. The binaries translated with vanilla mycpp are still available at their existing paths, e.g. _bin/cxx-dbg/osh.

The translated C++ for the entire program with optimizations is named _gen/bin/oils_for_unix.mycpp-souffle.cc.

andychu commented 2 months ago

Hm looks like an R failure

*<c2><a0>44, 45
*<c2><a0>35, 36
*<c2><a0>50, 51
*<c2><a0>53, 54
*<c2><a0>17, 18
Backtrace:
    x
 1. +-global main(commandArgs(TRUE))
 2. | \-global MyCppReport(in_dir, out_dir)
 3. |   \-... %>% arrange(`C++ : Python`)
 4. +-dplyr::arrange(., `C++ : Python`)
 5. +-dplyr::mutate(., `C++ : Python` = `C++`/Python)
 6. +-tidyr::spread(., key = impl, value = user_ms)
 7. \-tidyr:::spread.data.frame(., key = impl, value = user_ms)
 8.   \-cli::cli_abort(...)
 9.     \-rlang::abort(...)
Execution halted

Unfortunately the scripts don't have good error messages

I suspect probably some file or row was generated, or two files mismatched because the output file wasn't created ?

Feel free to DM me if you are having build system issues - this stuff is very fiddly

the "pea" stuff was only partially done I think

melvinw commented 2 months ago

Ah, yes. I noticed that the other day. Spending some time on this tonight. I'll PM you if I get lost

andychu commented 2 months ago

BTW I would just fix the Usage string

I will deal with the sh_path / shell_name stuff separately -- that is a smell that was already there

It's cool that this is working! what do we want to do before making it the default?

(Julian Brown has stress tested our C++ build pretty effectively! The bugs were more like "oversights" than fundamental design problems )

melvinw commented 2 months ago

Sounds good! I just pushed the usage fix.

I think we can make mycpp-souffle the default translator when at least the following conditions are satisfied:

  1. We hook mycpp-souffle up to the spec-cpp and interactive jobs
  2. We try out some other optimizations to make sure that this framework works well for things other than just the stack roots solver.
andychu commented 2 months ago

If you build two variants like cxx-opt and cxx-asan, the build.ninja should only create one copy of the original file

It shouldn't run mycpp twice

Unless there is a dependency graph bug (which is not hard to introduce)

melvinw commented 2 months ago

Oh good point. I'll spend some time figuring out what else could be going on here