search

immunant / c2rust

Migrate C code to Rust
https://c2rust.com/
Other
3.91k stars 229 forks source link

Parallelize transpiling #195

Open rinon opened 4 years ago

rinon commented 4 years ago

We should be able to easily transpile all translation units in parallel, either in separate processes or threads. Just needs to be implemented at some point.

kkysen commented 2 years ago

Running c2rust_transpile::transpile in parallel (with a .par_map) on completely separate compilation databases (the directories under tests/) actually causes panics regularly, and once I got a segfault, too. I'm guessing there's something in c2rust-ast-exporter or clang APIs that's stateful and causing lots of issues. This would be very nice to have, even if just for speeding up tests.

rinon commented 2 years ago

clang does keep internal global state, and I don't think that's going to be fixable. I think this will need to be process parallelization, i.e. what build systems do.

kkysen commented 2 years ago

Yeah, that seems a lot safer to do. Would it be possible to process-parallelize the clang and c2rust-ast-exporter parts up until serialization, and then from deserialization on, which is all in Rust, do normal thread-parallelization?