use cling or libclang for JIT C/C++ compilation in VM, and for tinycc backend replacement

[timotheecour]

example use cases

• for JIT compilation of importc procs (CT FFI)
• for evaluating const foo {.importc:”C_ISSOCK”.} as a const
• for replacement of tinycc backend (works on more platforms including osx, and supports C++)

links

• [Grant Proposal] Make Nim suitable for interactive programming (Implement hot code-reloading, a REPL and a Jupyter kernel) · Issue nim-lang/RFCs#511 · nim-lang/Nim
• NimVM eventually get replaced? - Nim forum
• vmhook : user defined vmops: proc fn(): int {.vmhook.} = impl to JIT-compile and dlopen fn at CT (using regular backend, not VM) · Issue nim-lang/Nim#598 · timotheecour/Nim

mir c2m JIT c compiler

• mir c2m: The MIR C interpreter and Just-in-Time (JIT) compiler - Red Hat Developer
• sources: https://github.com/vnmakarov/mir salient points:
• 0.90x speed of gcc -O2 on some benchmark (generated code quality) (tcc: 0.5x)
• 14x faster compilation vs gcc -02, 2x faster than gcc -O0 (tcc: 180x faster than gcc -O2)
• only C11, not C++; ABI compatiblity isn't complete (eg no apple M1 support)
• caveats:

  This code is in initial stages of development

cling

• root-project/cling: The cling C++ interpreter
• Cling - ROOT
• Cling Documentation | GOREPL
• cling: cling
• Replace cling with TinyCC-for-midge as the interpreter · Issue nim-lang/Nim#1 · ShimmyShaman/midge

  cling uses transactions (and c++), TinyCC is build to be fast and minimal.

• Cling Aims to Provide a High-performance C++ REPL
• Read-Compile-Run-Loop - a tiny REPL for C++ - Not just gamedev
• what are your CLING workflows? : cpp

[juancarlospaco]

• Whats a quick feature compare of Cling vs LibClang ?.
• Does this really meant to be a new Backend?, or a replacement/evolution of the VM ?.
• Does Cling work with C ?, docs only mention C++.
• Does that allow easy dynamic linking compilation?, the big problems of other LLVM-based are that compile all statically

Cling has LibClang and LLVM as dependency, that adds 3 big dependencies to the Nim compiler. :thinking:

[timotheecour]

Whats a quick feature compare of Cling vs LibClang ?

one is based on the other; I'm still exploring but at least cling adds a REPL, and use as a binary instead of having to link in libclang as library dependency, but I'm definitely not discarding using libclang directly, this needs more though.

Does this really meant to be a new Backend?, or a replacement/evolution of the VM ?.

some of the use cases don't involve a new backend: (eg making things like this possible: const foo {.importc:”C_ISSOCK”.}, or making CT ffi easier); others involve speeding up VM by doing jitting of vm code for hotspots, and yet other use cases involve replacing the existing (or deprecated) tcc backend with have with cling (which supports C++, and more platforms, than tcc; we'll need performance comparisons)

Does Cling work with C ?, docs only mention C++.

see other docs: Cling Cling is able to parse everything that clang can [...] Clang has support of C, objC, objC++ and we are looking forward to having more use-cases and extend our tool in that direction.

Does that allow easy dynamic linking compilation?

still exploring but this could be an advantage over using libclang directly, ie by invoking cling as a binary instead of linking as a library. TBD.

Cling has LibClang and LLVM as dependency, that adds 3 big dependencies to the Nim compiler.

that's fine if the dependency is optional. on osx (and other platforms probably the same) it's as simple as brew install cling. If user can get CT speedup or CT FFI using this, it's good enough of a use case.

[juancarlospaco]

Another option to research: https://www.keystone-engine.org/docs/beyond_llvm.html

Has projects using it to make REPL too.