[llvm] Add API to construct a benchmark from clang invocation.

[ChrisCummins]

This adds a new API for constructing a benchmark from a command line invocation of a compiler. For example, given the command line:

$ gcc a.c src/b.c -DNDEBUG -lm -Oz -o foo.exe

The make_benchmark_from_command_line() method interprets this command line and rewrites it to one that emits an LLVM-IR file:

/path/to/compilergym/clang a.c src/b.c -DNDEG -lm -Oz -emit-llvm -c -o /path/to/benchmark.bc -Xclang disable-llvm-optmzns

It then loads the resulting bitcode for use as a benchmark. When the user is done playing around with the benchmark, they call a special compiler() method on the benchmark which re-invokes the original command line, but this time using the bitcode as input:

/path/to/compilergym/clang /path/to/benchmark.nc -DNDEG -lm -Oz -o foo.exe

Putting this all together, here is an end-to-end example using CompilerGym as a substitute for LLVM's pipeline:

>>> import compiler_gym
>>> env = compiler_gym.make("llvm-v0")
>>> benchmark = env.make_benchmark_from_command_line(                                                                                                                                                                                                                       
...    "gcc a.c src/b.c -DNDEBUG -lm -Oz -o foo.exe"
... )                                                                                                                                                                                                                                                                       
>>> for _ in range(100):
...     _, _, done, _ env.step(env.action_space.sample()
...     if done:
...         env.reset()
>>> env.benchmark.compile()
# Now the file foo.exe is compiled

The idea is to make it easier to integrate CompilerGym into an existing build system, as it is normally possible to dump a verbose log of all of the compiler invocations run during a build.

[codecov-commenter]

Codecov Report

Merging #577 (72ef9b1) into development (63dbfac) will increase coverage by 0.17%. The diff coverage is 95.97%.

@@               Coverage Diff               @@
##           development     #577      +/-   ##
===============================================
+ Coverage        88.36%   88.54%   +0.17%     
===============================================
  Files              125      127       +2     
  Lines             7565     7761     +196     
===============================================
+ Hits              6685     6872     +187     
- Misses             880      889       +9     

Impacted Files	Coverage Δ
...piler_gym/envs/llvm/benchmark_from_command_line.py	92.68% <92.68%> (ø)
...ler_gym/third_party/gccinvocation/gccinvocation.py	95.91% <95.91%> (ø)
compiler_gym/envs/llvm/llvm_env.py	91.61% <98.27%> (+3.32%)	:arrow_up:
compiler_gym/bin/validate.py	87.09% <100.00%> (ø)
compiler_gym/envs/llvm/__init__.py	100.00% <100.00%> (ø)
compiler_gym/service/connection.py	77.92% <0.00%> (-0.98%)	:arrow_down:
...ompiler_gym/service/client_service_compiler_env.py	91.26% <0.00%> (+0.41%)	:arrow_up:

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 63dbfac...72ef9b1. Read the comment docs.

[ChrisCummins]

LGTM. Quick question though. How does it deal with mixed .c and .o files?

Good question! I added support so that .o files are excluded from the bitcode environment, but are kept for linking later.

The current implementation for this will not work if the CompilerGym backend and frontend are on different machines. To fix that, we need #325.

Cheers, Chris