search

facebookresearch / CompilerGym

Reinforcement learning environments for compiler and program optimization tasks
https://compilergym.ai/
MIT License
879 stars 123 forks source link

Action Space for LLVM 12 #788

Open cgao3 opened 1 year ago

cgao3 commented 1 year ago

❓ What would be action space for LLVM 12

I am wondering what would be a suitable action space definition for llvm 12. I know I can extract the "-O3" passes as following: 

-aa
-adce
-aggressive-instcombine
-alignment-from-assumptions
-always-inline
-annotation-remarks
-annotation2metadata
-argpromotion
-assumption-cache-tracker
-barrier
-basic-aa
-basiccg
-bdce
-block-freq
-branch-prob
-called-value-propagation
-callsite-splitting
-cg-profile
-constmerge
-correlated-propagation
-deadargelim
-demanded-bits
-div-rem-pairs
-domtree
-dse
-early-cse
-early-cse-memssa
-ee-instrument
-elim-avail-extern
-float2int
-forceattrs
-function-attrs
-globaldce
-globalopt
-globals-aa
-gvn
-indvars
-inferattrs
-inject-tli-mappings
-inline
-instcombine
-instsimplify
-ipsccp
-jump-threading
-lazy-block-freq
-lazy-branch-prob
-lazy-value-info
-lcssa
-lcssa-verification
-libcalls-shrinkwrap
-licm
-loop-accesses
-loop-deletion
-loop-distribute
-loop-idiom
-loop-load-elim
-loop-rotate
-loop-simplify
-loop-sink
-loop-unroll
-loop-unswitch
-loop-vectorize
-loops
-lower-constant-intrinsics
-lower-expect
-mem2reg
-memcpyopt
-memdep
-memoryssa
-mldst-motion
-openmpopt
-opt-remark-emitter
-pgo-memop-opt
-phi-values
-postdomtree
-profile-summary-info
-prune-eh
-reassociate
-rpo-function-attrs
-scalar-evolution
-sccp
-scoped-noalias-aa
-simplifycfg
-slp-vectorizer
-speculative-execution
-sroa
-strip-dead-prototypes
-tailcallelim
-targetlibinfo
-tbaa
-transform-warning
-tti
-vector-combine
-verify

However, it seems this set (94 passes) is incomplete. My question is then, what would be a complete version looks like? where can I find it?

Thanks

Attached. Action Space from LLVM 10 (current version)

Currently the llvm-v0 env contains 124 passes:

-adce
-add-discriminators
-aggressive-instcombine
-alignment-from-assumptions
-always-inline
-argpromotion
-attributor
-barrier
-bdce
-break-crit-edges
-called-value-propagation
-callsite-splitting
-canonicalize-aliases
-consthoist
-constmerge
-constprop
-coro-cleanup
-coro-early
-coro-elide
-coro-split
-correlated-propagation
-cross-dso-cfi
-dce
-deadargelim
-die
-div-rem-pairs
-dse
-early-cse
-early-cse-memssa
-ee-instrument
-elim-avail-extern
-flattencfg
-float2int
-forceattrs
-functionattrs
-globaldce
-globalopt
-globalsplit
-guard-widening
-gvn
-gvn-hoist
-hotcoldsplit
-indvars
-infer-address-spaces
-inferattrs
-inject-tli-mappings
-inline
-insert-gcov-profiling
-instcombine
-instnamer
-instsimplify
-ipconstprop
-ipsccp
-irce
-jump-threading
-lcssa
-libcalls-shrinkwrap
-licm
-load-store-vectorizer
-loop-data-prefetch
-loop-deletion
-loop-distribute
-loop-fusion
-loop-guard-widening
-loop-idiom
-loop-instsimplify
-loop-interchange
-loop-load-elim
-loop-predication
-loop-reduce
-loop-reroll
-loop-rotate
-loop-simplify
-loop-simplifycfg
-loop-sink
-loop-unroll
-loop-unroll-and-jam
-loop-unswitch
-loop-vectorize
-loop-versioning
-loop-versioning-licm
-lower-constant-intrinsics
-lower-expect
-lower-guard-intrinsic
-lower-matrix-intrinsics
-lower-widenable-condition
-loweratomic
-lowerinvoke
-lowerswitch
-mem2reg
-memcpyopt
-mergefunc
-mergeicmps
-mergereturn
-mldst-motion
-name-anon-globals
-nary-reassociate
-newgvn
-partial-inliner
-partially-inline-libcalls
-pgo-memop-opt
-post-inline-ee-instrument
-prune-eh
-reassociate
-redundant-dbg-inst-elim
-reg2mem
-rewrite-statepoints-for-gc
-rpo-functionattrs
-sancov
-scalarizer
-sccp
-separate-const-offset-from-gep
-simple-loop-unswitch
-simplifycfg
-sink
-slp-vectorizer
-slsr
-speculative-execution
-sroa
-strip
-strip-dead-prototypes
-strip-debug-declare
-strip-nondebug
-tailcallelim
ChrisCummins commented 1 year ago

Hi @cgao3, to adapt CompilerGym to use LLVM v12 rather than v10 you will need to modify the Action*.h files in https://github.com/facebookresearch/CompilerGym/tree/64bdd6cd39967d3d2fe5e6c72deb15e830b838bb/compiler_gym/envs/llvm/service/passes/10.0.0 .

Those files were generated by a script that would assemble the pass list from the LLVM sources, rather than parsing the output of clang -O3 or similar. The script to generate the pass list is in this directory:

https://github.com/facebookresearch/CompilerGym/tree/c6f46eb64af5cac11bc5c43eff90c6244ff085ff/compiler_gym/envs/llvm/service/passes

You would need to run this on the LLVM 12 source tree to generate an updated set of Action*.h headers, then build from source using CMake configured to build against LLVM 12 rather than 10.

Cheers, Chris

cgao3 commented 1 year ago

This is very helpful. Thanks so much.

cgao3 commented 1 year ago

@ChrisCummins some follow up..

So I was trying to build from source with CMake. At last step, I failed with following error

bash' '-c' '"/home/cgao3/CompilerGym/compiler_gym/make_config.py" --out-file-path "/home/cgao3/CompilerGym/build/compiler_gym/config.py"  --enable-llvm-env'
'bash' '-c' 'cd "/home/cgao3/CompilerGym/compiler_gym/.." && "/home/cgao3/anaconda3/bin/python3.9" setup.py --build-dir "/home/cgao3/CompilerGym/build/compiler_gym/../py_pkg/build" bdist_wheel --package-dir "build/compiler_gym/.." --dist-dir "/home/cgao3/CompilerGym/build/compiler_gym/../py_pkg/dist" --bdist-dir "/home/cgao3/CompilerGym/build/compiler_gym/../py_pkg/bdist" --get-wheel-filename'
-- Configuring done
CMake Error at build/_deps/grpc-src/CMakeLists.txt:2381 (target_link_libraries):
  Target "grpc" links to:

    absl::any_invocable

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

CMake Error at build/_deps/grpc-src/CMakeLists.txt:2943 (target_link_libraries):
  Target "grpc_unsecure" links to:

    absl::any_invocable

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

Any idea how to solve it? I tried multiple different versions for abseil, no help.

libin049 commented 1 year ago

@cgao3 I found same error, did you slover it?

@ChrisCummins some follow up..

So I was trying to build from source with CMake. At last step, I failed with following error

bash' '-c' '"/home/cgao3/CompilerGym/compiler_gym/make_config.py" --out-file-path "/home/cgao3/CompilerGym/build/compiler_gym/config.py"  --enable-llvm-env'
'bash' '-c' 'cd "/home/cgao3/CompilerGym/compiler_gym/.." && "/home/cgao3/anaconda3/bin/python3.9" setup.py --build-dir "/home/cgao3/CompilerGym/build/compiler_gym/../py_pkg/build" bdist_wheel --package-dir "build/compiler_gym/.." --dist-dir "/home/cgao3/CompilerGym/build/compiler_gym/../py_pkg/dist" --bdist-dir "/home/cgao3/CompilerGym/build/compiler_gym/../py_pkg/bdist" --get-wheel-filename'
-- Configuring done
CMake Error at build/_deps/grpc-src/CMakeLists.txt:2381 (target_link_libraries):
  Target "grpc" links to:

    absl::any_invocable

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

CMake Error at build/_deps/grpc-src/CMakeLists.txt:2943 (target_link_libraries):
  Target "grpc_unsecure" links to:

    absl::any_invocable

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

Any idea how to solve it? I tried multiple different versions for abseil, no help.

cgao3 commented 1 year ago

I was able to bypass this problem. I used clang10, and was building gRPC from source externally.

cgao3 commented 1 year ago

Some follow up.. I managed to build from source & made it support LLVM 12. Compilation seems worked well.

After installing the new whl package, I tried to test with python.

It gave me following error.. 

>>> env=compiler_gym.make("llvm-v0")
>>> env.action_space.n
129
>>> env.benchmark
benchmark://cbench-v1/qsort
>>> env.reset()
Traceback (most recent call last):
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 760, in _reset
    error, reply = _call_with_error(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 695, in _call_with_error
    return None, self.service(stub_method, *args, **kwargs)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/connection.py", line 813, in __call__
    return self.connection(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/connection.py", line 217, in __call__
    raise FileNotFoundError(e.details()) from None
FileNotFoundError: Benchmark not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/connection.py", line 201, in __call__
    return stub_method(request, timeout=timeout)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.UNAVAILABLE
        details = "Connection reset by peer"
        debug_error_string = "{"created":"@1683054550.266573637","description":"Error received from peer ipv6:[::1]:34143","file":"src/core/lib/surface/call.cc","file_line":1074,"grpc_message":"Connection reset by peer","grpc_status":14}"
>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 695, in _call_with_error
    return None, self.service(stub_method, *args, **kwargs)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/connection.py", line 813, in __call__
    return self.connection(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/connection.py", line 230, in __call__
    raise ServiceIsClosed("Service is offline")
compiler_gym.errors.service_errors.ServiceIsClosed: Service is offline

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/envs/llvm/llvm_env.py", line 366, in reset
    return super().reset(*args, **kwargs)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 624, in reset
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 681, in _retry
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 681, in _retry
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 681, in _retry
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 681, in _retry
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 681, in _retry
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 675, in _retry
    raise OSError(
OSError: Failed to reset environment using benchmark benchmark://cbench-v1/qsort after 4 attempts.
Last error (ServiceIsClosed): Service is offline

what could be reason for this? any ideas? @ChrisCummins thanks..

ChrisCummins commented 1 year ago

Hi @cgao3, I'm sorry I missed your earlier messages, just catching up now.

Congrats on your progress with the LLVM 12 port! That's great.

The error you're reporting appears to be thrown here:

https://github.com/facebookresearch/CompilerGym/blob/development/compiler_gym/service/runtime/CompilerGymServiceImpl.h#LL71C55-L71C73

You can add VLOG(1) macros to do printf debugging here to check that the values are what you expect. Then run with environment variable COMPILER_GYM_DEBUG=3 to report extra logging output. Without knowing the details of what you've changed I can't imagine any particular reason why that error would be thrown.

Cheers, Chris

sparks-code commented 1 year ago

Some follow up.. I managed to build from source & made it support LLVM 12. Compilation seems worked well.

After installing the new whl package, I tried to test with python.

It gave me following error..

>>> env=compiler_gym.make("llvm-v0")
>>> env.action_space.n
129
>>> env.benchmark
benchmark://cbench-v1/qsort
>>> env.reset()
Traceback (most recent call last):
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 760, in _reset
    error, reply = _call_with_error(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 695, in _call_with_error
    return None, self.service(stub_method, *args, **kwargs)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/connection.py", line 813, in __call__
    return self.connection(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/connection.py", line 217, in __call__
    raise FileNotFoundError(e.details()) from None
FileNotFoundError: Benchmark not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/connection.py", line 201, in __call__
    return stub_method(request, timeout=timeout)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.UNAVAILABLE
        details = "Connection reset by peer"
        debug_error_string = "{"created":"@1683054550.266573637","description":"Error received from peer ipv6:[::1]:34143","file":"src/core/lib/surface/call.cc","file_line":1074,"grpc_message":"Connection reset by peer","grpc_status":14}"
>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 695, in _call_with_error
    return None, self.service(stub_method, *args, **kwargs)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/connection.py", line 813, in __call__
    return self.connection(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/connection.py", line 230, in __call__
    raise ServiceIsClosed("Service is offline")
compiler_gym.errors.service_errors.ServiceIsClosed: Service is offline

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/envs/llvm/llvm_env.py", line 366, in reset
    return super().reset(*args, **kwargs)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 624, in reset
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 681, in _retry
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 681, in _retry
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 681, in _retry
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 681, in _retry
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 681, in _retry
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 675, in _retry
    raise OSError(
OSError: Failed to reset environment using benchmark benchmark://cbench-v1/qsort after 4 attempts.
Last error (ServiceIsClosed): Service is offline

what could be reason for this? any ideas? @ChrisCummins thanks..

This seems to be a GRPC problem. Are you sure you installed GRPC correctly?

cgao3 commented 1 year ago

@sparks-code It seems to me gRPC is installed properly. But, do you know how to test gRPC separately?

@ChrisCummins Here is a logging info after using COMPILER_GYM_DEBUG=3


(compiler_gym) cgao3@7B09:~/repos$ ls /home/cgao3/qsort.bc
/home/cgao3/qsort.bc

>>> import compiler_gym
>>> env = compiler_gym.make("llvm-v0")
I0509 13:17:40.248921 285208 CreateAndRunCompilerGymServiceImpl.h:122] Service "/dev/shm/compiler_gym_cgao3/s/0509T131740-211000-8211" listening on 40725, PID = 285208
I0509 13:17:40.323717 285212 CompilerGymServiceImpl.h:41] GetSpaces()
>>> b = env.make_benchmark("/home/cgao3/qsort.bc")
>>> env.reset(benchmark=b)
I0509 13:18:07.590183 285212 CompilerGymServiceImpl.h:60] StartSession(id=0, benchmark=benchmark://file-v0/home/cgao3/qsort.bc), 1 active sessions
I0509 13:18:07.590637 285223 CompilerGymServiceImpl.h:192] AddBenchmark()
I0509 13:18:07.590920 285223 CompilerGymServiceImpl.h:60] StartSession(id=0, benchmark=benchmark://file-v0/home/cgao3/qsort.bc), 1 active sessions
I0509 13:18:07.590981 285223 BenchmarkFactory.cc:41] BenchmarkFactory initialized
I0509 13:18:07.757074 285398 CreateAndRunCompilerGymServiceImpl.h:122] Service "/dev/shm/compiler_gym_cgao3/s/0509T131807-716206-5704" listening on 32821, PID = 285398
I0509 13:18:07.824682 285402 CompilerGymServiceImpl.h:41] GetSpaces()
I0509 13:18:07.825698 285402 CompilerGymServiceImpl.h:60] StartSession(id=0, benchmark=benchmark://file-v0/home/cgao3/qsort.bc), 1 active sessions
I0509 13:18:07.826129 285411 CompilerGymServiceImpl.h:192] AddBenchmark()
I0509 13:18:07.826404 285402 CompilerGymServiceImpl.h:60] StartSession(id=0, benchmark=benchmark://file-v0/home/cgao3/qsort.bc), 1 active sessions
I0509 13:18:07.826463 285402 BenchmarkFactory.cc:41] BenchmarkFactory initialized
I0509 13:18:07.977257 285415 CreateAndRunCompilerGymServiceImpl.h:122] Service "/dev/shm/compiler_gym_cgao3/s/0509T131807-963363-2023" listening on 41771, PID = 285415
I0509 13:18:08.071425 285419 CompilerGymServiceImpl.h:41] GetSpaces()
I0509 13:18:08.072410 285419 CompilerGymServiceImpl.h:60] StartSession(id=0, benchmark=benchmark://file-v0/home/cgao3/qsort.bc), 1 active sessions
I0509 13:18:08.072800 285428 CompilerGymServiceImpl.h:192] AddBenchmark()
I0509 13:18:08.073055 285419 CompilerGymServiceImpl.h:60] StartSession(id=0, benchmark=benchmark://file-v0/home/cgao3/qsort.bc), 1 active sessions
I0509 13:18:08.073107 285419 BenchmarkFactory.cc:41] BenchmarkFactory initialized
I0509 13:18:08.225723 285432 CreateAndRunCompilerGymServiceImpl.h:122] Service "/dev/shm/compiler_gym_cgao3/s/0509T131808-206968-2990" listening on 43565, PID = 285432
I0509 13:18:08.315249 285436 CompilerGymServiceImpl.h:41] GetSpaces()
I0509 13:18:08.316171 285436 CompilerGymServiceImpl.h:60] StartSession(id=0, benchmark=benchmark://file-v0/home/cgao3/qsort.bc), 1 active sessions
I0509 13:18:08.316583 285445 CompilerGymServiceImpl.h:192] AddBenchmark()
I0509 13:18:08.316841 285436 CompilerGymServiceImpl.h:60] StartSession(id=0, benchmark=benchmark://file-v0/home/cgao3/qsort.bc), 1 active sessions
I0509 13:18:08.316898 285436 BenchmarkFactory.cc:41] BenchmarkFactory initialized
I0509 13:18:08.466900 285449 CreateAndRunCompilerGymServiceImpl.h:122] Service "/dev/shm/compiler_gym_cgao3/s/0509T131808-450630-980a" listening on 35295, PID = 285449
I0509 13:18:08.558523 285453 CompilerGymServiceImpl.h:41] GetSpaces()
I0509 13:18:08.559523 285453 CompilerGymServiceImpl.h:60] StartSession(id=0, benchmark=benchmark://file-v0/home/cgao3/qsort.bc), 1 active sessions
I0509 13:18:08.559937 285462 CompilerGymServiceImpl.h:192] AddBenchmark()
I0509 13:18:08.560195 285453 CompilerGymServiceImpl.h:60] StartSession(id=0, benchmark=benchmark://file-v0/home/cgao3/qsort.bc), 1 active sessions
I0509 13:18:08.560254 285453 BenchmarkFactory.cc:41] BenchmarkFactory initialized
I0509 13:18:08.709897 285466 CreateAndRunCompilerGymServiceImpl.h:122] Service "/dev/shm/compiler_gym_cgao3/s/0509T131808-692542-be82" listening on 44661, PID = 285466
I0509 13:18:08.800768 285470 CompilerGymServiceImpl.h:41] GetSpaces()
I0509 13:18:08.801728 285470 CompilerGymServiceImpl.h:60] StartSession(id=0, benchmark=benchmark://file-v0/home/cgao3/qsort.bc), 1 active sessions
I0509 13:18:08.802135 285479 CompilerGymServiceImpl.h:192] AddBenchmark()
I0509 13:18:08.802397 285470 CompilerGymServiceImpl.h:60] StartSession(id=0, benchmark=benchmark://file-v0/home/cgao3/qsort.bc), 1 active sessions
I0509 13:18:08.802453 285470 BenchmarkFactory.cc:41] BenchmarkFactory initialized
Traceback (most recent call last):
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 760, in _reset
    error, reply = _call_with_error(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 695, in _call_with_error
    return None, self.service(stub_method, *args, **kwargs)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/connection.py", line 813, in __call__
    return self.connection(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/connection.py", line 217, in __call__
    raise FileNotFoundError(e.details()) from None
FileNotFoundError: Benchmark not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/connection.py", line 201, in __call__
    return stub_method(request, timeout=timeout)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/grpc/_channel.py", line 946,
in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/grpc/_channel.py", line 849,
in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.UNAVAILABLE
        details = "Connection reset by peer"
        debug_error_string = "{"created":"@1683659888.929474139","description":"Error received from peer
ipv6:[::1]:44661","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset
by peer","grpc_status":14}"
>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 695, in _call_with_error
    return None, self.service(stub_method, *args, **kwargs)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/connection.py", line 813, in __call__
    return self.connection(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/connection.py", line 230, in __call__
    raise ServiceIsClosed("Service is offline")
compiler_gym.errors.service_errors.ServiceIsClosed: Service is offline

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/envs/llvm/llvm_env.py", line 366, in reset
    return super().reset(*args, **kwargs)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 624, in reset
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 681, in _retry
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 681, in _retry
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 681, in _retry
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 681, in _retry
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 681, in _retry
    return self._reset(
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 778, in _reset
    return _retry(error)
  File "/home/cgao3/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py", line 675, in _retry
    raise OSError(
OSError: Failed to reset environment using benchmark benchmark://file-v0/home/cgao3/qsort.bc after 4 attempts.
Last error (ServiceIsClosed): Service is offline
ChrisCummins commented 1 year ago

@cgao3, okay, looks like its expecting /home/cgao3/qsort.bc file, which is I guess the file you made. I would start by stepping through the failing call and validating that everything looks as should (I would focus particularly here https://github.com/facebookresearch/CompilerGym/blob/9e0c0beb12da1e1ea82ae6ce920713ee28dda4c9/compiler_gym/service/client_service_compiler_env.py#L721-L724 ). You may need to add some additional debugging printouts to the compiled C++ backend and rebuild, else run the backend service under a debugger.

You may want to focus on BenchmarkCache.cc to see what it has populated at the time the error is raised to understand why the requested benchmark is "missing" https://github.com/facebookresearch/CompilerGym/blob/development/compiler_gym/service/runtime/BenchmarkCache.cc#L28-L56

Cheers, Chris

zcfh commented 11 months ago

to adapt CompilerGym to use LLVM v12 rather than v10 you will need to modify the Action*.h files in

use branch release/v0.2.5.

I run python3 build_tools/llvm/legacy_pass_manager/extract_passes_from_llvm_source_tree.py /path/to/llvm11/llvm | python3 build_tools/llvm/legacy_pass_manager/make_action_space_genfiles.py compiler_gym/envs/llvm /service/passes/11.1.0, and compiler_gym /envs/llvm/service/passes/11.1.0/flags.txt generate 608 flags. There is such a big difference in quantity, is it because my way of using it is wrong?

zcfh commented 10 months ago

I also ran into an error with an external project.

ld.lld: error: undefined symbol: llvm::parseIR(llvm::MemoryBufferRef, llvm::SMDiagnostic&, llvm::LLVMContext&, bool, llvm::StringRef)
>>> referenced by llvm.cc
>>>               llvm.pic.o:(programl::ir::llvm::BuildProgramGraph(llvm::MemoryBuffer const&, programl::ProgramGraph*, programl::ProgramGraphOptions const&)) in archive external/programl/programl/src/programl/bazel-bin/programl/ir/llvm/libllvm-10.a
cc_library(
    name = "llvm-10",
    ......
    deps = [
        # ......
        "@llvm//10.0.0",
    ],
)

I search the BUILD file of programl, it depends on llvm//10.0.0.

May I ask if you have encountered such a problem, how to solve it?