mlc-ai / relax

Apache License 2.0
149 stars 75 forks source link

error when call tune_tir for Android OpenCL by RPC #264

Open huanyingjun opened 1 year ago

huanyingjun commented 1 year ago

Dear I use rpc to tune my Tir Function On Android device for OpenCL. My code is as below:

rpc_host = "127.0.0.1"
rpc_port = 9190
rpc_key = "android"

my_rpc_config = ms.runner.RPCConfig(
            tracker_host=rpc_host,
            tracker_port=rpc_port,
            tracker_key=rpc_key,
            session_timeout_sec=180,
        )
my_workers = my_rpc_config.count_num_servers(allow_missing=False)

def get_runner():
    runner_config = {
        "evaluator_config": ms.runner.EvaluatorConfig(
            number=3,
            repeat=1,
            min_repeat_ms=100,
            enable_cpu_cache_flush=False,
        ),
        "alloc_repeat": 5,
    }
    runner = ms.runner.RPCRunner(
        rpc_config=my_rpc_config, max_workers=my_workers, **runner_config
    )

    return runner

my_target = tvm.target.Target("opencl", host="llvm -mtriple=aarch64-linux-android")
database = ms.tune_tir(
    mod=MyModule,
    target=my_target,
    max_trials_global=64,
    num_trials_per_iter=64,
    work_dir="./",
    runner=get_runner(),
)

got below error:

2023-07-11 16:55:32 [INFO] [task_scheduler.cc:121] [Task #0: main] Trial #2: Error in running:
RPCRunner: An exception occurred
Traceback (most recent call last):
  File "/home/qq/work/tvm_llm/relax/python/tvm/exec/popen_worker.py", line 87, in main
    result = fn(*args, **kwargs)
  File "/home/qq/work/tvm_llm/relax/python/tvm/meta_schedule/runner/rpc_runner.py", line 392, in _worker_func
    rt_mod: Module = f_upload_module(session, local_path, remote_path)
  File "/home/qq/work/tvm_llm/relax/python/tvm/meta_schedule/runner/rpc_runner.py", line 451, in default_upload_module
    rt_mod: Module = session.load_module(remote_path)
  File "/home/qq/work/tvm_llm/relax/python/tvm/rpc/client.py", line 161, in load_module
    return _ffi_api.LoadRemoteModule(self._sess, path)
  File "/home/qq/work/tvm_llm/relax/python/tvm/_ffi/_ctypes/packed_func.py", line 238, in __call__
    raise get_last_ffi_error()
tvm.error.RPCError: Traceback (most recent call last):
  5: TVMFuncCall
  4: _ZN3tvm7runtime13PackedFuncObj9ExtractorINS0_16PackedFuncSubObjIZNS0_15TypedPackedFuncIFNS0_6ModuleES5_NSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEE17AssignTypedLambdaINS0_UlS5_SB_E3_EEEvT_SB_EUlRKNS0_7TVMArgsEPNS0_11TVMRetValueEE_EEE4CallEPKS1_SH_SL_
  3: tvm::runtime::RPCWrappedFunc::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
  2: tvm::runtime::RPCClientSession::CallFunc(void*, TVMValue const*, int const*, int, std::function<void (tvm::runtime::TVMArgs)> const&)
  1: tvm::runtime::RPCEndpoint::CallFunc(void*, TVMValue const*, int const*, int, std::function<void (tvm::runtime::TVMArgs)>)
  0: tvm::runtime::RPCEndpoint::HandleUntilReturnEvent(bool, std::function<void (tvm::runtime::TVMArgs)>)
  File "/home/qq/work/tvm_llm/relax/src/runtime/rpc/rpc_endpoint.cc", line 380
RPCError: Error caught from RPC call:
[16:55:22] /home/qq/work/tvm_llm/relax/apps/cpp_rpc/rpc_env.cc:310: tar: chdir './rpc/tmp/': No such file or directory

Could please help check this ?