search

apache / tvm

Open deep learning compiler stack for cpu, gpu and specialized accelerators
https://tvm.apache.org/
Apache License 2.0
11.58k stars 3.43k forks source link

Error emitting Optimized C Code #8783

Open bhahn221 opened 3 years ago

bhahn221 commented 3 years ago

I first used AutoTVM with TVMC to tune the model, and I used the following command

tvmc tune --target "c" --output mynet-autotuner_records.json mynet.onnx

Then I tried to use that log to generate an optimized C code. Before I tried generating an unoptimized C code (without apply_history_best), I was successful using the following lines of code:

with tvm.transform.PassContext(opt_level=3):
    graph, lib, params = relay.build(mod, target="c", params=params)
    out_file = open("unoptimized.c", "w")
    out_file.write(lib.get_source())
    out_file.close()

However, when I tried generating an optimized C code (with apply_history_best), I was not successful and got the following error:

tvm._ffi.base.TVMError: Traceback (most recent call last):
  99: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  98: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::LetStmtNode const*)
  97: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  96: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::AttrStmtNode const*)
  95: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  94: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::LetStmtNode const*)
  93: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  92: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::LetStmtNode const*)
  91: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  90: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::LetStmtNode const*)
  89: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  88: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::AttrStmtNode const*)
  87: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  86: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::LetStmtNode const*)
  85: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  84: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::LetStmtNode const*)
  83: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  82: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::LetStmtNode const*)
  81: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  80: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::AttrStmtNode const*)
  79: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  78: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::LetStmtNode const*)
  77: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  76: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::LetStmtNode const*)
  75: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  74: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  73: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  72: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  71: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  70: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  69: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  68: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  67: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  66: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  65: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  64: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  63: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  62: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::SeqStmtNode const*)
  61: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  60: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  59: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  58: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  57: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  56: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  55: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  54: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  53: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  52: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  51: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  50: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::SeqStmtNode const*)
  49: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  48: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  47: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  46: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  45: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  44: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  43: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  42: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  41: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  40: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  39: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  38: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::SeqStmtNode const*)
  37: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  36: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  35: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  34: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  33: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  32: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  31: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  30: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  29: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  28: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  27: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  26: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::SeqStmtNode const*)
  25: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  24: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  23: tvm::codegen::CodeGenCHost::VisitStmt_(tvm::tir::AssertStmtNode const*)
  22: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::AttrStmtNode const*)
  21: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  20: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::AttrStmtNode const*)
  19: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  18: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::SeqStmtNode const*)
  17: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::LetStmtNode const*)
  16: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  15: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::SeqStmtNode const*)
  14: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::SeqStmtNode const*)
  13: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::ForNode const*)
  12: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  11: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::ForNode const*)
  10: tvm::tir::StmtFunctor<void (tvm::tir::Stmt const&)>::VisitStmt(tvm::tir::Stmt const&)
  9: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::StoreNode const*)
  8: tvm::codegen::CodeGenC::PrintExpr[abi:cxx11](tvm::PrimExpr const&)
  7: tvm::codegen::CodeGenC::PrintExpr(tvm::PrimExpr const&, std::ostream&)
  6: tvm::codegen::CodeGenCHost::VisitExpr_(tvm::tir::CallNode const*, std::ostream&)
  5: tvm::codegen::CodeGenC::VisitExpr_(tvm::tir::CallNode const*, std::ostream&)
  4: tvm::codegen::CodeGenC::PrintExpr(tvm::PrimExpr const&, std::ostream&)
  3: tvm::codegen::CodeGenC::VisitExpr_(tvm::tir::LoadNode const*, std::ostream&)
  2: tvm::codegen::CodeGenC::GetVecLoad[abi:cxx11](tvm::runtime::DataType, tvm::tir::VarNode const*, tvm::PrimExpr)
  1: tvm::codegen::CodeGenC::GetBufferRef[abi:cxx11](tvm::runtime::DataType, tvm::tir::VarNode const*, tvm::PrimExpr)
  0: tvm::codegen::CodeGenCHost::PrintType(tvm::runtime::DataType, std::ostream&)
  File "../src/target/source/codegen_c_host.cc", line 185
TVMError: Cannot convert type float32x24 to C type

The code I used looks like this:

log_file = "mynet-autotuner_records.json"
with autotvm.apply_history_best(log_file):
    print("Compile...")
    with tvm.transform.PassContext(opt_level=3):
        graph, lib, params = relay.build(mod, target="c", params=params)
        out_file = open("optimized.c", "w")
        out_file.write(lib.get_source())
        out_file.close()

and entries in the log looked like:

{"input": ["c -keys=cpu -link-params=0", "dense_pack.x86", [["TENSOR", [1, 1280], "float32"], ["TENSOR", [2, 1280], "float32"], null, "float32"], {}], "config": {"index": 29, "code_hash": null, "entity": [["tile_y", "sp", [-1, 1, 1]], ["tile_x", "sp", [-1, 1, 2]], ["tile_k", "sp", [-1, 40]], ["tile_inner", "sp", [-1, 1]]]}, "result": [[1.8415000000000003e-06], 0, 0.11197471618652344, 1629165541.5440676], "version": 0.2, "tvm_version": "0.8.dev0"}

Is this not supposed to work this way? Or is there a way to fix this?

Mousius commented 3 years ago

Hi @bhahn221,

This is a known issue with generating C code from TVM, it defaults to trying to use vector types which aren't supported - if you add --pass-config tir.disable_vectorize=1 to your tvmc command line it'll stop trying to do this and should generate valid C :smile_cat:

Mousius commented 3 years ago

CC @leandron / @areusch

leandron commented 3 years ago

Hi @bhahn221. I had a look into this issue and it seems like some similar issues we've seen in past with tvmc compile --target=c.

Based on the error message, it looks like we need to create a way to disable the vectorizer when using the C backend in this case? This is currently available via the --pass-config, only in tvmc compile.

https://github.com/apache/tvm/blob/e7748aac40bd4c263882323393ea8896837614a9/python/tvm/driver/tvmc/compiler.py#L86-L93

To do that in tvmc tune, my first impulse would be plumbing a similar --pass-config option, all the way from tvmc tune () to call_all_topic_funcs in tvm/auto_scheduler/relay_integration.py:

https://github.com/apache/tvm/blob/e7748aac40bd4c263882323393ea8896837614a9/python/tvm/auto_scheduler/relay_integration.py#L59-L66

I couldn't find the autotvm equivalent, on where the PassContext is actually called, but for now, in summary you would need to patch TVM to disable the vectorizer in the sources.

Mousius commented 3 years ago

Apologies @bhahn221, I think I misunderstood the initial issue, you can add the same pass config to your python code using:

with tvm.transform.PassContext(opt_level=3, config={"tir.disable_vectorize": True}):