Check failed: (pval != nullptr) is false: Cannot allocate memory symbolic tensor shape [?, 1]

[Itto1992]

I tried to run a script in a tutorial and encountered the error in the title.

Check failed: (pval != nullptr) is false: Cannot allocate memory symbolic tensor shape [?, 1]

This error was raised in the following block:

https://github.com/renesas-rz/rzv_drp-ai_tvm/blob/655000f7568aa61dc8b88f9a337481f6f6b71c68/tutorials/compile_onnx_model.py#L91-L97

When I use TVM, this sort of error can be avoided by using tvm.relay.vm.compile instead of tvm.relay.build_module.build. However, I cannot use the trick in this sample script.

How can I solve this problem? Thank you in advance!

[simobepu]

Hello Itto1992-san, Your AI model may contain dynamic shapes. DRP-AI TVM does not support dynamic shapes yet. Is it possible to try changing it to a static shape? Also, is it possible to share ONNX?

[Itto1992]

simobepu-san, Thank you for your quick reply!

Your AI model may contain dynamic shapes. DRP-AI TVM does not support dynamic shapes yet. Is it possible to try changing it to a static shape?

Since I am new to TVM, I have not understood what dynamic shapes are completely. I fix the input shape when converting the AI model into onnx. Isn't it enough to fix dynamic shapes? If you know some reference about the definition of dynamic shapes, could you share it?

Also, is it possible to share ONNX?

Here is the google drive link of my model. I appreciate all your help!

[Itto1992]

@simobepu After removing NMS operation from the model, I successfully avoided the error above.

However, I encountered another error like:

[DRP converter] Start
  * Input file names
    -A ./temp/20240112_040418/tvmgen_default_tvmgen_default_mera_drp_main_0/drp_compilation_output/out_tbl_addr_merge.txt
    -W ./temp/20240112_040418/tvmgen_default_tvmgen_default_mera_drp_main_0/drp_compilation_output/out_tbl_addr_weight.txt
    -D ./temp/20240112_040418/tvmgen_default_tvmgen_default_mera_drp_main_0/drp_compilation_output/out_tbl_addr_drp_config.txt
    -API ./temp/20240112_040418/tvmgen_default_tvmgen_default_mera_drp_main_0/drp_compilation_output/out.json
    -AMAP ./temp/20240112_040418/tvmgen_default_tvmgen_default_mera_drp_main_0/drp_compilation_output/out_addrmap_intm.yaml
    -OUT ./temp/20240112_040418/tvmgen_default_tvmgen_default_mera_drp_main_0/drp_compilation_output
  * Start API.json File Analyze
  * Write DRP lib info(./temp/20240112_040418/tvmgen_default_tvmgen_default_mera_drp_main_0/drp_compilation_output/drp_lib_info.txt)
[DRP converter] Success & Finished
[Generate address map file] Start
  > aimac_desc.bin size :   0x60ad0 (395984 Byte)
  > drp_desc.bin size   :    0x1de0 (7648 Byte)
  > drp_param.bin size  :    0x2360 (9056 Byte)

  [Address & data alignment check] PASS
  [Check address map overlap (memory leak)] PASS
  Address map file: ./temp/20240112_040418/tvmgen_default_tvmgen_default_mera_drp_main_0/drp_compilation_output/out_addrmap_intm.yaml is generated.
[Generate address map file] Finish
[Make Input/Output node information]
  Input node list : ./temp/20240112_040418/tvmgen_default_tvmgen_default_mera_drp_main_0/drp_compilation_output/out_data_in_list.txt is generated.
  Output node list : ./temp/20240112_040418/tvmgen_default_tvmgen_default_mera_drp_main_0/drp_compilation_output/out_data_out_list.txt is generated.
---------------------------------------------

At sub graph name tvmgen_default_tvmgen_default_mera_drp_main_0, required drp-ai memory: 161(Mbyte)
Function name: tvmgen_default_tvmgen_default_mera_drp_main_205
Traceback (most recent call last):
  File "compile_onnx_model.py", line 91, in <module>
    drp.build(mod, \
  File "python/tvm/relay/mera/drp/build.py", line 240, in tvm.relay.mera.drp.build.build
  File "python/tvm/relay/mera/drp/build.py", line 241, in tvm.relay.mera.drp.build.build
  File "/drp-ai_tvm/tvm/python/tvm/relay/build_module.py", line 364, in build
    graph_json, runtime_mod, params = bld_mod.build(
  File "/drp-ai_tvm/tvm/python/tvm/relay/build_module.py", line 161, in build
    self._build(
  File "/drp-ai_tvm/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 237, in __call__
    raise get_last_ffi_error()
tvm._ffi.base.TVMError: Traceback (most recent call last):
  16: TVMFuncCall
  15: tvm::relay::backend::RelayBuildModule::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#3}::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
  14: tvm::relay::backend::RelayBuildModule::BuildRelay(tvm::IRModule, tvm::runtime::String const&)
  13: tvm::runtime::PackedFuncObj::Extractor<tvm::runtime::PackedFuncSubObj<tvm::relay::backend::GraphExecutorCodegenModule::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#2}> >::Call(tvm::runtime::PackedFuncObj const*, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)
  12: tvm::relay::backend::GraphExecutorCodegen::Codegen(tvm::IRModule, tvm::relay::Function, tvm::runtime::String)
  11: tvm::transform::Pass::operator()(tvm::IRModule) const
  10: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const
  9: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const
  8: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const
  7: tvm::transform::ModulePassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const
  6: _ZN3tvm7runtime13PackedFuncObj9ExtractorINS0_16PackedFuncSubObjIZNS0_15TypedPackedFuncIFNS_8IRModuleES5_NS_9transform11PassContextEEE17AssignTypedLambdaIZNS_5relay3tec7LowerTEENS0_6StringENS_17CompilationConfigESt8functionIFvNS_8BaseFuncEEEEUlS5_S7_E_EEvT_EUlRKNS0_7TVMArgsEPNS0_11TVMRetValueEE_EEE4CallEPKS1_SL_SP_
  5: tvm::relay::tec::LowerTE(tvm::IRModule const&, tvm::runtime::String const&, std::function<void (tvm::BaseFunc)>, tvm::CompilationConfig)
  4: tvm::relay::tec::TECompilerImpl::LowerExternalFunctions()
  3: tvm::runtime::PackedFuncObj::Extractor<tvm::runtime::PackedFuncSubObj<tvm::runtime::TypedPackedFunc<tvm::runtime::Module (tvm::runtime::ObjectRef const&)>::AssignTypedLambda<tvm::runtime::Module (*)(tvm::runtime::ObjectRef const&)>(tvm::runtime::Module (*)(tvm::runtime::ObjectRef const&), std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}> >::Call(tvm::runtime::PackedFuncObj const*, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)
  2: tvm::relay::contrib::MERADRPCompiler(tvm::runtime::ObjectRef const&)
  1: tvm::relay::contrib::CompileModule(tvm::runtime::ObjectRef const&)
  0: tvm::relay::contrib::Compiler::NetworkInputType[abi:cxx11](int)
  File "/root/drp-tvm-share/src/relay/backend/contrib/mera/drp/mera_drp_codegen.cc", line 1073
TVMError: Only 3d or 1d input tensors are supported by drp translator

I am using Conv3D in the model. Is it related to this error?

[Itto1992]

@simobepu After changing the implementation of my model, this error was solved! Thank you for your advice!