Open willdla opened 11 months ago
also, to run tpat in my docker i change some code.
diff --git a/Dockerfile b/Dockerfile
index 243698e..f308fd9 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,4 +1,4 @@
-FROM nvcr.io/nvidia/tensorflow:20.06-tf1-py3
+FROM nvcr.nju.edu.cn/nvidia/tensorflow:20.06-tf1-py3
RUN wget -O "llvm-9.0.1.src.tar.xz" https://github.com/llvm/llvm-project/releases/download/llvmorg-9.0.1/llvm-9.0.1.src.tar.xz \
&& tar -xvf llvm-9.0.1.src.tar.xz && mkdir llvm-9.0.1.src/build \
&& cd llvm-9.0.1.src/build && cmake -G "Unix Makefiles" -DLLVM_TARGETS_TO_BUILD=X86 -DCMAKE_BUILD_TYPE="Release" -DCMAKE_INSTALL_PREFIX="/usr/local/llvm" .. && make -j8 && make install PREFIX="/usr/local/llvm"
diff --git a/examples/test_onehot_dynamic_direct.py b/examples/test_onehot_dynamic_direct.py
index 881fc0e..fd564e8 100644
--- a/examples/test_onehot_dynamic_direct.py
+++ b/examples/test_onehot_dynamic_direct.py
@@ -233,6 +233,7 @@ def main():
#trt_plugin_names = ['tpat_test_onehot_dynamic']
for trt_plugin_name in trt_plugin_names:
assert os.path.isfile(f"./trt_plugin/lib/{trt_plugin_name}.so")
+ print("load ./trt_plugin/lib/{}".format(trt_plugin_name))
ctypes.cdll.LoadLibrary("./trt_plugin/lib/{}.so".format(trt_plugin_name))
# build trt model by onnx model
diff --git a/python/onnx_modified.py b/python/onnx_modified.py
index 842808f..b55d8a2 100644
--- a/python/onnx_modified.py
+++ b/python/onnx_modified.py
@@ -122,7 +122,7 @@ class OnnxModified(object):
][0]
for inp in inferred_tuning_node.inputs:
if inp.__class__ == gs.Constant:
- onnx_original_tensor_type[inp.name] = inp.dtype.__name__
+ onnx_original_tensor_type[inp.name] = inp.dtype.name
elif not inp.is_empty():
onnx_original_tensor_type[inp.name] = inp.dtype.name
[
diff --git a/python/plugin_template_params.py b/python/plugin_template_params.py
index a7ba4dd..933b86d 100644
--- a/python/plugin_template_params.py
+++ b/python/plugin_template_params.py
@@ -212,9 +212,9 @@ class PluginTemplateParams(object):
tuning_node = tuning_nodes[0]
for inp in tuning_node.inputs:
if inp.__class__ == gs.Constant:
- self._onnx_input_python_type.append(tvm_to_c_type_mapping[inp.dtype.__name__])
+ self._onnx_input_python_type.append(tvm_to_c_type_mapping[inp.dtype.name])
self._onnx_tensor_type.append(
- python_to_trt_type_mapping[inp.dtype.__name__]
+ python_to_trt_type_mapping[inp.dtype.name]
)
elif not inp.is_empty():
self._onnx_input_python_type.append(tvm_to_c_type_mapping[inp.dtype.name])
diff --git a/python/trt_plugin/Makefile b/python/trt_plugin/Makefile
index ead82b4..b9a9591 100644
--- a/python/trt_plugin/Makefile
+++ b/python/trt_plugin/Makefile
@@ -16,7 +16,7 @@
CUDA_PATH = /usr/local/cuda-11.0/
#TRT_LIB_PATH = /root/workspace/download/TensorRT-7.2.2.3/lib
-TRT_LIB_PATH = /root/workspace/download/ft_local/TensorRT-8.0.0.3/lib
+TRT_LIB_PATH = /workspace/TensorRT-8.0.3.4/lib^M
Is there any update
@buptqq
@willdla It looks like that your program doesn't load tpat_test_onehot.so
plugin correctly. Please make sure you have tpat_test_onehost.so
in dir python/trt_plugin/lib/
and ctypes.cdll.LoadLibrary("./trt_plugin/lib/{}.so".format(trt_plugin_name))
loads it.
@willdla It looks like that your program doesn't load
tpat_test_onehot.so
plugin correctly. Please make sure you havetpat_test_onehost.so
in dirpython/trt_plugin/lib/
andctypes.cdll.LoadLibrary("./trt_plugin/lib/{}.so".format(trt_plugin_name))
loads it.
so在对应的位置:
python/trt_plugin/lib/
└── tpat_test_onehot.so
默认代码是有这个语句的 我这边也没做任何修改
ctypes.cdll.LoadLibrary("./trt_plugin/lib/{}.so".format(trt_plugin_name))
@willdla test_onehot_dynamic_direct.py
is in example directory. maybe you shold exec python test_onehot_dynamic_direct.py
in $ROOT/python/
directory or change ctypes.cdll.LoadLibrary("./trt_plugin/lib/{}.so".format(trt_plugin_name))
to ctypes.cdll.LoadLibrary("../python/trt_plugin/lib/{}.so".format(trt_plugin_name))
@willdla
test_onehot_dynamic_direct.py
is in example directory. maybe you shold execpython test_onehot_dynamic_direct.py
in$ROOT/python/
directory or changectypes.cdll.LoadLibrary("./trt_plugin/lib/{}.so".format(trt_plugin_name))
toctypes.cdll.LoadLibrary("../python/trt_plugin/lib/{}.so".format(trt_plugin_name))
两种方法都不起作用 报错和最开始相同
本地是否验证过可行?
@wenqf11
@willdla 这种情况一般是编译plugin的TensorRT版本与实际用的tensorRT版本不一致导致的,另外check一下MakeFile里的sm_XX与你的机型是否相同, V100(sm_70),A100(sm_80),A10(sm_86)
running in the docker created by docker file
get:
seems plugin not load properly.
how to fix it.
see full log below