search

NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.67k stars 2.12k forks source link

trtext failure of TensorRT 8.4.1 when running trtext --safe --calib on orin #3138

Closed fucker007 closed 1 year ago

fucker007 commented 1 year ago

Description

I try to run 

./trtexec --onnx=model.onnx --useDLACore=0 --safe --buildOnly --calib=calib.txt --int8 --inputIOFormats=int8:chw --outputIOFormats=int8:chw

hever I get this error: 

[10/01/2019-05:25:28] [I] FP32 and INT8 precisions have been specified - more performance might be enabled by additionally specifying --fp16 or --best
[10/01/2019-05:25:28] [E] Error[2]: [nvmRegionOptimizer.cpp::forceToUseNvmIO::175] Error Code 2: Internal Error (Assertion std::all_of(a->consumers.begin(), a->consumers.end(), [](Node* n) { return isDLA(n->backend); }) failed. )
[10/01/2019-05:25:28] [E] Error[2]: [builder.cpp::buildSerializedNetwork::619] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[10/01/2019-05:25:28] [E] Engine could not be created from network
[10/01/2019-05:25:28] [E] Building engine failed
[10/01/2019-05:25:29] [E] Failed to create engine from model or file.
[10/01/2019-05:25:29] [E] Engine set up failed

if I remove --safe option, it's work well, is suportting quantization on safe mode of tensorRT? I check the code, the daynamicRange can work well, but not work on --calib.
any suggestion is good, best wish.

Environment

TensorRT Version: 8.4.1

NVIDIA GPU: DLACore

NVIDIA Driver Version:

CUDA Version: 11.4

CUDNN Version: 8.4

zerollzeng commented 1 year ago

Could you please try TRT 8.5? e.g. 6.0.6.0, we switch from NvmediaDLA to cuDLA since that version.

fucker007 commented 1 year ago

I have try TensorRT 8.5.3.1, However I get this error. 

In file included from ../common/common.h:60,
                 from ../common/buffers.h:21,
                 from sampleAlgorithmSelector.cpp:29:
../common/safeCommon.h: In function 'uint32_t samplesCommon::elementSize(nvinfer1::DataType)':
../common/safeCommon.h:83:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8: return 1;
                              ^~~~~~
In file included from ../common/common.h:60,
                 from ../common/buffers.h:21,
                 from sampleAlgorithmSelector.cpp:29:
../common/safeCommon.h: In function 'uint32_t samplesCommon::elementSize(nvinfer1::DataType)':
../common/safeCommon.h:83:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8: return 1;
                              ^~~~~~
In file included from ../common/buffers.h:21,
                 from sampleAlgorithmSelector.cpp:29:
../common/common.h: In function 'uint32_t samplesCommon::getElementSize(nvinfer1::DataType)':
../common/common.h:637:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8:
                              ^~~~~~
In file included from ../common/buffers.h:21,
                 from sampleAlgorithmSelector.cpp:29:
../common/common.h: In function 'uint32_t samplesCommon::getElementSize(nvinfer1::DataType)':
../common/common.h:637:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8:
                              ^~~~~~
In file included from ../common/common.h:60,
                 from ../common/sampleUtils.h:35,
                 from ../common/sampleOptions.cpp:31:
../common/safeCommon.h: In function 'uint32_t samplesCommon::elementSize(nvinfer1::DataType)':
../common/safeCommon.h:83:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8: return 1;
                              ^~~~~~
In file included from ../common/common.h:60,
                 from ../common/sampleUtils.h:35,
                 from ../common/sampleEngines.h:31,
                 from ../common/sampleInference.h:21,
                 from ../common/sampleReporting.cpp:26:
../common/safeCommon.h: In function 'uint32_t samplesCommon::elementSize(nvinfer1::DataType)':
../common/safeCommon.h:83:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8: return 1;
                              ^~~~~~
In file included from ../common/common.h:60,
                 from ../common/sampleUtils.h:35,
                 from ../common/sampleUtils.cpp:18:
../common/safeCommon.h: In function 'uint32_t samplesCommon::elementSize(nvinfer1::DataType)':
../common/safeCommon.h:83:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8: return 1;
                              ^~~~~~
In file included from ../common/common.h:60,
                 from ../common/sampleEngines.cpp:35:
../common/safeCommon.h: In function 'uint32_t samplesCommon::elementSize(nvinfer1::DataType)':
../common/safeCommon.h:83:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8: return 1;
                              ^~~~~~
In file included from ../common/common.h:60,
                 from ../common/sampleUtils.h:35,
                 from ../common/sampleDevice.h:27,
                 from ../common/sampleInference.cpp:40:
../common/safeCommon.h: In function 'uint32_t samplesCommon::elementSize(nvinfer1::DataType)':
../common/safeCommon.h:83:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8: return 1;
                              ^~~~~~
In file included from ../common/sampleUtils.h:35,
                 from ../common/sampleOptions.cpp:31:
../common/common.h: In function 'uint32_t samplesCommon::getElementSize(nvinfer1::DataType)':
../common/common.h:637:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8:
                              ^~~~~~
In file included from ../common/common.h:60,
                 from ../common/sampleUtils.h:35,
                 from ../common/sampleDevice.h:27,
                 from ../common/sampleInference.cpp:40:
../common/safeCommon.h: In function 'uint32_t samplesCommon::elementSize(nvinfer1::DataType)':
../common/safeCommon.h:83:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8: return 1;
                              ^~~~~~
In file included from ../common/sampleUtils.h:35,
                 from ../common/sampleEngines.h:31,
                 from ../common/sampleInference.h:21,
                 from ../common/sampleReporting.cpp:26:
../common/common.h: In function 'uint32_t samplesCommon::getElementSize(nvinfer1::DataType)':
../common/common.h:637:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8:
                              ^~~~~~
In file included from ../common/sampleUtils.h:35,
                 from ../common/sampleUtils.cpp:18:
../common/common.h: In function 'uint32_t samplesCommon::getElementSize(nvinfer1::DataType)':
../common/common.h:637:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8:
                              ^~~~~~
In file included from ../common/sampleEngines.cpp:35:
../common/common.h: In function 'uint32_t samplesCommon::getElementSize(nvinfer1::DataType)':
../common/common.h:637:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8:
                              ^~~~~~
In file included from ../common/sampleUtils.h:35,
                 from ../common/sampleDevice.h:27,
                 from ../common/sampleInference.cpp:40:
../common/common.h: In function 'uint32_t samplesCommon::getElementSize(nvinfer1::DataType)':
../common/common.h:637:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8:
                              ^~~~~~
In file included from ../common/sampleUtils.h:35,
                 from ../common/sampleDevice.h:27,
                 from ../common/sampleInference.cpp:40:
../common/common.h: In function 'uint32_t samplesCommon::getElementSize(nvinfer1::DataType)':
../common/common.h:637:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8:
                              ^~~~~~
../common/sampleUtils.cpp: In function 'size_t sample::dataTypeSize(nvinfer1::DataType)':
../common/sampleUtils.cpp:34:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8:
                              ^~~~~~
In file included from ../common/sampleReporting.h:27,
                 from ../common/sampleInference.h:22,
                 from ../common/sampleReporting.cpp:26:
../common/sampleDevice.h: At global scope:
../common/sampleDevice.h:479:1: error: expected class-name before '{' token
 {
 ^
../common/sampleDevice.h:486:11: error: 'void* sample::OutputAllocator::reallocateOutput(const char*, void*, uint64_t, uint64_t)' marked 'override', but does not override
     void* reallocateOutput(
           ^~~~~~~~~~~~~~~~
../common/sampleDevice.h:500:10: error: 'void sample::OutputAllocator::notifyShape(const char*, const Dims&)' marked 'override', but does not override
     void notifyShape(char const* tensorName, nvinfer1::Dims const& dims) noexcept override {}
          ^~~~~~~~~~~
In file included from ../common/sampleInference.cpp:40:
../common/sampleDevice.h: At global scope:
../common/sampleDevice.h:479:1: error: expected class-name before '{' token
 {
 ^
../common/sampleDevice.h:486:11: error: 'void* sample::OutputAllocator::reallocateOutput(const char*, void*, uint64_t, uint64_t)' marked 'override', but does not override
     void* reallocateOutput(
           ^~~~~~~~~~~~~~~~
../common/sampleDevice.h:500:10: error: 'void sample::OutputAllocator::notifyShape(const char*, const Dims&)' marked 'override', but does not override
     void notifyShape(char const* tensorName, nvinfer1::Dims const& dims) noexcept override {}
          ^~~~~~~~~~~
In file included from ../common/sampleEngines.cpp:38:
../common/sampleDevice.h: At global scope:
../common/sampleDevice.h:479:1: error: expected class-name before '{' token
 {
 ^
../common/sampleDevice.h:486:11: error: 'void* sample::OutputAllocator::reallocateOutput(const char*, void*, uint64_t, uint64_t)' marked 'override', but does not override
     void* reallocateOutput(
           ^~~~~~~~~~~~~~~~
../common/sampleDevice.h:500:10: error: 'void sample::OutputAllocator::notifyShape(const char*, const Dims&)' marked 'override', but does not override
     void notifyShape(char const* tensorName, nvinfer1::Dims const& dims) noexcept override {}
          ^~~~~~~~~~~
../common/sampleOptions.cpp: In function 'std::__1::ostream& sample::{anonymous}::printTacticSources(std::__1::ostream&, nvinfer1::TacticSources, nvinfer1::TacticSources)':
../common/sampleOptions.cpp:519:71: error: 'kEDGE_MASK_CONVOLUTIONS' is not a member of 'nvinfer1::TacticSource'
         addSource(1U << static_cast<uint32_t>(nvinfer1::TacticSource::kEDGE_MASK_CONVOLUTIONS), "edge mask convolutions");
                                                                       ^~~~~~~~~~~~~~~~~~~~~~~
../common/sampleOptions.cpp:520:71: error: 'kJIT_CONVOLUTIONS' is not a member of 'nvinfer1::TacticSource'
         addSource(1U << static_cast<uint32_t>(nvinfer1::TacticSource::kJIT_CONVOLUTIONS), "JIT convolutions");
                                                                       ^~~~~~~~~~~~~~~~~
In file included from ../common/sampleInference.cpp:40:
../common/sampleDevice.h: At global scope:
../common/sampleDevice.h:479:1: error: expected class-name before '{' token
 {
 ^
../common/sampleDevice.h:486:11: error: 'void* sample::OutputAllocator::reallocateOutput(const char*, void*, uint64_t, uint64_t)' marked 'override', but does not override
     void* reallocateOutput(
           ^~~~~~~~~~~~~~~~
../common/sampleDevice.h:500:10: error: 'void sample::OutputAllocator::notifyShape(const char*, const Dims&)' marked 'override', but does not override
     void notifyShape(char const* tensorName, nvinfer1::Dims const& dims) noexcept override {}
          ^~~~~~~~~~~
../common/sampleOptions.cpp: At global scope:
../common/sampleOptions.cpp:596:36: error: 'PreviewFeature' was not declared in this scope
 std::string previewFeatureToString(PreviewFeature feature)
                                    ^~~~~~~~~~~~~~
../common/sampleOptions.cpp:596:36: note: suggested alternative: 'previewFeatureToString'
 std::string previewFeatureToString(PreviewFeature feature)
                                    ^~~~~~~~~~~~~~
                                    previewFeatureToString
../common/sampleOptions.cpp:597:1: error: expected ',' or ';' before '{' token
 {
 ^
../common/sampleOptions.cpp: In function 'std::__1::ostream& sample::{anonymous}::printPreviewFlags(std::__1::ostream&, const sample::BuildOptions&)':
../common/sampleOptions.cpp:616:30: error: 'PreviewFeature' has not been declared
     auto const addFlag = [&](PreviewFeature feat) {
                              ^~~~~~~~~~~~~~
../common/sampleOptions.cpp: In lambda function:
../common/sampleOptions.cpp:620:46: error: no match for call to '(std::__1::string {aka std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >}) (int&)'
             os << previewFeatureToString(feat) << (options.previewFeatures.at(featVal) ? " [ON], " : " [OFF], ");
                                              ^
../common/sampleOptions.cpp: In function 'std::__1::ostream& sample::{anonymous}::printPreviewFlags(std::__1::ostream&, const sample::BuildOptions&)':
../common/sampleOptions.cpp:624:13: error: 'PreviewFeature' has not been declared
     addFlag(PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805);
             ^~~~~~~~~~~~~~
../common/sampleOptions.cpp:625:13: error: 'PreviewFeature' has not been declared
     addFlag(PreviewFeature::kDISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805);
             ^~~~~~~~~~~~~~
../common/sampleOptions.cpp: In member function 'virtual void sample::BuildOptions::parse(sample::Arguments&)':
../common/sampleOptions.cpp:987:50: error: 'kEDGE_MASK_CONVOLUTIONS' is not a member of 'nvinfer1::TacticSource'
                 source = nvinfer1::TacticSource::kEDGE_MASK_CONVOLUTIONS;
                                                  ^~~~~~~~~~~~~~~~~~~~~~~
../common/sampleOptions.cpp:991:50: error: 'kJIT_CONVOLUTIONS' is not a member of 'nvinfer1::TacticSource'
                 source = nvinfer1::TacticSource::kJIT_CONVOLUTIONS;
                                                  ^~~~~~~~~~~~~~~~~
../common/sampleOptions.cpp:1051:9: error: 'PreviewFeature' was not declared in this scope
         PreviewFeature feat{};
         ^~~~~~~~~~~~~~
../common/sampleOptions.cpp:1051:9: note: suggested alternative: 'previewFeatures'
         PreviewFeature feat{};
         ^~~~~~~~~~~~~~
         previewFeatures
../common/sampleOptions.cpp:1054:13: error: 'feat' was not declared in this scope
             feat = PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805;
             ^~~~
../common/sampleOptions.cpp:1054:13: note: suggested alternative: 'read'
             feat = PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805;
             ^~~~
             read
../common/sampleOptions.cpp:1054:20: error: 'PreviewFeature' is not a class, namespace, or enumeration
             feat = PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805;
                    ^~~~~~~~~~~~~~
../common/sampleOptions.cpp:1058:13: error: 'feat' was not declared in this scope
             feat = PreviewFeature::kDISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805;
             ^~~~
../common/sampleOptions.cpp:1058:13: note: suggested alternative: 'read'
             feat = PreviewFeature::kDISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805;
             ^~~~
             read
../common/sampleOptions.cpp:1058:20: error: 'PreviewFeature' is not a class, namespace, or enumeration
             feat = PreviewFeature::kDISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805;
                    ^~~~~~~~~~~~~~
../common/sampleOptions.cpp:1064:46: error: 'feat' was not declared in this scope
         previewFeatures[static_cast<int32_t>(feat)] = enable;
                                              ^~~~
../common/sampleOptions.cpp:1064:46: note: suggested alternative: 'read'
         previewFeatures[static_cast<int32_t>(feat)] = enable;
                                              ^~~~
                                              read
../common/sampleUtils.cpp: In function 'void sample::sparsify(const nvinfer1::Weights&, int32_t, int32_t, std::__1::vector<signed char>&)':
../common/sampleUtils.cpp:370:20: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case DataType::kUINT8:
                    ^~~~~~
../common/sampleOptions.cpp: In function 'std::__1::ostream& sample::operator<<(std::__1::ostream&, nvinfer1::DataType)':
../common/sampleOptions.cpp:1416:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8:
                              ^~~~~~
../common/sampleEngines.cpp: In function 'void sample::{anonymous}::setPreviewFeatures(nvinfer1::IBuilderConfig&, const sample::BuildOptions&)':
../common/sampleEngines.cpp:584:30: error: 'PreviewFeature' has not been declared
     auto const setFlag = [&](PreviewFeature feat) {
                              ^~~~~~~~~~~~~~
../common/sampleEngines.cpp: In lambda function:
../common/sampleEngines.cpp:588:20: error: 'class nvinfer1::IBuilderConfig' has no member named 'setPreviewFeature'
             config.setPreviewFeature(feat, build.previewFeatures.at(featVal));
                    ^~~~~~~~~~~~~~~~~
../common/sampleEngines.cpp: In function 'void sample::{anonymous}::setPreviewFeatures(nvinfer1::IBuilderConfig&, const sample::BuildOptions&)':
../common/sampleEngines.cpp:591:13: error: 'PreviewFeature' has not been declared
     setFlag(PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805);
             ^~~~~~~~~~~~~~
../common/sampleEngines.cpp:592:13: error: 'PreviewFeature' has not been declared
     setFlag(PreviewFeature::kDISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805);
             ^~~~~~~~~~~~~~
../common/sampleEngines.cpp: In function 'bool sample::setupNetworkAndConfig(const sample::BuildOptions&, const sample::SystemOptions&, nvinfer1::IBuilder&, nvinfer1::INetworkDefinition&, nvinfer1::IBuilderConfig&, std::__1::unique_ptr<nvinfer1::IInt8Calibrator>&, std::__1::ostream&, std::__1::vector<std::__1::vector<signed char> >&)':
../common/sampleEngines.cpp:656:28: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
             case DataType::kUINT8:
                            ^~~~~~
../common/sampleInference.cpp: In member function 'void sample::FillBindingClosure<EngineType, ContextType>::getTensorInfo(sample::TensorInfo&) [with EngineType = nvinfer1::ICudaEngine; ContextType = nvinfer1::IExecutionContext]':
../common/sampleInference.cpp:169:36: error: 'const class nvinfer1::IExecutionContext' has no member named 'getTensorShape'
         tensorInfo.dims = context->getTensorShape(name);
                                    ^~~~~~~~~~~~~~
../common/sampleInference.cpp:172:36: error: 'const class nvinfer1::ICudaEngine' has no member named 'getTensorComponentsPerElement'; did you mean 'getBindingComponentsPerElement'?
         tensorInfo.comps = engine->getTensorComponentsPerElement(name);
                                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                    getBindingComponentsPerElement
../common/sampleInference.cpp:173:39: error: 'const class nvinfer1::IExecutionContext' has no member named 'getTensorStrides'; did you mean 'getStrides'?
         tensorInfo.strides = context->getTensorStrides(name);
                                       ^~~~~~~~~~~~~~~~
                                       getStrides
../common/sampleInference.cpp:174:45: error: 'const class nvinfer1::ICudaEngine' has no member named 'getTensorVectorizedDim'; did you mean 'getBindingVectorizedDim'?
         tensorInfo.vectorDimIndex = engine->getTensorVectorizedDim(name);
                                             ^~~~~~~~~~~~~~~~~~~~~~
                                             getBindingVectorizedDim
../common/sampleInference.cpp:175:38: error: 'const class nvinfer1::ICudaEngine' has no member named 'getTensorIOMode'
         tensorInfo.isInput = engine->getTensorIOMode(name) == TensorIOMode::kINPUT;
                                      ^~~~~~~~~~~~~~~
../common/sampleInference.cpp:175:63: error: 'TensorIOMode' has not been declared
         tensorInfo.isInput = engine->getTensorIOMode(name) == TensorIOMode::kINPUT;
                                                               ^~~~~~~~~~~~
../common/sampleInference.cpp:176:39: error: 'const class nvinfer1::ICudaEngine' has no member named 'getTensorDataType'; did you mean 'getBindingDataType'?
         tensorInfo.dataType = engine->getTensorDataType(name);
                                       ^~~~~~~~~~~~~~~~~
                                       getBindingDataType
../common/sampleInference.cpp: In member function 'void sample::FillBindingClosure<EngineType, ContextType>::getTensorInfo(sample::TensorInfo&) [with EngineType = nvinfer1::safe::ICudaEngine; ContextType = nvinfer1::safe::IExecutionContext]':
../common/sampleInference.cpp:186:31: error: 'const class nvinfer1::safe::ICudaEngine' has no member named 'getIOTensorName'
     auto const name = engine->getIOTensorName(b);
                               ^~~~~~~~~~~~~~~
../common/sampleInference.cpp:188:31: error: 'const class nvinfer1::safe::ICudaEngine' has no member named 'getTensorShape'
     tensorInfo.dims = engine->getTensorShape(name);
                               ^~~~~~~~~~~~~~
../common/sampleInference.cpp:190:32: error: 'const class nvinfer1::safe::ICudaEngine' has no member named 'getTensorComponentsPerElement'; did you mean 'getBindingComponentsPerElement'?
     tensorInfo.comps = engine->getTensorComponentsPerElement(name);
                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                getBindingComponentsPerElement
../common/sampleInference.cpp:191:35: error: 'const class nvinfer1::safe::IExecutionContext' has no member named 'getTensorStrides'; did you mean 'getStrides'?
     tensorInfo.strides = context->getTensorStrides(name);
                                   ^~~~~~~~~~~~~~~~
                                   getStrides
../common/sampleInference.cpp:192:41: error: 'const class nvinfer1::safe::ICudaEngine' has no member named 'getTensorVectorizedDim'; did you mean 'getBindingVectorizedDim'?
     tensorInfo.vectorDimIndex = engine->getTensorVectorizedDim(name);
                                         ^~~~~~~~~~~~~~~~~~~~~~
                                         getBindingVectorizedDim
../common/sampleInference.cpp:193:34: error: 'const class nvinfer1::safe::ICudaEngine' has no member named 'getTensorIOMode'
     tensorInfo.isInput = engine->getTensorIOMode(name) == TensorIOMode::kINPUT;
                                  ^~~~~~~~~~~~~~~
../common/sampleInference.cpp:193:59: error: 'TensorIOMode' has not been declared
     tensorInfo.isInput = engine->getTensorIOMode(name) == TensorIOMode::kINPUT;
                                                           ^~~~~~~~~~~~
../common/sampleInference.cpp:194:35: error: 'const class nvinfer1::safe::ICudaEngine' has no member named 'getTensorDataType'; did you mean 'getBindingDataType'?
     tensorInfo.dataType = engine->getTensorDataType(name);
                                   ^~~~~~~~~~~~~~~~~
                                   getBindingDataType
../common/sampleEngines.cpp:803:37: error: 'kENABLE_TACTIC_HEURISTIC' is not a member of 'nvinfer1::BuilderFlag'
         config.setFlag(BuilderFlag::kENABLE_TACTIC_HEURISTIC);
                                     ^~~~~~~~~~~~~~~~~~~~~~~~
../common/sampleInference.cpp: In function 'bool sample::setUpInference(sample::InferenceEnvironment&, const sample::InferenceOptions&, const sample::SystemOptions&)':
../common/sampleInference.cpp:260:13: error: 'class nvinfer1::IExecutionContext' has no member named 'setNvtxVerbosity'
         ec->setNvtxVerbosity(inference.nvtxVerbosity);
             ^~~~~~~~~~~~~~~~
../common/sampleInference.cpp:265:13: error: 'class nvinfer1::IExecutionContext' has no member named 'setPersistentCacheLimit'
         ec->setPersistentCacheLimit(persistentCacheLimit);
             ^~~~~~~~~~~~~~~~~~~~~~~
../common/sampleInference.cpp:278:45: error: 'class nvinfer1::ICudaEngine' has no member named 'getNbIOTensors'; did you mean 'getNbLayers'?
     int32_t const endBindingIndex = engine->getNbIOTensors();
                                             ^~~~~~~~~~~~~~
                                             getNbLayers
../common/sampleInference.cpp:301:36: error: 'class nvinfer1::ICudaEngine' has no member named 'getIOTensorName'
         auto const& name = engine->getIOTensorName(b);
                                    ^~~~~~~~~~~~~~~
../common/sampleInference.cpp:302:36: error: 'class nvinfer1::ICudaEngine' has no member named 'getTensorIOMode'
         auto const& mode = engine->getTensorIOMode(name);
                                    ^~~~~~~~~~~~~~~
../common/sampleInference.cpp:303:21: error: 'TensorIOMode' has not been declared
         if (mode == TensorIOMode::kINPUT)
                     ^~~~~~~~~~~~
../common/sampleInference.cpp:305:54: error: 'class nvinfer1::IExecutionContext' has no member named 'getTensorShape'
             Dims const dims = iEnv.contexts.front()->getTensorShape(name);
                                                      ^~~~~~~~~~~~~~
../common/sampleInference.cpp:309:46: error: 'class nvinfer1::ICudaEngine' has no member named 'isShapeInferenceIO'
                 isShapeInferenceIO = engine->isShapeInferenceIO(name);
                                              ^~~~~~~~~~~~~~~~~~
../common/sampleInference.cpp:374:37: error: 'class nvinfer1::IExecutionContext' has no member named 'setTensorAddress'
                             if (!c->setTensorAddress(name, shapeTensorData))
                                     ^~~~~~~~~~~~~~~~
../common/sampleInference.cpp:381:37: error: 'class nvinfer1::IExecutionContext' has no member named 'setInputShape'; did you mean 'setInputShapeBinding'?
                             if (!c->setInputShape(name, toDims(shapeData)))
                                     ^~~~~~~~~~~~~
                                     setInputShapeBinding
../common/sampleInference.cpp: In member function 'void sample::FillBindingClosure<EngineType, ContextType>::getTensorInfo(sample::TensorInfo&) [with EngineType = nvinfer1::ICudaEngine; ContextType = nvinfer1::IExecutionContext]':
../common/sampleInference.cpp:169:36: error: 'const class nvinfer1::IExecutionContext' has no member named 'getTensorShape'
         tensorInfo.dims = context->getTensorShape(name);
                                    ^~~~~~~~~~~~~~
../common/sampleInference.cpp:172:36: error: 'const class nvinfer1::ICudaEngine' has no member named 'getTensorComponentsPerElement'; did you mean 'getBindingComponentsPerElement'?
         tensorInfo.comps = engine->getTensorComponentsPerElement(name);
                                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                    getBindingComponentsPerElement
../common/sampleInference.cpp:173:39: error: 'const class nvinfer1::IExecutionContext' has no member named 'getTensorStrides'; did you mean 'getStrides'?
         tensorInfo.strides = context->getTensorStrides(name);
                                       ^~~~~~~~~~~~~~~~
                                       getStrides
../common/sampleInference.cpp:174:45: error: 'const class nvinfer1::ICudaEngine' has no member named 'getTensorVectorizedDim'; did you mean 'getBindingVectorizedDim'?
         tensorInfo.vectorDimIndex = engine->getTensorVectorizedDim(name);
                                             ^~~~~~~~~~~~~~~~~~~~~~
                                             getBindingVectorizedDim
../common/sampleInference.cpp:175:38: error: 'const class nvinfer1::ICudaEngine' has no member named 'getTensorIOMode'
         tensorInfo.isInput = engine->getTensorIOMode(name) == TensorIOMode::kINPUT;
                                      ^~~~~~~~~~~~~~~
../common/sampleInference.cpp:175:63: error: 'TensorIOMode' has not been declared
         tensorInfo.isInput = engine->getTensorIOMode(name) == TensorIOMode::kINPUT;
                                                               ^~~~~~~~~~~~
../common/sampleInference.cpp:176:39: error: 'const class nvinfer1::ICudaEngine' has no member named 'getTensorDataType'; did you mean 'getBindingDataType'?
         tensorInfo.dataType = engine->getTensorDataType(name);
                                       ^~~~~~~~~~~~~~~~~
                                       getBindingDataType
../common/sampleInference.cpp: In member function 'void sample::FillBindingClosure<EngineType, ContextType>::getTensorInfo(sample::TensorInfo&) [with EngineType = nvinfer1::safe::ICudaEngine; ContextType = nvinfer1::safe::IExecutionContext]':
../common/sampleInference.cpp:186:31: error: 'const class nvinfer1::safe::ICudaEngine' has no member named 'getIOTensorName'
     auto const name = engine->getIOTensorName(b);
                               ^~~~~~~~~~~~~~~
../common/sampleInference.cpp:188:31: error: 'const class nvinfer1::safe::ICudaEngine' has no member named 'getTensorShape'
     tensorInfo.dims = engine->getTensorShape(name);
                               ^~~~~~~~~~~~~~
../common/sampleInference.cpp:190:32: error: 'const class nvinfer1::safe::ICudaEngine' has no member named 'getTensorComponentsPerElement'; did you mean 'getBindingComponentsPerElement'?
     tensorInfo.comps = engine->getTensorComponentsPerElement(name);
                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                getBindingComponentsPerElement
../common/sampleInference.cpp:191:35: error: 'const class nvinfer1::safe::IExecutionContext' has no member named 'getTensorStrides'; did you mean 'getStrides'?
     tensorInfo.strides = context->getTensorStrides(name);
                                   ^~~~~~~~~~~~~~~~
                                   getStrides
../common/sampleInference.cpp:192:41: error: 'const class nvinfer1::safe::ICudaEngine' has no member named 'getTensorVectorizedDim'; did you mean 'getBindingVectorizedDim'?
     tensorInfo.vectorDimIndex = engine->getTensorVectorizedDim(name);
                                         ^~~~~~~~~~~~~~~~~~~~~~
                                         getBindingVectorizedDim
../common/sampleInference.cpp:193:34: error: 'const class nvinfer1::safe::ICudaEngine' has no member named 'getTensorIOMode'
     tensorInfo.isInput = engine->getTensorIOMode(name) == TensorIOMode::kINPUT;
                                  ^~~~~~~~~~~~~~~
../common/sampleInference.cpp:193:59: error: 'TensorIOMode' has not been declared
     tensorInfo.isInput = engine->getTensorIOMode(name) == TensorIOMode::kINPUT;
                                                           ^~~~~~~~~~~~
../common/sampleInference.cpp:194:35: error: 'const class nvinfer1::safe::ICudaEngine' has no member named 'getTensorDataType'; did you mean 'getBindingDataType'?
     tensorInfo.dataType = engine->getTensorDataType(name);
                                   ^~~~~~~~~~~~~~~~~
                                   getBindingDataType
../common/sampleInference.cpp: In function 'bool sample::setUpInference(sample::InferenceEnvironment&, const sample::InferenceOptions&, const sample::SystemOptions&)':
../common/sampleInference.cpp:260:13: error: 'class nvinfer1::IExecutionContext' has no member named 'setNvtxVerbosity'
         ec->setNvtxVerbosity(inference.nvtxVerbosity);
             ^~~~~~~~~~~~~~~~
../common/sampleInference.cpp: In member function 'bool sample::{anonymous}::EnqueueExplicit::operator()(sample::TrtCudaStream&) const':
../common/sampleInference.cpp:529:22: error: 'class nvinfer1::IExecutionContext' has no member named 'enqueueV3'; did you mean 'enqueueV2'?
         if (mContext.enqueueV3(stream.get()))
                      ^~~~~~~~~
                      enqueueV2
../common/sampleInference.cpp:265:13: error: 'class nvinfer1::IExecutionContext' has no member named 'setPersistentCacheLimit'
         ec->setPersistentCacheLimit(persistentCacheLimit);
             ^~~~~~~~~~~~~~~~~~~~~~~
../common/sampleInference.cpp:278:45: error: 'class nvinfer1::ICudaEngine' has no member named 'getNbIOTensors'; did you mean 'getNbLayers'?
     int32_t const endBindingIndex = engine->getNbIOTensors();
                                             ^~~~~~~~~~~~~~
                                             getNbLayers
../common/sampleInference.cpp: In member function 'bool sample::{anonymous}::EnqueueSafe::operator()(sample::TrtCudaStream&) const':
../common/sampleInference.cpp:614:22: error: 'class nvinfer1::safe::IExecutionContext' has no member named 'enqueueV3'; did you mean 'enqueueV2'?
         if (mContext.enqueueV3(stream.get()))
                      ^~~~~~~~~
                      enqueueV2
../common/sampleInference.cpp:301:36: error: 'class nvinfer1::ICudaEngine' has no member named 'getIOTensorName'
         auto const& name = engine->getIOTensorName(b);
                                    ^~~~~~~~~~~~~~~
../common/sampleInference.cpp:302:36: error: 'class nvinfer1::ICudaEngine' has no member named 'getTensorIOMode'
         auto const& mode = engine->getTensorIOMode(name);
                                    ^~~~~~~~~~~~~~~
../common/sampleInference.cpp:303:21: error: 'TensorIOMode' has not been declared
         if (mode == TensorIOMode::kINPUT)
                     ^~~~~~~~~~~~
../common/sampleInference.cpp:305:54: error: 'class nvinfer1::IExecutionContext' has no member named 'getTensorShape'
             Dims const dims = iEnv.contexts.front()->getTensorShape(name);
                                                      ^~~~~~~~~~~~~~
../common/sampleInference.cpp:309:46: error: 'class nvinfer1::ICudaEngine' has no member named 'isShapeInferenceIO'
                 isShapeInferenceIO = engine->isShapeInferenceIO(name);
                                              ^~~~~~~~~~~~~~~~~~
../common/sampleInference.cpp:374:37: error: 'class nvinfer1::IExecutionContext' has no member named 'setTensorAddress'
                             if (!c->setTensorAddress(name, shapeTensorData))
                                     ^~~~~~~~~~~~~~~~
../common/sampleInference.cpp:381:37: error: 'class nvinfer1::IExecutionContext' has no member named 'setInputShape'; did you mean 'setInputShapeBinding'?
                             if (!c->setInputShape(name, toDims(shapeData)))
                                     ^~~~~~~~~~~~~
                                     setInputShapeBinding
../common/sampleInference.cpp: In member function 'bool sample::{anonymous}::EnqueueExplicit::operator()(sample::TrtCudaStream&) const':
../common/sampleInference.cpp:529:22: error: 'class nvinfer1::IExecutionContext' has no member named 'enqueueV3'; did you mean 'enqueueV2'?
         if (mContext.enqueueV3(stream.get()))
                      ^~~~~~~~~
                      enqueueV2
../common/sampleInference.cpp: In member function 'bool sample::{anonymous}::EnqueueSafe::operator()(sample::TrtCudaStream&) const':
../common/sampleInference.cpp:614:22: error: 'class nvinfer1::safe::IExecutionContext' has no member named 'enqueueV3'; did you mean 'enqueueV2'?
         if (mContext.enqueueV3(stream.get()))
                      ^~~~~~~~~
                      enqueueV2
../common/sampleEngines.cpp: In function 'std::__1::vector<std::__1::pair<nvinfer1::WeightsRole, nvinfer1::Weights> > sample::getAllRefitWeightsForLayer(const nvinfer1::ILayer&)':
../common/sampleEngines.cpp:1242:24: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
         case DataType::kUINT8:
                        ^~~~~~
../common/sampleEngines.cpp:1283:21: error: 'kGRID_SAMPLE' is not a member of 'nvinfer1::LayerType'
     case LayerType::kGRID_SAMPLE:
                     ^~~~~~~~~~~~
../common/sampleEngines.cpp:1289:21: error: 'kNMS' is not a member of 'nvinfer1::LayerType'
     case LayerType::kNMS:
                     ^~~~
../common/sampleEngines.cpp:1290:21: error: 'kNON_ZERO' is not a member of 'nvinfer1::LayerType'
     case LayerType::kNON_ZERO:
                     ^~~~~~~~~
../common/sampleEngines.cpp:1291:21: error: 'kONE_HOT' is not a member of 'nvinfer1::LayerType'
     case LayerType::kONE_HOT:
                     ^~~~~~~~
../common/sampleInference.cpp: In member function 'void sample::Binding::fill()':
../common/sampleInference.cpp:1237:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8:
                              ^~~~~~
../common/sampleInference.cpp: In member function 'void sample::Binding::dump(std::__1::ostream&, nvinfer1::Dims, nvinfer1::Dims, int32_t, int32_t, std::__1::string) const':
../common/sampleInference.cpp:1284:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8:
                              ^~~~~~
../common/sampleInference.cpp: In member function 'bool sample::Bindings::setTensorAddresses(nvinfer1::IExecutionContext&) const':
../common/sampleInference.cpp:1473:51: error: 'const class nvinfer1::ICudaEngine' has no member named 'getTensorLocation'; did you mean 'getLocation'?
         auto const location = context.getEngine().getTensorLocation(name);
                                                   ^~~~~~~~~~~~~~~~~
                                                   getLocation
../common/sampleInference.cpp:1478:30: error: 'class nvinfer1::IExecutionContext' has no member named 'setOutputAllocator'
                 if (!context.setOutputAllocator(name, mBindings[b.second].outputAllocator.get()))
                              ^~~~~~~~~~~~~~~~~~
../common/sampleInference.cpp:1485:30: error: 'class nvinfer1::IExecutionContext' has no member named 'setTensorAddress'
                 if (!context.setTensorAddress(name, mDevicePointers[b.second]))
                              ^~~~~~~~~~~~~~~~
../common/sampleInference.cpp: In member function 'bool sample::Bindings::setSafeTensorAddresses(nvinfer1::safe::IExecutionContext&) const':
../common/sampleInference.cpp:1500:33: error: 'const class nvinfer1::safe::ICudaEngine' has no member named 'getTensorIOMode'
         if (context.getEngine().getTensorIOMode(name) == nvinfer1::TensorIOMode::kINPUT)
                                 ^~~~~~~~~~~~~~~
../common/sampleInference.cpp:1500:68: error: 'nvinfer1::TensorIOMode' has not been declared
         if (context.getEngine().getTensorIOMode(name) == nvinfer1::TensorIOMode::kINPUT)
                                                                    ^~~~~~~~~~~~
../common/sampleInference.cpp:1502:26: error: 'class nvinfer1::safe::IExecutionContext' has no member named 'setInputTensorAddress'
             if (!context.setInputTensorAddress(name, static_cast<void const*>(mDevicePointers[b.second])))
                          ^~~~~~~~~~~~~~~~~~~~~
../common/sampleInference.cpp:1509:26: error: 'class nvinfer1::safe::IExecutionContext' has no member named 'setOutputTensorAddress'
             if (!context.setOutputTensorAddress(name, mDevicePointers[b.second]))
                          ^~~~~~~~~~~~~~~~~~~~~~
../common/sampleInference.cpp: In member function 'void sample::Binding::fill()':
../common/sampleInference.cpp:1237:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8:
                              ^~~~~~
../common/sampleInference.cpp: In member function 'void sample::Binding::dump(std::__1::ostream&, nvinfer1::Dims, nvinfer1::Dims, int32_t, int32_t, std::__1::string) const':
../common/sampleInference.cpp:1284:30: error: 'kUINT8' is not a member of 'nvinfer1::DataType'
     case nvinfer1::DataType::kUINT8:
                              ^~~~~~
../common/sampleInference.cpp: In member function 'bool sample::Bindings::setTensorAddresses(nvinfer1::IExecutionContext&) const':
../common/sampleInference.cpp:1473:51: error: 'const class nvinfer1::ICudaEngine' has no member named 'getTensorLocation'; did you mean 'getLocation'?
         auto const location = context.getEngine().getTensorLocation(name);
                                                   ^~~~~~~~~~~~~~~~~
                                                   getLocation
../common/sampleInference.cpp:1478:30: error: 'class nvinfer1::IExecutionContext' has no member named 'setOutputAllocator'
                 if (!context.setOutputAllocator(name, mBindings[b.second].outputAllocator.get()))
                              ^~~~~~~~~~~~~~~~~~
../common/sampleInference.cpp:1485:30: error: 'class nvinfer1::IExecutionContext' has no member named 'setTensorAddress'
                 if (!context.setTensorAddress(name, mDevicePointers[b.second]))
                              ^~~~~~~~~~~~~~~~
../common/sampleInference.cpp: In member function 'bool sample::Bindings::setSafeTensorAddresses(nvinfer1::safe::IExecutionContext&) const':
../common/sampleInference.cpp:1500:33: error: 'const class nvinfer1::safe::ICudaEngine' has no member named 'getTensorIOMode'
         if (context.getEngine().getTensorIOMode(name) == nvinfer1::TensorIOMode::kINPUT)
                                 ^~~~~~~~~~~~~~~
../common/sampleInference.cpp:1500:68: error: 'nvinfer1::TensorIOMode' has not been declared
         if (context.getEngine().getTensorIOMode(name) == nvinfer1::TensorIOMode::kINPUT)
                                                                    ^~~~~~~~~~~~
../common/sampleInference.cpp:1502:26: error: 'class nvinfer1::safe::IExecutionContext' has no member named 'setInputTensorAddress'
             if (!context.setInputTensorAddress(name, static_cast<void const*>(mDevicePointers[b.second])))
                          ^~~~~~~~~~~~~~~~~~~~~
../common/sampleInference.cpp:1509:26: error: 'class nvinfer1::safe::IExecutionContext' has no member named 'setOutputTensorAddress'
             if (!context.setOutputTensorAddress(name, mDevicePointers[b.second]))
ttyio commented 1 year ago

@fucker007 Could we check if we are using the correct OSS branch? for the kUINT8 error, it is defined here

https://github.com/NVIDIA/TensorRT/blob/release/8.5/include/NvInferRuntimeCommon.h#L145

ttyio commented 1 year ago

closing since no activity for more than 3 weeks, thanks!

annb3 commented 3 months ago

I have the same problem into my docker environment.