lix19937
commented
1 month ago would change volume 230976 to 76992
Your network forward graph has dynamic ops, and onnx export process has error, make some dynamic as const value.
Open hello-lx opened 1 month ago
lix19937
commented
1 month ago would change volume 230976 to 76992
Your network forward graph has dynamic ops, and onnx export process has error, make some dynamic as const value.
hello-lx
commented
1 month ago This is the code I used to convert the pytorch model to onnx. ` import torch import os from yolact import Yolact from utils.functions import SavePath from data import set_cfg
device = 'cuda' if torch.cuda.is_available() else 'cpu'
trained_model = 'models/yolact_plus_resnet50_54_800000.pth'
model_path = SavePath.from_str(trained_model)
config = model_path.model_name + '_config'
set_cfg(config)
net = Yolact()
net.load_weights(trained_model, device=device)
net.eval()
net.to(device)
output_onnx = os.path.splitext(trained_model)[0] + '.onnx'
inputs = torch.randn(3, 3, 550, 550).to(device)
print('convert', output_onnx, 'begin')
input_names, output_names = ['image'], ["loc", "conf", "mask", "prior", "proto"]
torch.onnx.export(net, inputs, output_onnx, verbose=False, opset_version=12, input_names=input_names,
output_names=output_names, dynamic_axes={
input_names[0]: {0: 'batch_size'},
output_names[0]: {0: 'batch_size'},
output_names[1]: {0: 'batch_size'},
output_names[2]: {0: 'batch_size'},
output_names[3]: {0: 'batch_size'},
output_names[4]: {0: 'map_width', 1: 'map_height'},
}, operator_export_type=torch.onnx.OperatorExportTypes.ONNX_ATEN_FALLBACK)
print('convert', output_onnx, 'to onnx finish!!!')`
This is the code I used to convert the onnx model to tensorrt. ` import tensorrt as trt logger = trt.Logger(trt.Logger.ERROR)
def export1(onnx_name, workspace=4, dynamic=False, half=False):
logger = trt.Logger(trt.Logger.INFO)
builder = trt.Builder(logger)
config = builder.create_builder_config()
config.max_workspace_size = workspace * 1 << 30
flag = (1 << int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH))
network = builder.create_network(flag)
parser = trt.OnnxParser(network, logger)
onnx = onnx_name + '.onnx'
if not parser.parse_from_file(str(onnx)):
raise RuntimeError(f'failed to load ONNX file: {onnx}')
inputs = [(network.get_input(i).name, network.get_input(i).shape) for i in range(network.num_inputs)]
outputs = [(network.get_output(i).name, network.get_output(i).shape) for i in range(network.num_outputs)]
print(inputs)
print(outputs)
if dynamic:
# if im.shape[0] <= 1:
# logger.warning(f"{prefix} WARNING ⚠️ --dynamic model requires maximum --batch-size argument")
profile = builder.create_optimization_profile()
for item in inputs + outputs:
profile.set_shape(item[0], item[1], item[2], item[3])
config.add_optimization_profile(profile)
if builder.platform_has_fast_fp16 and half:
config.set_flag(trt.BuilderFlag.FP16)
# build engine 文件的写入这里的f是前面定义的engine文件
f = onnx_name + '.engine'
with builder.build_engine(network, config) as engine, open(f, 'wb') as t:
# 序列化model
t.write(engine.serialize())
return f, None`
I did not find any abnormalities during the process of converting the model. Can you provide some suggestion? Thank you very much
lix19937
commented
3 weeks ago Pay attention the log of torch.onnx.export process warnings . And close the operator_export_type=torch.onnx.OperatorExportTypes.ONNX_ATEN_FALLBACK @hello-lx
hello-lx
commented
2 weeks ago convert models/yolact_plus_resnet50_54_800000_2.onnx begin
D:\XSpace\CPP\DCNv2_Plugin\yolact-onnx-deploy-main\convert-onnx\yolact.py:272: TracerWarning: torch.Tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
self.priors = torch.Tensor(prior_data).view(-1, 4).detach().to(device)
D:\XSpace\CPP\DCNv2_Plugin\yolact-onnx-deploy-main\convert-onnx\yolact.py:272: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
self.priors = torch.Tensor(prior_data).view(-1, 4).detach().to(device)
Traceback (most recent call last):
File "D:\XSpace\CPP\DCNv2_Plugin\yolact-onnx-deploy-main\convert-onnx\convert_onnx.py", line 40, in
Process finished with exit code 1
lix19937
commented
2 weeks ago RuntimeError: shape '[57744, 4]' is invalid for input of size 692928
Process finished with exit code 1
error arise !
Description
Hello, I have a problem and maybe it happens even at the moment of converting the weights.pt after training YOLACK (https://github.com/dbolya/yolact) to the TensorRT model.
I tried to convert model to trt model yolack (not yolack++) on GPU, but it fails with the error below.
[05/30/2024-10:56:44] [TRT] [E] 4: kOPT values for profile 0 violate shape constraints: IShuffleLayer /Reshape: reshaping failed for tensor: /Concat_output_0 reshape would change volume 230976 to 76992 [05/30/2024-10:56:44] [TRT] [E] 4: [shapeCompiler.cpp::nvinfer1::builder::DynamicSlotBuilder::evaluateShapeChecks::1276] Error Code 4: Internal Error (kOPT values for profile 0 violate shape constraints: IShuffleLayer /Reshape: reshaping failed for tensor: /Concat_output_0 reshape would change volume 230976 to 76992)
Environment
TensorRT Version: 8.6.1
NVIDIA GPU: RTX 4050
NVIDIA Driver Version:
CUDA Version: 11.8
CUDNN Version:
Operating System: Windows 10
Python Version (if applicable): 3.9.19
PyTorch Version (if applicable): 2.2.2