PPQ can not process parameter variable(PPQ_Variable_108) with multiple destinations

Xiaochaosui commented 3 months ago

Hi,I see a issue when I use ppq to record model inference, what can I do? plea, help me? My Family!

This my code that encounter this issue.

 ir = quantize_onnx_model(
    onnx_import_file="model_new.onnx",
    calib_dataloader=sample_input,
    platform = TargetPlatform.TRT_INT8,
    calib_steps=16,
    do_quantize=False,
    input_shape=None,
    collate_fn=lambda x:x.to('cuda'),
    inputs = onnx_input
    )

This is error（TraceBack）


Traceback (most recent call last):
  File "/data/Projects_XS/HPTR/src/7_profile_test.py", line 55, in main
    ir = quantize_onnx_model(
  File "/data/envs_xs/anaconda3/envs/hptr2/lib/python3.9/site-packages/ppq-0.6.6-py3.9.egg/ppq/core/defs.py", line 54, in _wrapper
    return func(*args, **kwargs)
  File "/data/envs_xs/anaconda3/envs/hptr2/lib/python3.9/site-packages/ppq-0.6.6-py3.9.egg/ppq/api/interface.py", line 261, in quantize_onnx_model
    executor = TorchExecutor(graph=quantizer._graph, device=device)
  File "/data/envs_xs/anaconda3/envs/hptr2/lib/python3.9/site-packages/ppq-0.6.6-py3.9.egg/ppq/executor/torch.py", line 287, in __init__
    super().__init__(graph)
  File "/data/envs_xs/anaconda3/envs/hptr2/lib/python3.9/site-packages/ppq-0.6.6-py3.9.egg/ppq/executor/base.py", line 117, in __init__
    self.load_graph(graph=graph)
  File "/data/envs_xs/anaconda3/envs/hptr2/lib/python3.9/site-packages/ppq-0.6.6-py3.9.egg/ppq/executor/torch.py", line 608, in load_graph
    self._runnable_graph(GraphDeployCommand(device=self._device))
  File "/data/envs_xs/anaconda3/envs/hptr2/lib/python3.9/site-packages/ppq-0.6.6-py3.9.egg/ppq/IR/processer.py", line 130, in __call__
    return self.process(command)
  File "/data/envs_xs/anaconda3/envs/hptr2/lib/python3.9/site-packages/ppq-0.6.6-py3.9.egg/ppq/IR/deploy.py", line 35, in process
    return self.deploy(device)
  File "/data/envs_xs/anaconda3/envs/hptr2/lib/python3.9/site-packages/ppq-0.6.6-py3.9.egg/ppq/IR/deploy.py", line 125, in deploy
    raise PermissionError(
PermissionError: PPQ can not process parameter variable(PPQ_Variable_108) with multiple destinations(['/decoder/_decoders.0/tf_reinforce_tl.1/layers.0/norm_src/Add_1', '/decoder/_decoders.0/tf_reinforce_tl.1/layers.0/norm_src_1/Add_1']), split it first.

ZhangZhiPku commented 3 months ago

网络太复杂了吗？

Xiaochaosui commented 3 months ago

网络太复杂了吗？

对，是有一点，但是我看报错的这里是一些关于Tensor相加的问题好像，不知道如何解决，是ppq算子不支持嘛？。PermissionError: PPQ can not process parameter variable(PPQ_Variable_108) with multiple destinations(['/decoder/_decoders.0/tf_reinforce_tl.1/layers.0/norm_src/Add_1', '/decoder/_decoders.0/tf_reinforce_tl.1/layers.0/norm_src_1/Add_1']), split it first.

Xiaochaosui commented 3 months ago

我查看我的网络结构是在，这里做了一个nn.LayerNorm()归一化操作，然后我注释掉这个归一化操作则，可运行，所以我是不是可以认为是ppq没有这个相应的算子呀？或者我可以怎么样加这个对应算子嘛？然后我看ppq报错的这一行源码，是因为，variable.dest_ops >1 是因为这个算子需要多个输入所以报错嘛？因为我是刚接触量化菜鸡，不确定我的问题是否表现的像个luser，如果是的话就直接略过我把，我再继续看源码能否解决，不好意思打扰了。 @ZhangZhiPku

OpenPPL / ppq

PPQ can not process parameter variable(PPQ_Variable_108) with multiple destinations #568