search

cap-ntu / ML-Model-CI

MLModelCI is a complete MLOps platform for managing, converting, profiling, and deploying MLaaS (Machine Learning-as-a-Service), bridging the gap between current ML training and serving systems.
https://mlmodelci.com
Apache License 2.0
188 stars 33 forks source link

[Feat] support single-stage detection model #166

Closed HuaizhengZhang closed 3 years ago

HuaizhengZhang commented 3 years ago
HuaizhengZhang commented 3 years ago

转不了的错误信息，详细记录

univerone commented 3 years ago

Attempt to Convert TinaFace Model into ONNX format

Environment

My script

from modelci.hub.converter import ONNXConverter
from modelci.types.bo import IOShape, Framework, Task, Engine
from modelci.hub.utils import generate_path
from modelci.types.trtis_objects import ModelInputFormat
import torch
from vedacore.misc import Config, load_weights
from vedadet.models import build_detector

if __name__ == "__main__":
    cfg = Config.fromfile("repo/configs/trainval/tinaface/tinaface.py")
    model = build_detector(cfg.model)
    load_weights(model, "tinaface_r50_fpn_widerface.pth")

    device = 'cpu'
    model.to(device)
    model.forward = model.forward_impl

    shape = map(int, '3,800,1344'.split(','))
    dummy_input = torch.randn(1, *shape)
    onnx_save_path = generate_path(
        model_name='ResNet50',
        framework=Framework.PYTORCH,
        task=Task.OBJECT_DETECTION,
        engine=Engine.ONNX,
        version=1
    )
    inputs = [IOShape([-1, 3, 800, 1344], dtype=float, name='INPUT__0', format=ModelInputFormat.FORMAT_NCHW)]
    dev = torch.device("cpu")
    model.to(dev)
    ONNXConverter.from_torch_module(
        model,
        onnx_save_path,
        inputs=inputs,
    )

Error Message

/home/sherry/project/miniconda3/envs/modelci/bin/python /home/sherry/project/github/ML-Model-CI/example/notebook/test.py
Traceback (most recent call last):
  File "/home/sherry/project/github/ML-Model-CI/example/notebook/test.py", line 111, in <module>
    inputs=inputs,
  File "/home/sherry/project/github/ML-Model-CI/modelci/hub/converter.py", line 249, in from_torch_module
    **export_kwargs
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/onnx/__init__.py", line 230, in export
    custom_opsets, enable_onnx_checker, use_external_data_format)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/onnx/utils.py", line 91, in export
    use_external_data_format=use_external_data_format)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/onnx/utils.py", line 639, in _export
    dynamic_axes=dynamic_axes)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/onnx/utils.py", line 411, in _model_to_graph
    use_new_jit_passes)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/onnx/utils.py", line 379, in _create_jit_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/onnx/utils.py", line 342, in _trace_and_get_graph_from_model
    torch.jit._get_trace_graph(model, args, strict=False, _force_outplace=False, _return_inputs_states=True)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_trace.py", line 1148, in _get_trace_graph
    outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_trace.py", line 130, in forward
    self._force_outplace,
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_trace.py", line 116, in wrapper
    outs.append(self.inner(*trace_inputs))
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/nn/modules/module.py", line 725, in _call_impl
    result = self._slow_forward(*input, **kwargs)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/nn/modules/module.py", line 709, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/home/sherry/project/github/ML-Model-CI/example/notebook/repo/vedadet/models/detectors/single_stage_detector.py", line 35, in forward_impl
    feats = self.backbone(x)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/nn/modules/module.py", line 725, in _call_impl
    result = self._slow_forward(*input, **kwargs)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/nn/modules/module.py", line 709, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/home/sherry/project/github/ML-Model-CI/example/notebook/repo/vedadet/models/backbones/resnet.py", line 630, in forward
    x = res_layer(x)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/nn/modules/module.py", line 725, in _call_impl
    result = self._slow_forward(*input, **kwargs)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/nn/modules/module.py", line 709, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/nn/modules/container.py", line 117, in forward
    input = module(input)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/nn/modules/module.py", line 725, in _call_impl
    result = self._slow_forward(*input, **kwargs)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/nn/modules/module.py", line 709, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/home/sherry/project/github/ML-Model-CI/example/notebook/repo/vedadet/models/backbones/resnet.py", line 297, in forward
    out = _inner_forward(x)
  File "/home/sherry/project/github/ML-Model-CI/example/notebook/repo/vedadet/models/backbones/resnet.py", line 274, in _inner_forward
    out = self.conv2(out)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/nn/modules/module.py", line 725, in _call_impl
    result = self._slow_forward(*input, **kwargs)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/nn/modules/module.py", line 709, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/home/sherry/project/github/ML-Model-CI/example/notebook/repo/vedadet/ops/dcn/deform_conv.py", line 302, in forward
    self.dilation, self.groups, self.deformable_groups)
  File "/home/sherry/project/github/ML-Model-CI/example/notebook/repo/vedadet/ops/dcn/deform_conv.py", line 46, in forward
    raise NotImplementedError
NotImplementedError
univerone commented 3 years ago

Attempt to Convert TinaFace into TorchScript

My script

from modelci.hub.converter import TorchScriptConverter
from modelci.types.bo import IOShape, Framework, Task, Engine
from modelci.hub.utils import generate_path
import torch
from vedacore.misc import Config, load_weights
from vedadet.models import build_detector

if __name__ == "__main__":
    cfg = Config.fromfile("repo/configs/trainval/tinaface/tinaface.py")
    model = build_detector(cfg.model)
    load_weights(model, "tinaface_r50_fpn_widerface.pth")

    device = 'cpu'
    model.to(device)
    model.forward = model.forward_impl

    shape = map(int, '3,800,1344'.split(','))
    dummy_input = torch.randn(1, *shape)
    onnx_save_path = generate_path(
        model_name='ResNet50',
        framework=Framework.PYTORCH,
        task=Task.OBJECT_DETECTION,
        engine=Engine.ONNX,
        version=1
    )
    dev = torch.device("cpu")
    model.to(dev)
    TorchScriptConverter.from_torch_module(
        model,
        onnx_save_path,
    )

Relevant Snippet

https://github.com/Media-Smart/vedadet/blob/main/vedadet/models/backbones/resnet.py#L264

Error Log

/home/sherry/project/miniconda3/envs/modelci/bin/python /home/sherry/project/github/ML-Model-CI/example/notebook/test.py
Traceback (most recent call last):
  File "/home/sherry/project/github/ML-Model-CI/example/notebook/test.py", line 110, in <module>
    onnx_save_path,
  File "/home/sherry/project/github/ML-Model-CI/modelci/hub/converter.py", line 137, in from_torch_module
    traced = torch.jit.script(model)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_script.py", line 898, in script
    obj, torch.jit._recursive.infer_methods_to_compile
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_recursive.py", line 352, in create_script_module
    return create_script_module_impl(nn_module, concrete_type, stubs_fn)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_recursive.py", line 406, in create_script_module_impl
    script_module = torch.jit.RecursiveScriptModule._construct(cpp_module, init_fn)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_script.py", line 388, in _construct
    init_fn(script_module)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_recursive.py", line 388, in init_fn
    scripted = create_script_module_impl(orig_value, sub_concrete_type, stubs_fn)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_recursive.py", line 406, in create_script_module_impl
    script_module = torch.jit.RecursiveScriptModule._construct(cpp_module, init_fn)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_script.py", line 388, in _construct
    init_fn(script_module)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_recursive.py", line 388, in init_fn
    scripted = create_script_module_impl(orig_value, sub_concrete_type, stubs_fn)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_recursive.py", line 406, in create_script_module_impl
    script_module = torch.jit.RecursiveScriptModule._construct(cpp_module, init_fn)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_script.py", line 388, in _construct
    init_fn(script_module)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_recursive.py", line 388, in init_fn
    scripted = create_script_module_impl(orig_value, sub_concrete_type, stubs_fn)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_recursive.py", line 364, in create_script_module_impl
    method_stubs = stubs_fn(nn_module)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_recursive.py", line 591, in infer_methods_to_compile
    stubs.append(make_stub_from_method(nn_module, method))
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_recursive.py", line 52, in make_stub_from_method
    return make_stub(func, method_name)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/_recursive.py", line 37, in make_stub
    ast = get_jit_def(func, name, self_name="RecursiveScriptModule")
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/frontend.py", line 221, in get_jit_def
    return build_def(ctx, fn_def, type_line, def_name, self_name=self_name)
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/frontend.py", line 255, in build_def
    build_stmts(ctx, body))
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/frontend.py", line 127, in build_stmts
    stmts = [build_stmt(ctx, s) for s in stmts]
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/frontend.py", line 127, in <listcomp>
    stmts = [build_stmt(ctx, s) for s in stmts]
  File "/home/sherry/project/miniconda3/envs/modelci/lib/python3.7/site-packages/torch/jit/frontend.py", line 228, in __call__
    raise UnsupportedNodeError(ctx, node)
torch.jit.frontend.UnsupportedNodeError: function definitions aren't supported:
  File "/home/sherry/project/github/ML-Model-CI/example/notebook/repo/vedadet/models/backbones/resnet.py", line 264
        """Forward function."""

        def _inner_forward(x):
        ~~~ <--- HERE
            identity = x
univerone commented 3 years ago

Refer to the official documentation: https://github.com/Media-Smart/vedadet#deploy Deformable Conv operations are unsupported for model conversion.