search

openvpi / DiffSinger

An advanced singing voice synthesis system with high fidelity, expressiveness, controllability and flexibility based on DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism
Apache License 2.0
2.62k stars 275 forks source link

Can not export ONNX model #121

Closed Katistic closed 11 months ago

Katistic commented 11 months ago

After training acoustic model with diffsinger v2.0.0 and trying to export using pytorch v2.0.1, can't use aten::unflatten layer

Traceback (most recent call last):
  File "/content/DiffSinger/scripts/export.py", line 218, in <module>
    main()
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/content/DiffSinger/scripts/export.py", line 133, in acoustic
    exporter.export(out)
  File "/content/DiffSinger/deployment/exporters/acoustic_exporter.py", line 84, in export
    self.export_model(path / f'{model_name}.onnx')
  File "/content/DiffSinger/deployment/exporters/acoustic_exporter.py", line 88, in export_model
    self._torch_export_model()
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/content/DiffSinger/deployment/exporters/acoustic_exporter.py", line 166, in _torch_export_model
    torch.onnx.export(
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 506, in export
    _export(
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 1548, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 1117, in _model_to_graph
    graph = _optimize_graph(
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 665, in _optimize_graph
    graph = _C._jit_pass_onnx(graph, operator_export_type)
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 1901, in _run_symbolic_function
    raise errors.UnsupportedOperatorError(
torch.onnx.errors.UnsupportedOperatorError: Exporting the operator 'aten::unflatten' to ONNX opset version 15 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub: https://github.com/pytorch/pytorch/issues.

I found this to be a torch bug, and updated my torch to the nightly 2.1 release (as of making this issue, torch==2.1.0.dev20230721+cu121)

I then get this error using the above torch version

Tracing WaveNet denoiser...
Scripting GaussianDiffusion...
Exporting GaussianDiffusion...
/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py:842: UserWarning: no signature found for <torch.ScriptMethod object at 0x7a81884b12b0>, skipping _decide_input_format
  warnings.warn(f"{e}, skipping _decide_input_format")
======= Diagnostic Run torch.onnx.export version 2.1.0.dev20230721+cu121 =======
verbose: False, log level: 40
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

Traceback (most recent call last):
  File "/content/DiffSinger/scripts/export.py", line 218, in <module>
    main()
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/content/DiffSinger/scripts/export.py", line 133, in acoustic
    exporter.export(out)
  File "/content/DiffSinger/deployment/exporters/acoustic_exporter.py", line 84, in export
    self.export_model(path / f'{model_name}.onnx')
  File "/content/DiffSinger/deployment/exporters/acoustic_exporter.py", line 88, in export_model
    self._torch_export_model()
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/content/DiffSinger/deployment/exporters/acoustic_exporter.py", line 210, in _torch_export_model
    torch.onnx.export(
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 511, in export
    _export(
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 1577, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 1134, in _model_to_graph
    graph = _optimize_graph(
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 672, in _optimize_graph
    graph = _C._jit_pass_onnx(graph, operator_export_type)
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 1919, in _run_symbolic_function
    return symbolic_fn(graph_context, *inputs, **attrs)
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/symbolic_opset9.py", line 7064, in prim_if
    torch._C._jit_pass_onnx_block(
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 1919, in _run_symbolic_function
    return symbolic_fn(graph_context, *inputs, **attrs)
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/symbolic_opset9.py", line 6975, in prim_loop
    torch._C._jit_pass_onnx_block(
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 1919, in _run_symbolic_function
    return symbolic_fn(graph_context, *inputs, **attrs)
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/symbolic_helper.py", line 392, in wrapper
    return fn(g, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/symbolic_opset9.py", line 1416, in relu
    return _op_with_optional_float_cast(g, "Relu", input, opset_before=14)
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/symbolic_opset9.py", line 1380, in _op_with_optional_float_cast
    dtype_0 = _type_utils.JitScalarType.from_value(inputs[0])
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/_type_utils.py", line 201, in from_value
    raise errors.SymbolicValueError(
torch.onnx.errors.SymbolicValueError: Cannot determine scalar type for this '<class 'torch.TensorType'>' instance and a default value was not provided.  [Caused by the value 'input.3 defined in (%input.3 : Tensor = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%input, %diffusion.denoise_fn.input_projection.weight, %diffusion.denoise_fn.input_projection.bias), scope: deployment.modules.toplevel.DiffSingerAcousticONNX::/deployment.modules.diffusion.GaussianDiffusionONNX::diffusion/modules.diffusion.wavenet.WaveNet::denoise_fn/modules.diffusion.wavenet.Conv1d::input_projection # /usr/local/lib/python3.10/dist-packages/torch/nn/modules/conv.py:309:0
)' (type 'Tensor') in the TorchScript graph. The containing node has kind 'onnx::Conv'.]

I'll try tinkering and see what I can do, but please keep me updated if this is something I'm being stupid with or if you need anything else :)

yqzhishen commented 11 months ago

https://github.com/openvpi/DiffSinger/blob/main/docs/GettingStarted.md#deployment

Please use PyTorch 1.13 instead.

Katistic commented 11 months ago

That fixes it! I'm not sure how I missed that, I read over it like a dozen times. Thanks!