search

IDRnD / ReDimNet

The official pytorch implemention of the Intespeech 2024 paper "Reshape Dimensions Network for Speaker Recognition"
MIT License
118 stars 5 forks source link

Cannot export Pytorch model (ReDimNet) to ONNX #13

Open pongthang opened 1 month ago

pongthang commented 1 month ago

I try to export ReDimNet model from pytorch to ONNX. Please help me this out. The code I use is :

import torch
model = torch.hub.load('IDRnD/ReDimNet', 'b3', pretrained=True, finetuned=False)
input_sample = torch.randn(1, 32000)
model.eval()
with torch.no_grad():
    onnx_program_redim = torch.onnx.dynamo_export(model, input_sample)

The Error is :

Unsupported Traceback (most recent call last)
File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py:1509, in dynamo_export(model, export_options, *model_args, **model_kwargs)
1503 try:
1504 return Exporter(
1505 options=resolved_export_options,
1506 model=model,
1507 model_args=model_args,
1508 model_kwargs=model_kwargs,
-> 1509 ).export()
1510 except Exception as e:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py:1236, in Exporter.export(self)
1231 with self.options.diagnostic_context, decomposition_skip.enable_decomposition_skips(
1232 self.options
1233 ), torch._dynamo.config.patch(
1234 dataclasses.asdict(DEFAULT_EXPORT_DYNAMO_CONFIG)
1235 ):
-> 1236 graph_module = self.options.fx_tracer.generate_fx(
1237 self.options, self.model, self.model_args, self.model_kwargs
1238 )
1239 # TODO: Defer import onnxscript out of import torch path
1240 # https://github.com/pytorch/pytorch/issues/103764

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/onnx/_internal/fx/dynamo_graph_extractor.py:214, in DynamoExport.generate_fx(self, options, model, model_args, model_kwargs)
213 with fake_mode: # type: ignore[attr-defined]
--> 214 graph_module, graph_guard = torch._dynamo.export(
215 wrapped_model,
216 tracing_mode=fx_mode,
217 )(
218 *model_args,
219 **model_kwargs,
220 )
221 del graph_guard # Unused

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py:1379, in export..inner(*args, **kwargs)
1378 try:
-> 1379 result_traced = opt_f(*args, **kwargs)
1380 except ConstraintViolationError as e:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py:433, in _TorchDynamoContext.call.._fn(*args, **kwargs)
432 try:
--> 433 return fn(*args, **kwargs)
434 finally:
435 # Restore the dynamic layer stack depth if necessary.

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/onnx/_internal/fx/dynamo_graph_extractor.py:169, in _wrap_model_with_output_adapter..wrapped(*args, **kwargs)
167 @functools.wraps(model_func)
168 def wrapped(*args, **kwargs):
--> 169 return output_adapter.apply(model_func(*args, **kwargs), model=model)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:1116, in CatchErrorsWrapper.call(self, frame, cache_entry, frame_state)
1114 with compile_lock, _disable_current_modes():
1115 # skip=1: skip this frame
-> 1116 return self._torchdynamo_orig_callable(
1117 frame, cache_entry, self.hooks, frame_state, skip=1
1118 )

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:472, in ConvertFrameAssert.call(self, frame, cache_entry, hooks, frame_state, skip)
460 signpost_event(
461 "dynamo",
462 "_convert_frame_assert._compile",
(...)
469 },
470 )
--> 472 return _compile(
473 frame.f_code,
474 frame.f_globals,
475 frame.f_locals,
476 frame.f_builtins,
477 self._torchdynamo_orig_callable,
478 self._one_graph,
479 self._export,
480 self._export_constraints,
481 hooks,
482 cache_entry,
483 cache_size,
484 frame,
485 frame_state=frame_state,
486 compile_id=compile_id,
487 skip=skip + 1,
488 )

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_utils_internal.py:84, in compile_time_strobelight_meta..compile_time_strobelight_meta_inner..wrapper_function(*args, **kwargs)
83 kwargs["skip"] = kwargs["skip"] + 1
---> 84 return StrobelightCompileTimeProfiler.profile_compile_time(
85 function, phase_name, *args, **kwargs
86 )

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_strobelight/compile_time_profiler.py:129, in StrobelightCompileTimeProfiler.profile_compile_time(cls, func, phase_name, *args, **kwargs)
128 if not cls.enabled:
--> 129 return func(*args, **kwargs)
131 if cls.profiler is None:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/contextlib.py:79, in ContextDecorator.call..inner(*args, **kwds)
78 with self._recreate_cm():
---> 79 return func(*args, **kwds)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:817, in _compile(code, globals, locals, builtins, compiler_fn, one_graph, export, export_constraints, hooks, cache_entry, cache_size, frame, frame_state, compile_id, skip)
816 try:
--> 817 guarded_code = compile_inner(code, one_graph, hooks, transform)
818 return guarded_code

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/utils.py:231, in dynamo_timed..dynamo_timed_inner..time_wrapper(*args, **kwargs)
230 t0 = time.time()
--> 231 r = func(*args, **kwargs)
232 time_spent = time.time() - t0

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:636, in _compile..compile_inner(code, one_graph, hooks, transform)
635 try:
--> 636 out_code = transform_code_object(code, transform)
637 break

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py:1185, in transform_code_object(code, transformations, safe)
1183 propagate_line_nums(instructions)
-> 1185 transformations(instructions, code_options)
1186 return clean_and_assemble_instructions(instructions, keys, code_options)[1]

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:178, in preserve_global_state.._fn(*args, **kwargs)
177 try:
--> 178 return fn(*args, **kwargs)
179 finally:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:582, in _compile..transform(instructions, code_options)
581 with tracing(tracer.output.tracing_context), tracer.set_current_tx():
--> 582 tracer.run()
583 except exc.UnspecializeRestartAnalysis:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:2451, in InstructionTranslator.run(self)
2450 def run(self):
-> 2451 super().run()

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:893, in InstructionTranslatorBase.run(self)
892 self.output.push_tx(self)
--> 893 while self.step():
894 pass

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:805, in InstructionTranslatorBase.step(self)
804 try:
--> 805 self.dispatch_table[inst.opcode](self, inst)
806 return not self.output.should_exit

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:499, in break_graph_if_unsupported..decorator..wrapper(self, inst)
498 try:
--> 499 return inner_fn(self, inst)
500 except Unsupported as excp:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:1459, in InstructionTranslatorBase.CALL_FUNCTION(self, inst)
1458 fn = self.pop()
-> 1459 self.call_function(fn, args, {})

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:743, in InstructionTranslatorBase.call_function(self, fn, args, kwargs)
742 raise AssertionError(f"Attempt to trace forbidden callable {inner_fn}")
--> 743 self.push(fn.call_function(self, args, kwargs))

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py:437, in NNModuleVariable.call_function(self, tx, args, kwargs)
436 assert istype(fn, types.FunctionType)
--> 437 return tx.inline_user_function_return(
438 variables.UserFunctionVariable(fn, source=fn_source),
439 args,
440 kwargs,
441 )

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:749, in InstructionTranslatorBase.inline_user_function_return(self, fn, args, kwargs)
746 """
747 A call to some user defined function by inlining it.
748 """
--> 749 return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2666, in InliningInstructionTranslator.inline_call(cls, parent, func, args, kwargs)
2665 with patch.dict(counters, {"unimplemented": counters["inline_call"]}):
-> 2666 return cls.inline_call(parent, func, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2782, in InliningInstructionTranslator.inline_call(parent, func, args, kwargs)
2781 with strict_ctx:
-> 2782 tracer.run()
2783 except exc.ObservedException as e:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:893, in InstructionTranslatorBase.run(self)
892 self.output.push_tx(self)
--> 893 while self.step():
894 pass

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:805, in InstructionTranslatorBase.step(self)
804 try:
--> 805 self.dispatch_table[inst.opcode](self, inst)
806 return not self.output.should_exit

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:499, in break_graph_if_unsupported..decorator..wrapper(self, inst)
498 try:
--> 499 return inner_fn(self, inst)
500 except Unsupported as excp:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:1500, in InstructionTranslatorBase.CALL_FUNCTION_EX(self, inst)
1499 kwargsvars = kwargsvars.keys_as_python_constant()
-> 1500 self.call_function(fn, argsvars.items, kwargsvars)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:743, in InstructionTranslatorBase.call_function(self, fn, args, kwargs)
742 raise AssertionError(f"Attempt to trace forbidden callable {inner_fn}")
--> 743 self.push(fn.call_function(self, args, kwargs))

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:344, in UserMethodVariable.call_function(self, tx, args, kwargs)
343 return invoke_and_store_as_constant(tx, fn, self.get_name(), args, kwargs)
--> 344 return super().call_function(tx, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:293, in UserFunctionVariable.call_function(self, tx, args, kwargs)
289 return invoke_and_store_as_constant(
290 tx, self.fn, self.get_name(), args, kwargs
291 )
--> 293 return super().call_function(tx, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:90, in BaseUserFunctionVariable.call_function(self, tx, args, kwargs)
87 def call_function(
88 self, tx, args: "List[VariableTracker]", kwargs: "Dict[str, VariableTracker]"
89 ) -> "VariableTracker":
---> 90 return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:749, in InstructionTranslatorBase.inline_user_function_return(self, fn, args, kwargs)
746 """
747 A call to some user defined function by inlining it.
748 """
--> 749 return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2666, in InliningInstructionTranslator.inline_call(cls, parent, func, args, kwargs)
2665 with patch.dict(counters, {"unimplemented": counters["inline_call"]}):
-> 2666 return cls.inline_call(parent, func, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2782, in InliningInstructionTranslator.inline_call(parent, func, args, kwargs)
2781 with strict_ctx:
-> 2782 tracer.run()
2783 except exc.ObservedException as e:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:893, in InstructionTranslatorBase.run(self)
892 self.output.push_tx(self)
--> 893 while self.step():
894 pass

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:805, in InstructionTranslatorBase.step(self)
804 try:
--> 805 self.dispatch_table[inst.opcode](self, inst)
806 return not self.output.should_exit

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:499, in break_graph_if_unsupported..decorator..wrapper(self, inst)
498 try:
--> 499 return inner_fn(self, inst)
500 except Unsupported as excp:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:1459, in InstructionTranslatorBase.CALL_FUNCTION(self, inst)
1458 fn = self.pop()
-> 1459 self.call_function(fn, args, {})

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:743, in InstructionTranslatorBase.call_function(self, fn, args, kwargs)
742 raise AssertionError(f"Attempt to trace forbidden callable {inner_fn}")
--> 743 self.push(fn.call_function(self, args, kwargs))

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py:366, in NNModuleVariable.call_function(self, tx, args, kwargs)
365 for child_name, submod in mod._modules.items():
--> 366 tx.call_function(
367 tx.output.register_attr_or_module(
368 submod,
369 self.module_key,
370 child_name,
371 source=NNModuleSource(AttrSource(self.source, child_name)),
372 ),
373 [arg],
374 {},
375 )
376 arg = tx.pop()

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:743, in InstructionTranslatorBase.call_function(self, fn, args, kwargs)
742 raise AssertionError(f"Attempt to trace forbidden callable {inner_fn}")
--> 743 self.push(fn.call_function(self, args, kwargs))

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py:437, in NNModuleVariable.call_function(self, tx, args, kwargs)
436 assert istype(fn, types.FunctionType)
--> 437 return tx.inline_user_function_return(
438 variables.UserFunctionVariable(fn, source=fn_source),
439 args,
440 kwargs,
441 )

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:749, in InstructionTranslatorBase.inline_user_function_return(self, fn, args, kwargs)
746 """
747 A call to some user defined function by inlining it.
748 """
--> 749 return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2666, in InliningInstructionTranslator.inline_call(cls, parent, func, args, kwargs)
2665 with patch.dict(counters, {"unimplemented": counters["inline_call"]}):
-> 2666 return cls.inline_call(parent, func, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2782, in InliningInstructionTranslator.inline_call(parent, func, args, kwargs)
2781 with strict_ctx:
-> 2782 tracer.run()
2783 except exc.ObservedException as e:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:893, in InstructionTranslatorBase.run(self)
892 self.output.push_tx(self)
--> 893 while self.step():
894 pass

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:805, in InstructionTranslatorBase.step(self)
804 try:
--> 805 self.dispatch_table[inst.opcode](self, inst)
806 return not self.output.should_exit

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:499, in break_graph_if_unsupported..decorator..wrapper(self, inst)
498 try:
--> 499 return inner_fn(self, inst)
500 except Unsupported as excp:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:1500, in InstructionTranslatorBase.CALL_FUNCTION_EX(self, inst)
1499 kwargsvars = kwargsvars.keys_as_python_constant()
-> 1500 self.call_function(fn, argsvars.items, kwargsvars)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:743, in InstructionTranslatorBase.call_function(self, fn, args, kwargs)
742 raise AssertionError(f"Attempt to trace forbidden callable {inner_fn}")
--> 743 self.push(fn.call_function(self, args, kwargs))

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:344, in UserMethodVariable.call_function(self, tx, args, kwargs)
343 return invoke_and_store_as_constant(tx, fn, self.get_name(), args, kwargs)
--> 344 return super().call_function(tx, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:293, in UserFunctionVariable.call_function(self, tx, args, kwargs)
289 return invoke_and_store_as_constant(
290 tx, self.fn, self.get_name(), args, kwargs
291 )
--> 293 return super().call_function(tx, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:90, in BaseUserFunctionVariable.call_function(self, tx, args, kwargs)
87 def call_function(
88 self, tx, args: "List[VariableTracker]", kwargs: "Dict[str, VariableTracker]"
89 ) -> "VariableTracker":
---> 90 return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:749, in InstructionTranslatorBase.inline_user_function_return(self, fn, args, kwargs)
746 """
747 A call to some user defined function by inlining it.
748 """
--> 749 return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2666, in InliningInstructionTranslator.inline_call(cls, parent, func, args, kwargs)
2665 with patch.dict(counters, {"unimplemented": counters["inline_call"]}):
-> 2666 return cls.inline_call(parent, func, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2782, in InliningInstructionTranslator.inline_call(parent, func, args, kwargs)
2781 with strict_ctx:
-> 2782 tracer.run()
2783 except exc.ObservedException as e:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:893, in InstructionTranslatorBase.run(self)
892 self.output.push_tx(self)
--> 893 while self.step():
894 pass

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:805, in InstructionTranslatorBase.step(self)
804 try:
--> 805 self.dispatch_table[inst.opcode](self, inst)
806 return not self.output.should_exit

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:499, in break_graph_if_unsupported..decorator..wrapper(self, inst)
498 try:
--> 499 return inner_fn(self, inst)
500 except Unsupported as excp:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:1459, in InstructionTranslatorBase.CALL_FUNCTION(self, inst)
1458 fn = self.pop()
-> 1459 self.call_function(fn, args, {})

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:743, in InstructionTranslatorBase.call_function(self, fn, args, kwargs)
742 raise AssertionError(f"Attempt to trace forbidden callable {inner_fn}")
--> 743 self.push(fn.call_function(self, args, kwargs))

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py:437, in NNModuleVariable.call_function(self, tx, args, kwargs)
436 assert istype(fn, types.FunctionType)
--> 437 return tx.inline_user_function_return(
438 variables.UserFunctionVariable(fn, source=fn_source),
439 args,
440 kwargs,
441 )

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:749, in InstructionTranslatorBase.inline_user_function_return(self, fn, args, kwargs)
746 """
747 A call to some user defined function by inlining it.
748 """
--> 749 return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2666, in InliningInstructionTranslator.inline_call(cls, parent, func, args, kwargs)
2665 with patch.dict(counters, {"unimplemented": counters["inline_call"]}):
-> 2666 return cls.inline_call(parent, func, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2782, in InliningInstructionTranslator.inline_call(parent, func, args, kwargs)
2781 with strict_ctx:
-> 2782 tracer.run()
2783 except exc.ObservedException as e:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:893, in InstructionTranslatorBase.run(self)
892 self.output.push_tx(self)
--> 893 while self.step():
894 pass

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:805, in InstructionTranslatorBase.step(self)
804 try:
--> 805 self.dispatch_table[inst.opcode](self, inst)
806 return not self.output.should_exit

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:499, in break_graph_if_unsupported..decorator..wrapper(self, inst)
498 try:
--> 499 return inner_fn(self, inst)
500 except Unsupported as excp:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:1500, in InstructionTranslatorBase.CALL_FUNCTION_EX(self, inst)
1499 kwargsvars = kwargsvars.keys_as_python_constant()
-> 1500 self.call_function(fn, argsvars.items, kwargsvars)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:743, in InstructionTranslatorBase.call_function(self, fn, args, kwargs)
742 raise AssertionError(f"Attempt to trace forbidden callable {inner_fn}")
--> 743 self.push(fn.call_function(self, args, kwargs))

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:344, in UserMethodVariable.call_function(self, tx, args, kwargs)
343 return invoke_and_store_as_constant(tx, fn, self.get_name(), args, kwargs)
--> 344 return super().call_function(tx, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:293, in UserFunctionVariable.call_function(self, tx, args, kwargs)
289 return invoke_and_store_as_constant(
290 tx, self.fn, self.get_name(), args, kwargs
291 )
--> 293 return super().call_function(tx, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:90, in BaseUserFunctionVariable.call_function(self, tx, args, kwargs)
87 def call_function(
88 self, tx, args: "List[VariableTracker]", kwargs: "Dict[str, VariableTracker]"
89 ) -> "VariableTracker":
---> 90 return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:749, in InstructionTranslatorBase.inline_user_function_return(self, fn, args, kwargs)
746 """
747 A call to some user defined function by inlining it.
748 """
--> 749 return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2666, in InliningInstructionTranslator.inline_call(cls, parent, func, args, kwargs)
2665 with patch.dict(counters, {"unimplemented": counters["inline_call"]}):
-> 2666 return cls.inline_call(parent, func, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2782, in InliningInstructionTranslator.inline_call(parent, func, args, kwargs)
2781 with strict_ctx:
-> 2782 tracer.run()
2783 except exc.ObservedException as e:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:893, in InstructionTranslatorBase.run(self)
892 self.output.push_tx(self)
--> 893 while self.step():
894 pass

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:805, in InstructionTranslatorBase.step(self)
804 try:
--> 805 self.dispatch_table[inst.opcode](self, inst)
806 return not self.output.should_exit

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:499, in break_graph_if_unsupported..decorator..wrapper(self, inst)
498 try:
--> 499 return inner_fn(self, inst)
500 except Unsupported as excp:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:1459, in InstructionTranslatorBase.CALL_FUNCTION(self, inst)
1458 fn = self.pop()
-> 1459 self.call_function(fn, args, {})

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:743, in InstructionTranslatorBase.call_function(self, fn, args, kwargs)
742 raise AssertionError(f"Attempt to trace forbidden callable {inner_fn}")
--> 743 self.push(fn.call_function(self, args, kwargs))

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:293, in UserFunctionVariable.call_function(self, tx, args, kwargs)
289 return invoke_and_store_as_constant(
290 tx, self.fn, self.get_name(), args, kwargs
291 )
--> 293 return super().call_function(tx, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:90, in BaseUserFunctionVariable.call_function(self, tx, args, kwargs)
87 def call_function(
88 self, tx, args: "List[VariableTracker]", kwargs: "Dict[str, VariableTracker]"
89 ) -> "VariableTracker":
---> 90 return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:749, in InstructionTranslatorBase.inline_user_function_return(self, fn, args, kwargs)
746 """
747 A call to some user defined function by inlining it.
748 """
--> 749 return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2666, in InliningInstructionTranslator.inline_call(cls, parent, func, args, kwargs)
2665 with patch.dict(counters, {"unimplemented": counters["inline_call"]}):
-> 2666 return cls.inline_call(parent, func, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2782, in InliningInstructionTranslator.inline_call(parent, func, args, kwargs)
2781 with strict_ctx:
-> 2782 tracer.run()
2783 except exc.ObservedException as e:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:893, in InstructionTranslatorBase.run(self)
892 self.output.push_tx(self)
--> 893 while self.step():
894 pass

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:805, in InstructionTranslatorBase.step(self)
804 try:
--> 805 self.dispatch_table[inst.opcode](self, inst)
806 return not self.output.should_exit

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:499, in break_graph_if_unsupported..decorator..wrapper(self, inst)
498 try:
--> 499 return inner_fn(self, inst)
500 except Unsupported as excp:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:1459, in InstructionTranslatorBase.CALL_FUNCTION(self, inst)
1458 fn = self.pop()
-> 1459 self.call_function(fn, args, {})

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:743, in InstructionTranslatorBase.call_function(self, fn, args, kwargs)
742 raise AssertionError(f"Attempt to trace forbidden callable {inner_fn}")
--> 743 self.push(fn.call_function(self, args, kwargs))

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:293, in UserFunctionVariable.call_function(self, tx, args, kwargs)
289 return invoke_and_store_as_constant(
290 tx, self.fn, self.get_name(), args, kwargs
291 )
--> 293 return super().call_function(tx, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:90, in BaseUserFunctionVariable.call_function(self, tx, args, kwargs)
87 def call_function(
88 self, tx, args: "List[VariableTracker]", kwargs: "Dict[str, VariableTracker]"
89 ) -> "VariableTracker":
---> 90 return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:749, in InstructionTranslatorBase.inline_user_function_return(self, fn, args, kwargs)
746 """
747 A call to some user defined function by inlining it.
748 """
--> 749 return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2666, in InliningInstructionTranslator.inline_call(cls, parent, func, args, kwargs)
2665 with patch.dict(counters, {"unimplemented": counters["inline_call"]}):
-> 2666 return cls.inline_call(parent, func, args, kwargs)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py:2782, in InliningInstructionTranslator.inline_call(parent, func, args, kwargs)
2781 with strict_ctx:
-> 2782 tracer.run()
2783 except exc.ObservedException as e:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:893, in InstructionTranslatorBase.run(self)
892 self.output.push_tx(self)
--> 893 while self.step():
894 pass

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:805, in InstructionTranslatorBase.step(self)
804 try:
--> 805 self.dispatch_table[inst.opcode](self, inst)
806 return not self.output.should_exit

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:499, in break_graph_if_unsupported..decorator..wrapper(self, inst)
498 try:
--> 499 return inner_fn(self, inst)
500 except Unsupported as excp:

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:1459, in InstructionTranslatorBase.CALL_FUNCTION(self, inst)
1458 fn = self.pop()
-> 1459 self.call_function(fn, args, {})

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:743, in InstructionTranslatorBase.call_function(self, fn, args, kwargs)
742 raise AssertionError(f"Attempt to trace forbidden callable {inner_fn}")
--> 743 self.push(fn.call_function(self, args, kwargs))

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:665, in SkipFunctionVariable.call_function(self, tx, args, kwargs)
664 msg += f"', {self.reason}'" if self.reason else ""
--> 665 unimplemented(msg)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/_dynamo/exc.py:221, in unimplemented(msg, from_exc)
220 raise Unsupported(msg) from from_exc
--> 221 raise Unsupported(msg)

Unsupported: 'skip function isinstance in file /home/miko/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/jit/init.py'

from user code:
File "/home/miko/.cache/torch/hub/IDRnD_ReDimNet_master/redimnet.py", line 953, in forward
x = self.spec(x).unsqueeze(1)
File "/home/miko/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/miko/.cache/torch/hub/IDRnD_ReDimNet_master/redimnet.py", line 116, in forward
x = self.torchfbank(x)+1e-6
File "/home/miko/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/miko/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torchaudio/transforms/_transforms.py", line 619, in forward
specgram = self.spectrogram(waveform)
File "/home/miko/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/miko/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torchaudio/transforms/_transforms.py", line 110, in forward
return F.spectrogram(
File "/home/miko/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torchaudio/functional/functional.py", line 119, in spectrogram
frame_length_norm, window_norm = _get_spec_norms(normalized)
File "/home/miko/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torchaudio/functional/functional.py", line 233, in _get_spec_norms
if torch.jit.isinstance(normalized, str):

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information

The above exception was the direct cause of the following exception:

OnnxExporterError Traceback (most recent call last)
Cell In[43], line 3
1 model.eval()
2 with torch.no_grad():
----> 3 onnx_program_redim = torch.onnx.dynamo_export(model, input_sample)

File ~/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py:1520, in dynamo_export(model, export_options, *model_args, **model_kwargs)
1512 resolved_export_options.diagnostic_context.dump(sarif_report_path)
1513 message = (
1514 f"Failed to export the model to ONNX. Generating SARIF report at '{sarif_report_path}'. "
1515 "SARIF is a standard format for the output of static analysis tools. "
(...)
1518 f"Please report a bug on PyTorch Github: {_PYTORCH_GITHUB_ISSUES_URL}"
1519 )
-> 1520 raise OnnxExporterError(
1521 ONNXProgram._from_failure(e, resolved_export_options.diagnostic_context),
1522 message,
1523 ) from e

OnnxExporterError: Failed to export the model to ONNX. Generating SARIF report at 'report_dynamo_export.sarif'. SARIF is a standard format for the output of static analysis tools. SARIF logs can be loaded in VS Code SARIF viewer extension, or SARIF web viewer (https://microsoft.github.io/sarif-web-component/). Please report a bug on PyTorch Github: [https://github.com/pytorch/pytorch/issues```](https://github.com/pytorch/pytorch/issues%60%60%60)
vanIvan commented 1 month ago

Hello! Thank you for submitting issue. I see the problem is in features, especially in torchaudio package:

from user code:
File "/home/miko/.cache/torch/hub/IDRnD_ReDimNet_master/redimnet.py", line 953, in forward
x = self.spec(x).unsqueeze(1)
File "/home/miko/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/miko/.cache/torch/hub/IDRnD_ReDimNet_master/redimnet.py", line 116, in forward
x = self.torchfbank(x)+1e-6
File "/home/miko/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/miko/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torchaudio/transforms/_transforms.py", line 619, in forward
specgram = self.spectrogram(waveform)
File "/home/miko/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/miko/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torchaudio/transforms/_transforms.py", line 110, in forward
return F.spectrogram(
File "/home/miko/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torchaudio/functional/functional.py", line 119, in spectrogram
frame_length_norm, window_norm = _get_spec_norms(normalized)
File "/home/miko/miniconda3/envs/speaker_recog/lib/python3.10/site-packages/torchaudio/functional/functional.py", line 233, in _get_spec_norms
if torch.jit.isinstance(normalized, str):

We are going to release soon more accurate models pretrained on voxblink2 + cnceleb + vox2 and we'll finetune them with different features, that are based on conv1d operations and that should be convertable to onnx.

pongthang commented 1 month ago

Thank you for your reply. "that are based on conv1d operations and that should be convertable to onnx" , will it be similar to https://github.com/adobe-research/convmelspec , "Convmelspec: Convertible Melspectrograms via 1D Convolutions"

vanIvan commented 1 month ago

Yes, it will be similar in a way that both solutions are using convolution of signal and discrete fourier transform kernels, but implementations will differ.

pongthang commented 1 month ago

Yes, it will be similar in a way that both solutions are using convolution of signal and discrete fourier transform kernels, but implementations will differ.

Hi, I try to find a solution for this while you are developing the custom spectrogram implementation. I follow one github issue and I can export the redimnet model to onnx successfully . But I am not sure how it affects the model performance. Here is the modification: inside - torchaudio/functional/functional.py --> def spectrogram() line - 126

     .....
    # default values are consistent with librosa.core.spectrum._spectrogram
    spec_f = torch.stft(
        input=waveform,
        n_fft=n_fft,
        hop_length=hop_length,
        win_length=win_length,
        window=window,
        center=center,
        pad_mode=pad_mode,
        normalized=frame_length_norm,
        onesided=onesided,
        # return_complex=True,
        return_complex=False, #new changes
    )

    # From imaginary and real values to absolute value
    spec_f = torch.sqrt(torch.pow(spec_f[:, :, :, 0], 2.0) + torch.pow(spec_f[:, :, :, 1], 2.0)) # adding this newline
    # unpack batch
    ...

Code use for exporting:

model.eval()
with torch.amp.autocast("cuda", enabled=False):
    with torch.no_grad():
        torch.onnx.export(
        model, 
        input_sample, 
        "model_success_redimnet.onnx",
    )

Is this good enough or will this affect the model performance?

vanIvan commented 1 week ago

@pongthang thanks for looking into it, I'm sorry, but we currently don't have resources and time for evaluation of the method proposed by you. It should be pretty simple to check model performance after conversion.

vanIvan commented 1 week ago

Happy to share good news, we have released first models pretrained on voxblink2, you could find them on evaluation page. The first example there is with the best voxblink2 finetuned model. This models have the same features, but we are going to update audio features to convertable with next bunch of models.

tebie6 commented 1 day ago

Yes, it will be similar in a way that both solutions are using convolution of signal and discrete fourier transform kernels, but implementations will differ.是的，这两种解决方案都使用信号卷积和离散傅立叶变换内核，这在某种程度上是相似的，但实现方式会有所不同。

Hi, I try to find a solution for this while you are developing the custom spectrogram implementation. I follow one github issue and I can export the redimnet model to onnx successfully . But I am not sure how it affects the model performance.您好，在您开发自定义频谱图实现时，我尝试为此找到解决方案。我关注了一个github问题，我可以成功将redimnet模型导出到onnx。但我不确定它如何影响模型性能。 Here is the modification:这是修改： inside - torchaudio/functional/functional.py --> def spectrogram()里面 - torchaudio/features/functions.py --> def 频谱图() line - 126 线路 - 126

     .....
    # default values are consistent with librosa.core.spectrum._spectrogram
    spec_f = torch.stft(
        input=waveform,
        n_fft=n_fft,
        hop_length=hop_length,
        win_length=win_length,
        window=window,
        center=center,
        pad_mode=pad_mode,
        normalized=frame_length_norm,
        onesided=onesided,
        # return_complex=True,
        return_complex=False, #new changes
    )

    # From imaginary and real values to absolute value
    spec_f = torch.sqrt(torch.pow(spec_f[:, :, :, 0], 2.0) + torch.pow(spec_f[:, :, :, 1], 2.0)) # adding this newline
    # unpack batch
    ...

Code use for exporting: 导出时使用的代码：

model.eval()
with torch.amp.autocast("cuda", enabled=False):
    with torch.no_grad():
        torch.onnx.export(
        model, 
        input_sample, 
        "model_success_redimnet.onnx",
    )

Is this good enough or will this affect the model performance?这足够好还是会影响模型性能？

@pongthang Hi, it’s great to hear that ONNX can be exported. How is the performance and speed of ONNX?

pongthang commented 1 day ago

@pongthang Hi, it’s great to hear that ONNX can be exported. How is the performance and speed of ONNX?

Hi, performance is good , and same as pytorch . Speed is also similar.