"Unsupported node" when saving Faster RCNN eval model in ONNX format

[riverar]

I'm trying to go down the WinML route, so I made a change to fasterRCNN_train.py to spit out ONNX models eval_model.save(model_path, format=cntk.ModelFormat.ONNX). The CNTK, however, throws an exception:

Traceback (most recent call last):
  File "run_faster_rcnn.py", line 35, in <module>
    trained_model = train_faster_rcnn(cfg)
  File "C:\Users\Rafael\Desktop\CNTK-release-latest\Examples\Image\Detection\FasterRCNN\FasterRCNN_train.py", line 295, in train_faster_rcnn
    eval_model.save(model_path, format=cntk.ModelFormat.ONNX)
  File "C:\Users\Rafael\py3\lib\site-packages\cntk\internal\swig_helper.py", line 69, in wrapper
    result = f(*args, **kwds)
  File "C:\Users\Rafael\py3\lib\site-packages\cntk\ops\functions.py", line 1504, in save
    return super(Function, self).save(filename, format.value)
  File "C:\Users\Rafael\py3\lib\site-packages\cntk\cntk_py.py", line 2028, in save
    return _cntk_py.Function_save(self, *args)
RuntimeError: Node 'Combine: Output('cls_pred', [#], [* x 2]), Output('rpn_rois', [#], [* x 4]), Output('bbox_regr', [#], [* x 8]) -> Output('cls_pred', [#], [* x 2]), Output('rpn_rois', [#], [* x 4]), Output('bbox_regr', [#], [* x 8])': Unsupported node.

[CALL STACK]
    > CNTK::Internal::  UseSparseGradientAggregationInDataParallelSGD
    - CNTK::  XavierInitializer (x4)
    - CNTK::Function::  Save
    - PyInit__cntk_py (x2)
    - PyEval_EvalFrameDefault
    - Py_CheckFunctionResult
    - PyObject_CallFunctionObjArgs
    - PyEval_EvalFrameDefault
    - Py_CheckFunctionResult
    - PyList_Size
    - PyEval_EvalFrameDefault
    - Py_CheckFunctionResult

What's the guidance for getting this model into WinML?

[serenaz]

Unfortunately, for this specific model (Fast-RCNN), the CNTK implementation uses some custom code that isn’t exportable to ONNX :(

[ke1337]

Is it the same as #2989 ? Combine is not supported in ONNX. Can you try export without combine? However, Fast-RCNN relies on NMS python modules that may not available in ONNX as @serenaz mentioned. @spandantiwari may have info on ONNX support for NMS in Fast-RCNN.

[ebarsoumMS]

As @serenaz said, we have custom python code in Fast and Faster R-CNN that need to be ported to C++, in addition we need to add TopK ONNX support.

[spandantiwari]

That's correct. Fast and FasterRCNN use custom Python code (UDFs) that are need to be ported to CNTK/ONNX ops fully before it can be exported to ONNX. We will let you know when the models are ready for ONNX export.

[riverar]

@spandantiwari I don't think it's appropriate to close this issue until Fast(er)RCNN support ONNX export.

[spandantiwari]

@riverar There have been multiple requests for this and related work. They will be tracked by https://github.com/Microsoft/CNTK/issues/3052.

[riverar]

@spandantiwari Great, thanks. Will keep an eye on that. It's a little weird you opened a new issue instead of simply using this one.