add additional paramater 'use_ncnn_network_format'

[HubertSotnowski]

Many networks have the capability to run inference via ncnn, but require their .param file to be edited and adjust accordingly to properly work.

Currently if you give vs-ncnn a onnx network it results in onnx2ncnn giving similar output to this:

Shape not supported yet!
Shape not supported yet!
Cast not supported yet!
  # to=7
Cast not supported yet!
  # to=7
Cast not supported yet!
  # to=7
Cast not supported yet!
  # to=7
Unsupported unsqueeze axes !
Unsupported unsqueeze axes !
Unsupported unsqueeze axes !
Unsupported unsqueeze axes !
Shape not supported yet!
Cast not supported yet!
  # to=7
ConstantOfShape not supported yet!
  # value 4
Unsupported slice step !
Cast not supported yet!
  # to=7
Unknown data type 0

This is easily fixable by editing the .param and replacing ops and adjusting layers. Could we maybe have an additional paramater use_ncnn_network_format to skip the onnx2ncnn step and give vs-ncnn .bin and .param directly? This would allow for much more networks to run.

[WolframRhodium]

Thanks for your suggestion, this could be implemented later, would you like to share your model?

[HubertSotnowski]

Alright, here you go, this happens with most networks though, I don't want you guys to fix the model in particular, just the flexibility to run more networks in general, which is impossible without editing param.

https://cdn.discordapp.com/attachments/1014241789948993577/1095448166033272862/shufflecugan.onnx

This should be quite easy to do, as you already load bin and param after you convert with onnx2ncnn in memory.

[WolframRhodium]

But then one have to manually specify number of input channels, number of output channels and tile size for both input and output.

[WolframRhodium]

usage:

in_tile_channels = 3
in_tile_height = 128
in_tile_width = 128
out_tile_channels = 3
out_tile_height = 256
out_tile_width = 256

ncnn_shape_hint = (in_tile_channels, in_tile_height, in_tile_width, out_tile_channels, out_tile_height, out_tile_width)

output = core.ncnn.Model(rgbs, "RealESRGANv2/RealESRGANv2-animevideo-xsx2-opt.bin", use_ncnn_network_format=True, ncnn_shape_hint=ncnn_shape_hint)

[Bercraft]

How to use this? can we have an example thank you.

i wrote :

scaler_output2 = core.resize.Lanczos( filter_output6, width = 1920, height = 1080, format = vs.RGBS, matrix_in_s = "709", )

in_tile_channels = 3 in_tile_height = 128 in_tile_width = 128 out_tile_channels = 3 out_tile_height = 256 out_tile_width = 256

ncnn_shape_hint = (in_tile_channels, in_tile_height, in_tile_width, out_tile_channels, out_tile_height, out_tile_width)

rife_output = core.ncnn.Model(scaler_output2, "G:\python-interp\vs-plugins\models\flownet.bin", use_ncnn_network_format=True, ncnn_shape_hint=ncnn_shape_hint)

video_output = core.resize.Lanczos( rife_output, width = 1920, height = 1080,
format=vs.YUV420P8, matrix_in_s = "709", matrix_s = "709", )

video_output.set_output()

i got this :

Script evaluation failed:

Python exception: Model: Function does not take argument(s) named use_ncnn_network_format, ncnn_shape_hint

Traceback (most recent call last): File "src\cython\vapoursynth.pyx", line 3233, in vapoursynth._vpy_evaluate File "src\cython\vapoursynth.pyx", line 3234, in vapoursynth._vpy_evaluate File "interp.vpy", line 324, in rife_output = core.ncnn.Model(scaler_output2, "G:\python-interp\vs-plugins\models\flownet.bin", use_ncnn_network_format=True, ncnn_shape_hint=ncnn_shape_hint) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "src\cython\vapoursynth.pyx", line 2937, in vapoursynth.Function.call vapoursynth.Error: Model: Function does not take argument(s) named use_ncnn_network_format, ncnn_shape_hint

[WolframRhodium]

The branch ncnn-format has not been merged due to inactivity of this issue.

If you are trying to run the RIFE model, it will not work since some code required is missing in the main ncnn repository.