I want to use dynamic weights in F.conv2d in pytorch.

[jjxxmiin]

I want to use dynamic weights in F.conv2d in pytorch.

    def forward(self, x, style):
        b, c, h, w = x.shape  # c = c_in

        style = self.modulation(style).view(b, 1, c, 1, 1)
        weight = self.weight * style  # (b, c_out, c_in, k, k)
        weight = weight.view(b * self.out_channels, c, self.kernel_size, self.kernel_size)

        b, c, h, w = x.shape
        x = x.view(1, b * c, h, w)

        out = F.conv2d(x, weight, padding=self.padding, groups=b)
        out = out.view(b, self.out_channels, *out.shape[2:4])

        return out

However, only constant weights are supported.

Is there any way to solve this?

Asset import failed, "Assets/Scenes/Face Parsing/gfpgan.onnx" > OnnxImportException: Unexpected error while parsing layer 649 of type Conv.
Currently only constant tensors are supported for `W` input in node of type Conv. Instead 649.W is pointing to non constant node 628.

Json: { "input": [ "648", "628", "stylegan_decoder.style_conv1.modulated_conv.conv2d.bias" ], "output": [ "649" ], "name": "Conv_238", "opType": "Conv", "attribute": [ { "name": "dilations", "ints": [ "1", "1" ], "type": "INTS" }, { "name": "group", "i": "1", "type": "INT" }, { "name": "kernel_shape", "ints": [ "3", "3" ], "type": "INTS" }, { "name": "pads", "ints": [ "1", "1", "1", "1" ], "type": "INTS" }, { "name": "strides", "ints": [ "1", "1" ], "type": "INTS" } ] }
  at Unity.Barracuda.ONNX.ONNXNodeWrapper.GetRequiredInputAsConstant (System.String input, System.String onnxLayout, System.String onnxName) [0x0008f] in C:\Users\user\Desktop\Git_store\unityaiproject\Library\PackageCache\com.unity.barracuda@3.0.0\Barracuda\Runtime\ONNX\ONNXNodeWrapper.cs:236 
  at Unity.Barracuda.ONNX.ONNXNodeWrapper.Input1Constant (System.String onnxLayout, System.String name) [0x00000] in C:\Users\user\Desktop\Git_store\unityaiproject\Library\PackageCache\com.unity.barracuda@3.0.0\Barracuda\Runtime\ONNX\ONNXNodeWrapper.cs:73 
  at Unity.Barracuda.ONNX.ONNXModelConverter+<>c__DisplayClass27_0.<UseStandardImporter>b__87 (Unity.Barracuda.ModelBuilder net, Unity.Barracuda.ONNX.ONNXNodeWrapper node) [0x00030] in C:\Users\user\Desktop\Git_store\unityaiproject\Library\PackageCache\com.unity.barracuda@3.0.0\Barracuda\Runtime\ONNX\ONNXModelConverter.cs:946 
  at Unity.Barracuda.ONNX.ONNXModelConverter.ConvertOnnxModel (Onnx.ModelProto onnxModel) [0x00409] in C:\Users\user\Desktop\Git_store\unityaiproject\Library\PackageCache\com.unity.barracuda@3.0.0\Barracuda\Runtime\ONNX\ONNXModelConverter.cs:2934 

Unity.Barracuda.ONNX.ONNXModelConverter.Err (Unity.Barracuda.Model model, System.String layerName, System.String message, System.String extendedMessage, System.String debugMessage) (at Library/PackageCache/com.unity.barracuda@3.0.0/Barracuda/Runtime/ONNX/ONNXModelConverter.cs:3434)
Unity.Barracuda.ONNX.ONNXModelConverter.ConvertOnnxModel (Onnx.ModelProto onnxModel) (at Library/PackageCache/com.unity.barracuda@3.0.0/Barracuda/Runtime/ONNX/ONNXModelConverter.cs:2943)
Unity.Barracuda.ONNX.ONNXModelConverter.Convert (Google.Protobuf.CodedInputStream inputStream) (at Library/PackageCache/com.unity.barracuda@3.0.0/Barracuda/Runtime/ONNX/ONNXModelConverter.cs:170)
Unity.Barracuda.ONNX.ONNXModelConverter.Convert (System.String filePath) (at Library/PackageCache/com.unity.barracuda@3.0.0/Barracuda/Runtime/ONNX/ONNXModelConverter.cs:98)
Unity.Barracuda.ONNXModelImporter.OnImportAsset (UnityEditor.AssetImporters.AssetImportContext ctx) (at Library/PackageCache/com.unity.barracuda@3.0.0/Barracuda/Editor/ONNXModelImporter.cs:65)
UnityEditor.AssetImporters.ScriptedImporter.GenerateAssetData (UnityEditor.AssetImporters.AssetImportContext ctx) (at <68089899e4c84456bfc1de3436accf4a>:0)
UnityEditorInternal.InternalEditorUtility:ProjectWindowDrag(HierarchyProperty, Boolean)
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr, Boolean&)

UnityEngine.GUIUtility:ProcessEvent (int,intptr,bool&)

[AlexRibard]

Right now it is not possible unfortunately. We only support constant weights coming in from onnx. However, you can call the op individually:

var op = new ReferenceComputeOps();
var res = op.Conv2D(X,W,B);

(Only use ReferenceCompute or Burst op for this, for ComputeOp, we expect the weights to be stored in a particular packing with the bias)

[magicse]

Hi @jjeamin could You explain please how convert stylegan2 to onnx format?

[jjxxmiin]

@magicse Unfortunately I have not been able to solve it yet.

[jjxxmiin]

https://github.com/TencentARC/GFPGAN/issues/184#event-7435266857