search

sonos / tract

Tiny, no-nonsense, self-contained, Tensorflow and ONNX inference
Other
2.18k stars 210 forks source link

Failed to dump ONNX model to NNEF #1167

Closed mush42 closed 1 year ago

mush42 commented 1 year ago

Hello

Attempting to convert a standard transformer model from onnx format to nnef format fails.

The model works as expected in onnx format when ran using tract-onnx.

Here is the command I used for conversion:

tract --nnef-extended-identifier --nnef-tract-pulse --nnef-tract-onnx --nnef-tract-core  -i 1,300,int64,1,1,int64 model.onnx dump --nnef model.nnef.tgz

And here is the output I get:

Command output

┏ 1 Source input_lengths ┃ ━━━ 1,1,I64 ┣┻ 108 Squeeze13 /Squeeze ┃ ━━━ 1,I64 ┣┻ 109 Unsqueeze13 /Unsqueeze ┃ ━━━ 1,1,I64 ┣┻ 110 Less /Less ┃ ━━━ 1,300,Bool ┣┓ ┃┣┻ 144 Reshape /transformer/attn_layers/layers.0.1/Reshape_10 ┃┃ ━━━ 1,300,1,Bool ┃┣ 145 Not /transformer/attn_layers/layers.0.1/Not_2 ┃┃┏ 0 Source char_inputs ┃┃┃ ━━━ 1,300,int64,1,1,int64,? ┃┃┣┻ 111 Gather /transformer/token_emb/emb/Gather ┃┃┃ ━━━ 1,300,144,? ┃┃┣┻ 112 Add /transformer/Add ┃┃┣┓ ┃┃┃┣ 113 Reduce /transformer/attn_layers/layers.0.0.0/ReduceMean ┃┃┃┃ ━━━ 1,300,1,? ┃┃┣━┓┃ ┃┃┃┣┻ 114 Sub /transformer/attn_layers/layers.0.0.0/Sub ┃┃┃┣┓ ┃┃┃┃┣┻ 115 Pow /transformer/attn_layers/layers.0.0.0/Pow ┃┃┃┃┣ 116 Reduce /transformer/attn_layers/layers.0.0.0/ReduceMean_1 ┃┃┃┃┃ ━━━ 1,300,1,? ┃┃┃┃┣┻ 117 Add /transformer/attn_layers/layers.0.0.0/Add ┃┃┃┃┣ 118 Sqrt /transformer/attn_layers/layers.0.0.0/Sqrt ┃ ┃┃┃┣┻ 119 Div /transformer/attn_layers/layers.0.0.0/Div ┃┃┃┣┻ 120 Mul /transformer/attn_layers/layers.0.0.0/Mul ┃┃┃┣┻ 121 Add /transformer/attn_layers/layers.0.0.0/Add_1 ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 122 MatMulInference 880 ┃┃┃┃ ━━━ 1,300,1536,F32 ┃┃┃┣╋┳ 123 Split13 881 ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,F32 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 124 Reshape /transformer/attn_layers/layers.0.1/Reshape ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 125 PermuteAxes /transformer/attn_layers/layers.0.1/Transpose ┃┃┃┃┃┃ ━━━ 1,8,300,64,? ┃┃┃┃┃┣┻ 134 Mul /transformer/attn_layers/layers.0.1/attend/Mul_1 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 126 Reshape /transformer/attn_layers/layers.0.1/Reshape_2 ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 133 PermuteAxes /transformer/attn_layers/layers.0.1/attend/Transpose ┃┃┃┃┃┃ ━━━ 1,8,64,300,? ┃┃┃┃┃┣┻ 135 Mul /transformer/attn_layers/layers.0.1/attend/Mul_2 ┃┃┃┃┣┻ 136 MatMulInference /transformer/attn_layers/layers.0.1/attend/MatMul ┃┃┃┃┃ ━━━ 1,8,300,300,? ┗━━━┓ ┃┃┃┃┣┻ 129 Reshape /transformer/attn_layers/layers.0.1/Reshape_7 ┃┃┃┃┃ ━━━ 1,1,1,300,Bool ┃┃┃┃┣ 130 Not /transformer/attn_layers/layers.0.1/Not ┃┃┃┃┣ 131 Not /transformer/attn_layers/layers.0.1/Not_1 ┃┃┃┃┣┻ 132 MultiBroadcastTo /transformer/attn_layers/layers.0.1/attend/Expand ┃┃┃┃┃ ━━━ 1,8,300,300,Bool ┃┃┃┃┣┻┻ 137 Iff /transformer/attn_layers/layers.0.1/attend/Where_1 ┃┃┃┃┃ ━━━ 1,8,300,300,F32 ┃┃┃┃┣┓ ┃┃┃┗┓┃ ┃┃┃┃┣┻ 138 Add /transformer/attn_layers/layers.0.1/attend/Add ┃┃┃┃┃ ━━━ 1,8,300,300,F32 ┃┃┃┃┣ 139 Softmax /transformer/attn_layers/layers.0.1/attend/Softmax ┃┃┗━┓ ┃┃┃┃┣┻ 127 Reshape /transformer/attn_layers/layers.0.1/Reshape_4 ┃┃┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┃┃┣ 128 PermuteAxes /transformer/attn_layers/layers.0.1/Transpose_2 ┃┃┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┃┣┻ 140 MatMulInference /transformer/attn_layers/layers.0.1/attend/MatMul_1 ┃┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┃┣ 141 PermuteAxes /transformer/attn_layers/layers.0.1/Transpose_3 ┃┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┃┣┻ 142 Reshape /transformer/attn_layers/layers.0.1/Reshape_9 ┃┃┃┃ ━━━ 1,300,512,F32 ┃┃┃┣┻ 143 MatMulInference /transformer/attn_layers/layers.0.1/to_out/MatMul ┃┃┃┃ ━━━ 1,300,144,F32 ┣━━━┓ ┃ ┃┃┃┣┻┻ 146 Iff /transformer/attn_layers/layers.0.1/Where ┃┃┃┃ ━━━ 1,300,144,F32 ┃┗━┓ ┃┃┣┻ 147 Add /transformer/attn_layers/layers.0.2/Add ┃┃┃ ━━━ 1,300,144,? ┃┃┣┓ ┃┃┃┣ 148 Reduce /transformer/attn_layers/layers.1.0.0/ReduceMean ┃┃┃┃ ━━━ 1,300,1,? ┃┃┣━┓┃ ┃┃┃┣┻ 149 Sub /transformer/attn_layers/layers.1.0.0/Sub ┃┃┃┣┓ ┃┃┃┃┣┻ 150 Pow /transformer/attn_layers/layers.1.0.0/Pow ┃┃┃┃┣ 151 Reduce /transformer/attn_layers/layers.1.0.0/ReduceMean_1 ┃┃┃┃┃ ━━━ 1,300,1,? ┃┃┃┃┣┻ 152 Add /transformer/attn_layers/layers.1.0.0/Add ┃┃┃┃┣ 153 Sqrt /transformer/attn_layers/layers.1.0.0/Sqrt ┃┃┃┣┻ 154 Div /transformer/attn_layers/layers.1.0.0/Div ┃┃┃┣┻ 155 Mul /transformer/attn_layers/layers.1.0.0/Mul ┃┃┃┣┻ 156 Add /transformer/attn_layers/layers.1.0.0/Add_1 ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 157 MatMulInference /transformer/attn_layers/layers.1.1/ff/ff.0/ff.0.0/MatMul ┃┃┃┃ ━━━ 1,300,576,F32 ┃┃┃┣┻ 158 Add /transformer/attn_layers/layers.1.1/ff/ff.0/ff.0.0/Add ┃┃┃┃ ━━━ 1,300,576,F32 ┃┃┃┣┓ ┃┃┃┃┣┻ 159 Div /transformer/attn_layers/layers.1.1/ff/ff.0/ff.0.1/Div ┃┃┃┃┣ 160 Erf /transformer/attn_layers/layers.1.1/ff/ff.0/ff.0.1/Erf ┃┃┃┃┣┻ 161 Add /transformer/attn_layers/layers.1.1/ff/ff.0/ff.0.1/Add ┃ ┃┃┃┣┻ 162 Mul /transformer/attn_layers/layers.1.1/ff/ff.0/ff.0.1/Mul ┃┃┃┣┻ 163 Mul /transformer/attn_layers/layers.1.1/ff/ff.0/ff.0.1/Mul_1 ┃┃┃┣┻ 164 MatMulInference /transformer/attn_layers/layers.1.1/ff/ff.3/MatMul ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 165 Add /transformer/attn_layers/layers.1.1/ff/ff.3/Add ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┗┓ ┃┃┣┻ 166 Add /transformer/attn_layers/layers.1.2/Add ┃┃┃ ━━━ 1,300,144,? ┃┃┣┓ ┃┃┃┣ 167 Reduce /transformer/attn_layers/layers.2.0.0/ReduceMean ┃┃┃┃ ━━━ 1,300,1,? ┃┃┣━┓┃ ┃┃┃┣┻ 168 Sub /transformer/attn_layers/layers.2.0.0/Sub ┃┃┃┣┓ ┃┃┃┃┣┻ 169 Pow /transformer/attn_layers/layers.2.0.0/Pow ┃┃┃┃┣ 170 Reduce /transformer/attn_layers/layers.2.0.0/ReduceMean_1 ┃┃┃┃┃ ━━━ 1,300,1,? ┃┃┃┃┣┻ 171 Add /transformer/attn_layers/layers.2.0.0/Add ┃┃┃┃┣ 172 Sqrt /transformer/attn_layers/layers.2.0.0/Sqrt ┃┃┃┣┻ 173 Div /transformer/attn_layers/layers.2.0.0/Div ┃┃┃┣┻ 174 Mul /transformer/attn_layers/layers.2.0.0/Mul ┃┃┃┣┻ 175 Add /transformer/attn_layers/layers.2.0.0/Add_1 ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 176 MatMulInference 887 ┃┃┃┃ ━━━ 1,300,1536,F32 ┃┃┃┣╋┳ 177 Split13 888 ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,F32 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 178 Reshape /transformer/attn_layers/layers.2.1/Reshape ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 179 PermuteAxes /transformer/attn_layers/layers.2.1/Transpose ┃┃┃┃┃┃ ━━━ 1,8,300,64,? ┃┃┃┃┃┣┻ 184 Mul /transformer/attn_layers/layers.2.1/attend/Mul_1 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 180 Reshape /transformer/attn_layers/layers.2.1/Reshape_2 ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 183 PermuteAxes /transformer/attn_layers/layers.2.1/attend/Transpose ┃┃┃┃┃┃ ━━━ 1,8,64,300,? ┃┃┃┃┃┣┻ 185 Mul /transformer/attn_layers/layers.2.1/attend/Mul_2 ┃┃┃┃┣┻ 186 MatMulInference /transformer/attn_layers/layers.2.1/attend/MatMul ┃┃┃┃┃ ━━━ 1,8,300,300,? ┃┣━━━┓ ┃┃┃┃┣┻ 187 Add /transformer/attn_layers/layers.2.1/attend/Add ┃┃┃┃┃ ━━━ 1,8,300,300,F32 ┃┃┃┃┣ 188 Softmax /transformer/attn_layers/layers.2.1/attend/Softmax ┃┃┃┗┓ ┃┃┃┃┣┻ 181 Reshape /transformer/attn_layers/layers.2.1/Reshape_4 ┃┃┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┃┃┣ 182 PermuteAxes /transformer/attn_layers/layers.2.1/Transpose_2 ┃┃┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┃┣┻ 189 MatMulInference /transformer/attn_layers/layers.2.1/attend/MatMul_1 ┃┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┃┣ 190 PermuteAxes /transformer/attn_layers/layers.2.1/Transpose_3 ┃┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┃┣┻ 191 Reshape /transformer/attn_layers/layers.2.1/Reshape_7 ┃┃┃┃ ━━━ 1,300,512,F32 ┃┃┃┣┻ 192 MatMulInference /transformer/attn_layers/layers.2.1/to_out/MatMul ┃┃┃┃ ━━━ 1,300,144,F32 ┣━━━┓ ┃ ┃┃┃┣┻┻ 193 Iff /transformer/attn_layers/layers.2.1/Where ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┗┓ ┃┃┣┻ 194 Add /transformer/attn_layers/layers.2.2/Add ┃┃┃ ━━━ 1,300,144,? ┃┃┣┓ ┃┃┃┣ 195 Reduce /transformer/attn_layers/layers.3.0.0/ReduceMean ┃┃┃┃ ━━━ 1,300,1,? ┃┃┣━┓┃ ┃┃┃┣┻ 196 Sub /transformer/attn_layers/layers.3.0.0/Sub ┃┃┃┣┓ ┃┃┃┃┣┻ 197 Pow /transformer/attn_layers/layers.3.0.0/Pow ┃┃┃┃┣ 198 Reduce /transformer/attn_layers/layers.3.0.0/ReduceMean_1 ┃┃┃┃┃ ━━━ 1,300,1,? ┃┃┃┃┣┻ 199 Add /transformer/attn_layers/layers.3.0.0/Add ┃┃┃┃┣ 200 Sqrt /transformer/attn_layers/layers.3.0.0/Sqrt ┃┃┃┣┻ 201 Div /transformer/attn_layers/layers.3.0.0/Div ┃┃┃┣┻ 202 Mul /transformer/attn_layers/layers.3.0.0/Mul ┃┃┃┣┻ 203 Add /transformer/attn_layers/layers.3.0.0/Add_1 ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 204 MatMulInference /transformer/attn_layers/layers.3.1/ff/ff.0/ff.0.0/MatMul ┃┃┃┃ ━━━ 1,300,576,F32 ┃┃┃┣┻ 205 Add /transformer/attn_layers/layers.3.1/ff/ff.0/ff.0.0/Add ┃┃┃┃ ━━━ 1,300,576,F32 ┃┃┃┣┓ ┃┃┃┃┣┻ 206 Div /transformer/attn_layers/layers.3.1/ff/ff.0/ff.0.1/Div ┃┃┃┃┣ 207 Erf /transformer/attn_layers/layers.3.1/ff/ff.0/ff.0.1/Erf ┃┃┃┃┣┻ 208 Add /transformer/attn_layers/layers.3.1/ff/ff.0/ff.0.1/Add ┃┃┃┣┻ 209 Mul /transformer/attn_layers/layers.3.1/ff/ff.0/ff.0.1/Mul ┃┃┃┣┻ 210 Mul /transformer/attn_layers/layers.3.1/ff/ff.0/ff.0.1/Mul_1 ┃┃┃┣┻ 211 MatMulInference /transformer/attn_layers/layers.3.1/ff/ff.3/MatMul ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 212 Add /transformer/attn_layers/layers.3.1/ff/ff.3/Add ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┗┓ ┃┃┣┻ 213 Add /transformer/attn_layers/layers.3.2/Add ┃┃┃ ━━━ 1,300,144,? ┃┃┣┓ ┃┃┃┣ 214 Reduce /transformer/attn_layers/layers.4.0.0/ReduceMean ┃┃┃┃ ━━━ 1,300,1,? ┃┃┣━┓┃ ┃┃┃┣┻ 215 Sub /transformer/attn_layers/layers.4.0.0/Sub ┃┃┃┣┓ ┃┃┃┃┣┻ 216 Pow /transformer/attn_layers/layers.4.0.0/Pow ┃┃┃┃┣ 217 Reduce /transformer/attn_layers/layers.4.0.0/ReduceMean_1 ┃┃┃┃┃ ━━━ 1,300,1,? ┃┃┃┃┣┻ 218 Add /transformer/attn_layers/layers.4.0.0/Add ┃┃┃┃┣ 219 Sqrt /transformer/attn_layers/layers.4.0.0/Sqrt ┃┃┃┣┻ 220 Div /transformer/attn_layers/layers.4.0.0/Div ┃┃┃┣┻ 221 Mul /transformer/attn_layers/layers.4.0.0/Mul ┃┃┃┣┻ 222 Add /transformer/attn_layers/layers.4.0.0/Add_1 ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 223 MatMulInference 894 ┃┃┃┃ ━━━ 1,300,1536,F32 ┃┃┃┣╋┳ 224 Split13 895 ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,F32 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 225 Reshape /transformer/attn_layers/layers.4.1/Reshape ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 226 PermuteAxes /transformer/attn_layers/layers.4.1/Transpose ┃┃┃┃┃┃ ━━━ 1,8,300,64,? ┃┃┃┃┃┣┻ 231 Mul /transformer/attn_layers/layers.4.1/attend/Mul_1 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 227 Reshape /transformer/attn_layers/layers.4.1/Reshape_2 ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 230 PermuteAxes /transformer/attn_layers/layers.4.1/attend/Transpose ┃┃┃┃┃┃ ━━━ 1,8,64,300,? ┃┃┃┃┃┣┻ 232 Mul /transformer/attn_layers/layers.4.1/attend/Mul_2 ┃┃┃┃┣┻ 233 MatMulInference /transformer/attn_layers/layers.4.1/attend/MatMul ┃┃┃┃┃ ━━━ 1,8,300,300,? ┃┣━━━┓ ┃┃┃┃┣┻ 234 Add /transformer/attn_layers/layers.4.1/attend/Add ┃┃┃┃┃ ━━━ 1,8,300,300,F32 ┃┃┃┃┣ 235 Softmax /transformer/attn_layers/layers.4.1/attend/Softmax ┃┃┃┗┓ ┃┃┃┃┣┻ 228 Reshape /transformer/attn_layers/layers.4.1/Reshape_4 ┃┃┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┃┃┣ 229 PermuteAxes /transformer/attn_layers/layers.4.1/Transpose_2 ┃┃┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┃┣┻ 236 MatMulInference /transformer/attn_layers/layers.4.1/attend/MatMul_1 ┃┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┃┣ 237 PermuteAxes /transformer/attn_layers/layers.4.1/Transpose_3 ┃┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┃┣┻ 238 Reshape /transformer/attn_layers/layers.4.1/Reshape_7 ┃┃┃┃ ━━━ 1,300,512,F32 ┃┃┃┣┻ 239 MatMulInference /transformer/attn_layers/layers.4.1/to_out/MatMul ┃┃┃┃ ━━━ 1,300,144,F32 ┣━━━┓ ┃ ┃┃┃┣┻┻ 240 Iff /transformer/attn_layers/layers.4.1/Where ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┗┓ ┃┃┣┻ 241 Add /transformer/attn_layers/layers.4.2/Add ┃┃┃ ━━━ 1,300,144,? ┃┃┣┓ ┃┃┃┣ 242 Reduce /transformer/attn_layers/layers.5.0.0/ReduceMean ┃┃┃┃ ━━━ 1,300,1,? ┃┃┣━┓┃ ┃┃┃┣┻ 243 Sub /transformer/attn_layers/layers.5.0.0/Sub ┃┃┃┣┓ ┃┃┃┃┣┻ 244 Pow /transformer/attn_layers/layers.5.0.0/Pow ┃┃┃┃┣ 245 Reduce /transformer/attn_layers/layers.5.0.0/ReduceMean_1 ┃┃┃┃┃ ━━━ 1,300,1,? ┃┃┃┃┣┻ 246 Add /transformer/attn_layers/layers.5.0.0/Add ┃┃┃┃┣ 247 Sqrt /transformer/attn_layers/layers.5.0.0/Sqrt ┃┃┃┣┻ 248 Div /transformer/attn_layers/layers.5.0.0/Div ┃┃┃┣┻ 249 Mul /transformer/attn_layers/layers.5.0.0/Mul ┃┃┃┣┻ 250 Add /transformer/attn_layers/layers.5.0.0/Add_1 ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 251 MatMulInference /transformer/attn_layers/layers.5.1/ff/ff.0/ff.0.0/MatMul ┃┃┃┃ ━━━ 1,300,576,F32 ┃┃┃┣┻ 252 Add /transformer/attn_layers/layers.5.1/ff/ff.0/ff.0.0/Add ┃┃┃┃ ━━━ 1,300,576,F32 ┃┃┃┣┓ ┃┃┃┃┣┻ 253 Div /transformer/attn_layers/layers.5.1/ff/ff.0/ff.0.1/Div ┃┃┃┃┣ 254 Erf /transformer/attn_layers/layers.5.1/ff/ff.0/ff.0.1/Erf ┃┃┃┃┣┻ 255 Add /transformer/attn_layers/layers.5.1/ff/ff.0/ff.0.1/Add ┃┃┃┣┻ 256 Mul /transformer/attn_layers/layers.5.1/ff/ff.0/ff.0.1/Mul ┃┃┃┣┻ 257 Mul /transformer/attn_layers/layers.5.1/ff/ff.0/ff.0.1/Mul_1 ┃┃┃┣┻ 258 MatMulInference /transformer/attn_layers/layers.5.1/ff/ff.3/MatMul ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 259 Add /transformer/attn_layers/layers.5.1/ff/ff.3/Add ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┗┓ ┃┃┣┻ 260 Add /transformer/attn_layers/layers.5.2/Add ┃┃┃ ━━━ 1,300,144,? ┃┃┣┓ ┃┃┃┣ 261 Reduce /transformer/attn_layers/layers.6.0.0/ReduceMean ┃┃┃┃ ━━━ 1,300,1,? ┃┃┣━┓┃ ┃┃┃┣┻ 262 Sub /transformer/attn_layers/layers.6.0.0/Sub ┃┃┃┣┓ ┃┃┃┃┣┻ 263 Pow /transformer/attn_layers/layers.6.0.0/Pow ┃┃┃┃┣ 264 Reduce /transformer/attn_layers/layers.6.0.0/ReduceMean_1 ┃┃┃┃┃ ━━━ 1,300,1,? ┃┃┃┃┣┻ 265 Add /transformer/attn_layers/layers.6.0.0/Add ┃┃┃┃┣ 266 Sqrt /transformer/attn_layers/layers.6.0.0/Sqrt ┃┃┃┣┻ 267 Div /transformer/attn_layers/layers.6.0.0/Div ┃┃┃┣┻ 268 Mul /transformer/attn_layers/layers.6.0.0/Mul ┃┃┃┣┻ 269 Add /transformer/attn_layers/layers.6.0.0/Add_1 ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 270 MatMulInference 901 ┃┃┃┃ ━━━ 1,300,1536,F32 ┃┃┃┣╋┳ 271 Split13 902 ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,F32 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 272 Reshape /transformer/attn_layers/layers.6.1/Reshape ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 273 PermuteAxes /transformer/attn_layers/layers.6.1/Transpose ┃┃┃┃┃┃ ━━━ 1,8,300,64,? ┃┃┃┃┃┣┻ 278 Mul /transformer/attn_layers/layers.6.1/attend/Mul_1 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 274 Reshape /transformer/attn_layers/layers.6.1/Reshape_2 ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 277 PermuteAxes /transformer/attn_layers/layers.6.1/attend/Transpose ┃┃┃┃┃┃ ━━━ 1,8,64,300,? ┃┃┃┃┃┣┻ 279 Mul /transformer/attn_layers/layers.6.1/attend/Mul_2 ┃┃┃┃┣┻ 280 MatMulInference /transformer/attn_layers/layers.6.1/attend/MatMul ┃┃┃┃┃ ━━━ 1,8,300,300,? ┃┣━━━┓ ┃┃┃┃┣┻ 281 Add /transformer/attn_layers/layers.6.1/attend/Add ┃┃┃┃┃ ━━━ 1,8,300,300,F32 ┃┃┃┃┣ 282 Softmax /transformer/attn_layers/layers.6.1/attend/Softmax ┃┃┃┗┓ ┃┃┃┃┣┻ 275 Reshape /transformer/attn_layers/layers.6.1/Reshape_4 ┃┃┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┃┃┣ 276 PermuteAxes /transformer/attn_layers/layers.6.1/Transpose_2 ┃┃┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┃┣┻ 283 MatMulInference /transformer/attn_layers/layers.6.1/attend/MatMul_1 ┃┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┃┣ 284 PermuteAxes /transformer/attn_layers/layers.6.1/Transpose_3 ┃┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┃┣┻ 285 Reshape /transformer/attn_layers/layers.6.1/Reshape_7 ┃┃┃┃ ━━━ 1,300,512,F32 ┃┃┃┣┻ 286 MatMulInference /transformer/attn_layers/layers.6.1/to_out/MatMul ┃┃┃┃ ━━━ 1,300,144,F32 ┣━━━┓ ┃ ┃┃┃┣┻┻ 287 Iff /transformer/attn_layers/layers.6.1/Where ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┗┓ ┃┃┣┻ 288 Add /transformer/attn_layers/layers.6.2/Add ┃┃┃ ━━━ 1,300,144,? ┃┃┣┓ ┃┃┃┣ 289 Reduce /transformer/attn_layers/layers.7.0.0/ReduceMean ┃┃┃┃ ━━━ 1,300,1,? ┃┃┣━┓┃ ┃┃┃┣┻ 290 Sub /transformer/attn_layers/layers.7.0.0/Sub ┃┃┃┣┓ ┃┃┃┃┣┻ 291 Pow /transformer/attn_layers/layers.7.0.0/Pow ┃┃┃┃┣ 292 Reduce /transformer/attn_layers/layers.7.0.0/ReduceMean_1 ┃┃┃┃┃ ━━━ 1,300,1,? ┃┃┃┃┣┻ 293 Add /transformer/attn_layers/layers.7.0.0/Add ┃┃┃┃┣ 294 Sqrt /transformer/attn_layers/layers.7.0.0/Sqrt ┃┃┃┣┻ 295 Div /transformer/attn_layers/layers.7.0.0/Div ┃┃┃┣┻ 296 Mul /transformer/attn_layers/layers.7.0.0/Mul ┃┃┃┣┻ 297 Add /transformer/attn_layers/layers.7.0.0/Add_1 ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 298 MatMulInference /transformer/attn_layers/layers.7.1/ff/ff.0/ff.0.0/MatMul ┃┃┃┃ ━━━ 1,300,576,F32 ┃┃┃┣┻ 299 Add /transformer/attn_layers/layers.7.1/ff/ff.0/ff.0.0/Add ┃┃┃┃ ━━━ 1,300,576,F32 ┃┃┃┣┓ ┃┃┃┃┣┻ 300 Div /transformer/attn_layers/layers.7.1/ff/ff.0/ff.0.1/Div ┃┃┃┃┣ 301 Erf /transformer/attn_layers/layers.7.1/ff/ff.0/ff.0.1/Erf ┃┃┃┃┣┻ 302 Add /transformer/attn_layers/layers.7.1/ff/ff.0/ff.0.1/Add ┃┃┃┣┻ 303 Mul /transformer/attn_layers/layers.7.1/ff/ff.0/ff.0.1/Mul ┃┃┃┣┻ 304 Mul /transformer/attn_layers/layers.7.1/ff/ff.0/ff.0.1/Mul_1 ┃┃┃┣┻ 305 MatMulInference /transformer/attn_layers/layers.7.1/ff/ff.3/MatMul ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 306 Add /transformer/attn_layers/layers.7.1/ff/ff.3/Add ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┗┓ ┃┃┣┻ 307 Add /transformer/attn_layers/layers.7.2/Add ┃┃┃ ━━━ 1,300,144,? ┃┃┣┓ ┃┃┃┣ 308 Reduce /transformer/attn_layers/layers.8.0.0/ReduceMean ┃┃┃┃ ━━━ 1,300,1,? ┃┃┣━┓┃ ┃┃┃┣┻ 309 Sub /transformer/attn_layers/layers.8.0.0/Sub ┃┃┃┣┓ ┃┃┃┃┣┻ 310 Pow /transformer/attn_layers/layers.8.0.0/Pow ┃┃┃┃┣ 311 Reduce /transformer/attn_layers/layers.8.0.0/ReduceMean_1 ┃┃┃┃┃ ━━━ 1,300,1,? ┃┃┃┃┣┻ 312 Add /transformer/attn_layers/layers.8.0.0/Add ┃┃┃┃┣ 313 Sqrt /transformer/attn_layers/layers.8.0.0/Sqrt ┃┃┃┣┻ 314 Div /transformer/attn_layers/layers.8.0.0/Div ┃┃┃┣┻ 315 Mul /transformer/attn_layers/layers.8.0.0/Mul ┃┃┃┣┻ 316 Add /transformer/attn_layers/layers.8.0.0/Add_1 ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 317 MatMulInference 908 ┃┃┃┃ ━━━ 1,300,1536,F32 ┃┃┃┣╋┳ 318 Split13 909 ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,F32 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 319 Reshape /transformer/attn_layers/layers.8.1/Reshape ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 320 PermuteAxes /transformer/attn_layers/layers.8.1/Transpose ┃┃┃┃┃┃ ━━━ 1,8,300,64,? ┃┃┃┃┃┣┻ 325 Mul /transformer/attn_layers/layers.8.1/attend/Mul_1 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 321 Reshape /transformer/attn_layers/layers.8.1/Reshape_2 ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 324 PermuteAxes /transformer/attn_layers/layers.8.1/attend/Transpose ┃┃┃┃┃┃ ━━━ 1,8,64,300,? ┃┃┃┃┃┣┻ 326 Mul /transformer/attn_layers/layers.8.1/attend/Mul_2 ┃┃┃┃┣┻ 327 MatMulInference /transformer/attn_layers/layers.8.1/attend/MatMul ┃┃┃┃┃ ━━━ 1,8,300,300,? ┃┣━━━┓ ┃┃┃┃┣┻ 328 Add /transformer/attn_layers/layers.8.1/attend/Add ┃┃┃┃┃ ━━━ 1,8,300,300,F32 ┃┃┃┃┣ 329 Softmax /transformer/attn_layers/layers.8.1/attend/Softmax ┃┃┃┗┓ ┃┃┃┃┣┻ 322 Reshape /transformer/attn_layers/layers.8.1/Reshape_4 ┃┃┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┃┃┣ 323 PermuteAxes /transformer/attn_layers/layers.8.1/Transpose_2 ┃┃┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┃┣┻ 330 MatMulInference /transformer/attn_layers/layers.8.1/attend/MatMul_1 ┃┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┃┣ 331 PermuteAxes /transformer/attn_layers/layers.8.1/Transpose_3 ┃┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┃┣┻ 332 Reshape /transformer/attn_layers/layers.8.1/Reshape_7 ┃┃┃┃ ━━━ 1,300,512,F32 ┃┃┃┣┻ 333 MatMulInference /transformer/attn_layers/layers.8.1/to_out/MatMul ┃┃┃┃ ━━━ 1,300,144,F32 ┣━━━┓ ┃ ┃┃┃┣┻┻ 334 Iff /transformer/attn_layers/layers.8.1/Where ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┗┓ ┃┃┣┻ 335 Add /transformer/attn_layers/layers.8.2/Add ┃┃┃ ━━━ 1,300,144,? ┃┃┣┓ ┃┃┃┣ 336 Reduce /transformer/attn_layers/layers.9.0.0/ReduceMean ┃┃┃┃ ━━━ 1,300,1,? ┃┃┣━┓┃ ┃┃┃┣┻ 337 Sub /transformer/attn_layers/layers.9.0.0/Sub ┃┃┃┣┓ ┃┃┃┃┣┻ 338 Pow /transformer/attn_layers/layers.9.0.0/Pow ┃┃┃┃┣ 339 Reduce /transformer/attn_layers/layers.9.0.0/ReduceMean_1 ┃┃┃┃┃ ━━━ 1,300,1,? ┃┃┃┃┣┻ 340 Add /transformer/attn_layers/layers.9.0.0/Add ┃┃┃┃┣ 341 Sqrt /transformer/attn_layers/layers.9.0.0/Sqrt ┃┃┃┣┻ 342 Div /transformer/attn_layers/layers.9.0.0/Div ┃┃┃┣┻ 343 Mul /transformer/attn_layers/layers.9.0.0/Mul ┃┃┃┣┻ 344 Add /transformer/attn_layers/layers.9.0.0/Add_1 ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 345 MatMulInference /transformer/attn_layers/layers.9.1/ff/ff.0/ff.0.0/MatMul ┃┃┃┃ ━━━ 1,300,576,F32 ┃┃┃┣┻ 346 Add /transformer/attn_layers/layers.9.1/ff/ff.0/ff.0.0/Add ┃┃┃┃ ━━━ 1,300,576,F32 ┃┃┃┣┓ ┃┃┃┃┣┻ 347 Div /transformer/attn_layers/layers.9.1/ff/ff.0/ff.0.1/Div ┃┃┃┃┣ 348 Erf /transformer/attn_layers/layers.9.1/ff/ff.0/ff.0.1/Erf ┃┃┃┃┣┻ 349 Add /transformer/attn_layers/layers.9.1/ff/ff.0/ff.0.1/Add ┃┃┃┣┻ 350 Mul /transformer/attn_layers/layers.9.1/ff/ff.0/ff.0.1/Mul ┃┃┃┣┻ 351 Mul /transformer/attn_layers/layers.9.1/ff/ff.0/ff.0.1/Mul_1 ┃┃┃┣┻ 352 MatMulInference /transformer/attn_layers/layers.9.1/ff/ff.3/MatMul ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 353 Add /transformer/attn_layers/layers.9.1/ff/ff.3/Add ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┗┓ ┃┃┣┻ 354 Add /transformer/attn_layers/layers.9.2/Add ┃┃┃ ━━━ 1,300,144,? ┃┃┣┓ ┃┃┃┣ 355 Reduce /transformer/attn_layers/layers.10.0.0/ReduceMean ┃┃┃┃ ━━━ 1,300,1,? ┃┃┣━┓┃ ┃┃┃┣┻ 356 Sub /transformer/attn_layers/layers.10.0.0/Sub ┃┃┃┣┓ ┃┃┃┃┣┻ 357 Pow /transformer/attn_layers/layers.10.0.0/Pow ┃┃┃┃┣ 358 Reduce /transformer/attn_layers/layers.10.0.0/ReduceMean_1 ┃┃┃┃┃ ━━━ 1,300,1,? ┃┃┃┃┣┻ 359 Add /transformer/attn_layers/layers.10.0.0/Add ┃┃┃┃┣ 360 Sqrt /transformer/attn_layers/layers.10.0.0/Sqrt ┃┃┃┣┻ 361 Div /transformer/attn_layers/layers.10.0.0/Div ┃┃┃┣┻ 362 Mul /transformer/attn_layers/layers.10.0.0/Mul ┃┃┃┣┻ 363 Add /transformer/attn_layers/layers.10.0.0/Add_1 ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 364 MatMulInference 915 ┃┃┃┃ ━━━ 1,300,1536,F32 ┃┃┃┣╋┳ 365 Split13 916 ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,F32 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 366 Reshape /transformer/attn_layers/layers.10.1/Reshape ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 367 PermuteAxes /transformer/attn_layers/layers.10.1/Transpose ┃┃┃┃┃┃ ━━━ 1,8,300,64,? ┃┃┃┃┃┣┻ 372 Mul /transformer/attn_layers/layers.10.1/attend/Mul_1 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 368 Reshape /transformer/attn_layers/layers.10.1/Reshape_2 ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 371 PermuteAxes /transformer/attn_layers/layers.10.1/attend/Transpose ┃┃┃┃┃┃ ━━━ 1,8,64,300,? ┃┃┃┃┃┣┻ 373 Mul /transformer/attn_layers/layers.10.1/attend/Mul_2 ┃┃┃┃┣┻ 374 MatMulInference /transformer/attn_layers/layers.10.1/attend/MatMul ┃┃┃┃┃ ━━━ 1,8,300,300,? ┃┣━━━┓ ┃┃┃┃┣┻ 375 Add /transformer/attn_layers/layers.10.1/attend/Add ┃┃┃┃┃ ━━━ 1,8,300,300,F32 ┃┃┃┃┣ 376 Softmax /transformer/attn_layers/layers.10.1/attend/Softmax ┃┃┃┗┓ ┃┃┃┃┣┻ 369 Reshape /transformer/attn_layers/layers.10.1/Reshape_4 ┃┃┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┃┃┣ 370 PermuteAxes /transformer/attn_layers/layers.10.1/Transpose_2 ┃┃┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┃┣┻ 377 MatMulInference /transformer/attn_layers/layers.10.1/attend/MatMul_1 ┃┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┃┣ 378 PermuteAxes /transformer/attn_layers/layers.10.1/Transpose_3 ┃┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┃┣┻ 379 Reshape /transformer/attn_layers/layers.10.1/Reshape_7 ┃┃┃┃ ━━━ 1,300,512,F32 ┃┃┃┣┻ 380 MatMulInference /transformer/attn_layers/layers.10.1/to_out/MatMul ┃┃┃┃ ━━━ 1,300,144,F32 ┣━━━┓ ┃ ┃┃┃┣┻┻ 381 Iff /transformer/attn_layers/layers.10.1/Where ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┗┓ ┃┃┣┻ 382 Add /transformer/attn_layers/layers.10.2/Add ┃┃┃ ━━━ 1,300,144,? ┃┃┣┓ ┃┃┃┣ 383 Reduce /transformer/attn_layers/layers.11.0.0/ReduceMean ┃┃┃┃ ━━━ 1,300,1,? ┃┃┣━┓┃ ┃┃┃┣┻ 384 Sub /transformer/attn_layers/layers.11.0.0/Sub ┃┃┃┣┓ ┃┃┃┃┣┻ 385 Pow /transformer/attn_layers/layers.11.0.0/Pow ┃┃┃┃┣ 386 Reduce /transformer/attn_layers/layers.11.0.0/ReduceMean_1 ┃┃┃┃┃ ━━━ 1,300,1,? ┃┃┃┃┣┻ 387 Add /transformer/attn_layers/layers.11.0.0/Add ┃┃┃┃┣ 388 Sqrt /transformer/attn_layers/layers.11.0.0/Sqrt ┃┃┃┣┻ 389 Div /transformer/attn_layers/layers.11.0.0/Div ┃┃┃┣┻ 390 Mul /transformer/attn_layers/layers.11.0.0/Mul ┃┃┃┣┻ 391 Add /transformer/attn_layers/layers.11.0.0/Add_1 ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 392 MatMulInference /transformer/attn_layers/layers.11.1/ff/ff.0/ff.0.0/MatMul ┃┃┃┃ ━━━ 1,300,576,F32 ┃┃┃┣┻ 393 Add /transformer/attn_layers/layers.11.1/ff/ff.0/ff.0.0/Add ┃┃┃┃ ━━━ 1,300,576,F32 ┃┃┃┣┓ ┃┃┃┃┣┻ 394 Div /transformer/attn_layers/layers.11.1/ff/ff.0/ff.0.1/Div ┃┃┃┃┣ 395 Erf /transformer/attn_layers/layers.11.1/ff/ff.0/ff.0.1/Erf ┃┃┃┃┣┻ 396 Add /transformer/attn_layers/layers.11.1/ff/ff.0/ff.0.1/Add ┃┃┃┣┻ 397 Mul /transformer/attn_layers/layers.11.1/ff/ff.0/ff.0.1/Mul ┃┃┃┣┻ 398 Mul /transformer/attn_layers/layers.11.1/ff/ff.0/ff.0.1/Mul_1 ┃┃┃┣┻ 399 MatMulInference /transformer/attn_layers/layers.11.1/ff/ff.3/MatMul ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 400 Add /transformer/attn_layers/layers.11.1/ff/ff.3/Add ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┗┓ ┃┃┣┻ 401 Add /transformer/attn_layers/layers.11.2/Add ┃┃┃ ━━━ 1,300,144,? ┃┃┣┓ ┃┃┃┣ 402 Reduce /transformer/attn_layers/layers.12.0.0/ReduceMean ┃┃┃┃ ━━━ 1,300,1,? ┃┃┣━┓┃ ┃┃┃┣┻ 403 Sub /transformer/attn_layers/layers.12.0.0/Sub ┃┃┃┣┓ ┃┃┃┃┣┻ 404 Pow /transformer/attn_layers/layers.12.0.0/Pow ┃┃┃┃┣ 405 Reduce /transformer/attn_layers/layers.12.0.0/ReduceMean_1 ┃┃┃┃┃ ━━━ 1,300,1,? ┃┃┃┃┣┻ 406 Add /transformer/attn_layers/layers.12.0.0/Add ┃┃┃┃┣ 407 Sqrt /transformer/attn_layers/layers.12.0.0/Sqrt ┃┃┃┣┻ 408 Div /transformer/attn_layers/layers.12.0.0/Div ┃┃┃┣┻ 409 Mul /transformer/attn_layers/layers.12.0.0/Mul ┃┃┃┣┻ 410 Add /transformer/attn_layers/layers.12.0.0/Add_1 ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 411 MatMulInference 922 ┃┃┃┃ ━━━ 1,300,1536,F32 ┃┃┃┣╋┳ 412 Split13 923 ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,F32 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 413 Reshape /transformer/attn_layers/layers.12.1/Reshape ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 414 PermuteAxes /transformer/attn_layers/layers.12.1/Transpose ┃┃┃┃┃┃ ━━━ 1,8,300,64,? ┃┃┃┃┃┣┻ 419 Mul /transformer/attn_layers/layers.12.1/attend/Mul_1 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 415 Reshape /transformer/attn_layers/layers.12.1/Reshape_2 ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 418 PermuteAxes /transformer/attn_layers/layers.12.1/attend/Transpose ┃┃┃┃┃┃ ━━━ 1,8,64,300,? ┃┃┃┃┃┣┻ 420 Mul /transformer/attn_layers/layers.12.1/attend/Mul_2 ┃┃┃┃┣┻ 421 MatMulInference /transformer/attn_layers/layers.12.1/attend/MatMul ┃┃┃┃┃ ━━━ 1,8,300,300,? ┃┣━━━┓ ┃┃┃┃┣┻ 422 Add /transformer/attn_layers/layers.12.1/attend/Add ┃┃┃┃┃ ━━━ 1,8,300,300,F32 ┃┃┃┃┣ 423 Softmax /transformer/attn_layers/layers.12.1/attend/Softmax ┃┃┃┗┓ ┃┃┃┃┣┻ 416 Reshape /transformer/attn_layers/layers.12.1/Reshape_4 ┃┃┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┃┃┣ 417 PermuteAxes /transformer/attn_layers/layers.12.1/Transpose_2 ┃┃┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┃┣┻ 424 MatMulInference /transformer/attn_layers/layers.12.1/attend/MatMul_1 ┃┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┃┣ 425 PermuteAxes /transformer/attn_layers/layers.12.1/Transpose_3 ┃┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┃┣┻ 426 Reshape /transformer/attn_layers/layers.12.1/Reshape_7 ┃┃┃┃ ━━━ 1,300,512,F32 ┃┃┃┣┻ 427 MatMulInference /transformer/attn_layers/layers.12.1/to_out/MatMul ┃┃┃┃ ━━━ 1,300,144,F32 ┣━━━┓ ┃ ┃┃┃┣┻┻ 428 Iff /transformer/attn_layers/layers.12.1/Where ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┗┓ ┃┃┣┻ 429 Add /transformer/attn_layers/layers.12.2/Add ┃┃┃ ━━━ 1,300,144,? ┃┃┣┓ ┃┃┃┣ 430 Reduce /transformer/attn_layers/layers.13.0.0/ReduceMean ┃┃┃┃ ━━━ 1,300,1,? ┃┃┣━┓┃ ┃┃┃┣┻ 431 Sub /transformer/attn_layers/layers.13.0.0/Sub ┃┃┃┣┓ ┃┃┃┃┣┻ 432 Pow /transformer/attn_layers/layers.13.0.0/Pow ┃┃┃┃┣ 433 Reduce /transformer/attn_layers/layers.13.0.0/ReduceMean_1 ┃┃┃┃┃ ━━━ 1,300,1,? ┃┃┃┃┣┻ 434 Add /transformer/attn_layers/layers.13.0.0/Add ┃┃┃┃┣ 435 Sqrt /transformer/attn_layers/layers.13.0.0/Sqrt ┃┃┃┣┻ 436 Div /transformer/attn_layers/layers.13.0.0/Div ┃┃┃┣┻ 437 Mul /transformer/attn_layers/layers.13.0.0/Mul ┃┃┃┣┻ 438 Add /transformer/attn_layers/layers.13.0.0/Add_1 ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 439 MatMulInference /transformer/attn_layers/layers.13.1/ff/ff.0/ff.0.0/MatMul ┃┃┃┃ ━━━ 1,300,576,F32 ┃┃┃┣┻ 440 Add /transformer/attn_layers/layers.13.1/ff/ff.0/ff.0.0/Add ┃┃┃┃ ━━━ 1,300,576,F32 ┃┃┃┣┓ ┃┃┃┃┣┻ 441 Div /transformer/attn_layers/layers.13.1/ff/ff.0/ff.0.1/Div ┃┃┃┃┣ 442 Erf /transformer/attn_layers/layers.13.1/ff/ff.0/ff.0.1/Erf ┃┃┃┃┣┻ 443 Add /transformer/attn_layers/layers.13.1/ff/ff.0/ff.0.1/Add ┃┃┃┣┻ 444 Mul /transformer/attn_layers/layers.13.1/ff/ff.0/ff.0.1/Mul ┃┃┃┣┻ 445 Mul /transformer/attn_layers/layers.13.1/ff/ff.0/ff.0.1/Mul_1 ┃┃┃┣┻ 446 MatMulInference /transformer/attn_layers/layers.13.1/ff/ff.3/MatMul ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 447 Add /transformer/attn_layers/layers.13.1/ff/ff.3/Add ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┗┓ ┃┃┣┻ 448 Add /transformer/attn_layers/layers.13.2/Add ┃┃┃ ━━━ 1,300,144,? ┃┃┣┓ ┃┃┃┣ 449 Reduce /transformer/attn_layers/layers.14.0.0/ReduceMean ┃┃┃┃ ━━━ 1,300,1,? ┃┃┣━┓┃ ┃┃┃┣┻ 450 Sub /transformer/attn_layers/layers.14.0.0/Sub ┃┃┃┣┓ ┃┃┃┃┣┻ 451 Pow /transformer/attn_layers/layers.14.0.0/Pow ┃┃┃┃┣ 452 Reduce /transformer/attn_layers/layers.14.0.0/ReduceMean_1 ┃┃┃┃┃ ━━━ 1,300,1,? ┃┃┃┃┣┻ 453 Add /transformer/attn_layers/layers.14.0.0/Add ┃┃┃┃┣ 454 Sqrt /transformer/attn_layers/layers.14.0.0/Sqrt ┃┃┃┣┻ 455 Div /transformer/attn_layers/layers.14.0.0/Div ┃┃┃┣┻ 456 Mul /transformer/attn_layers/layers.14.0.0/Mul ┃┃┃┣┻ 457 Add /transformer/attn_layers/layers.14.0.0/Add_1 ┃┃┃┃ ━━━ 1,300,144,F32 ┃┃┃┣┻ 458 MatMulInference 929 ┃┃┃┃ ━━━ 1,300,1536,F32 ┃┃┃┣╋┳ 459 Split13 930 ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,? ┃┃┃┃┃┃ ━━━ 1,300,512,F32 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 460 Reshape /transformer/attn_layers/layers.14.1/Reshape ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 461 PermuteAxes /transformer/attn_layers/layers.14.1/Transpose ┃┃┃┃┃┃ ━━━ 1,8,300,64,? ┃┃┃┃┃┣┻ 466 Mul /transformer/attn_layers/layers.14.1/attend/Mul_1 ┃┃┃┗━┓ ┃┃┃┃┃┣┻ 462 Reshape /transformer/attn_layers/layers.14.1/Reshape_2 ┃┃┃┃┃┃ ━━━ 1,300,8,64,? ┃┃┃┃┃┣ 465 PermuteAxes /transformer/attn_layers/layers.14.1/attend/Transpose ┃┃┃┃┃┃ ━━━ 1,8,64,300,? ┃┃┃┃┃┣┻ 467 Mul /transformer/attn_layers/layers.14.1/attend/Mul_2 ┃┃┃┃┣┻ 468 MatMulInference /transformer/attn_layers/layers.14.1/attend/MatMul ┃┃┃┃┃ ━━━ 1,8,300,300,? ┃┗━━┓ ┃┃┃┣┻ 469 Add /transformer/attn_layers/layers.14.1/attend/Add ┃┃┃┃ ━━━ 1,8,300,300,F32 ┃┃┃┣ 470 Softmax /transformer/attn_layers/layers.14.1/attend/Softmax ┃┃┗┓ ┃┃┃┣┻ 463 Reshape /transformer/attn_layers/layers.14.1/Reshape_4 ┃┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┃┣ 464 PermuteAxes /transformer/attn_layers/layers.14.1/Transpose_2 ┃┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┣┻ 471 MatMulInference /transformer/attn_layers/layers.14.1/attend/MatMul_1 ┃┃┃ ━━━ 1,8,300,64,F32 ┃┃┣ 472 PermuteAxes /transformer/attn_layers/layers.14.1/Transpose_3 ┃┃┃ ━━━ 1,300,8,64,F32 ┃┃┣┻ 473 Reshape /transformer/attn_layers/layers.14.1/Reshape_7 ┃┃┃ ━━━ 1,300,512,F32 ┃┃┣┻ 474 MatMulInference /transformer/attn_layers/layers.14.1/to_out/MatMul ┃┃┃ ━━━ 1,300,144,F32 ┗┓ ┃ ┃┣┻┻ 475 Iff /transformer/attn_layers/layers.14.1/Where ┃┃ ━━━ 1,300,144,F32 ┗┓ ┣┻ 476 Add /transformer/attn_layers/layers.14.2/Add ┃ ━━━ 1,300,144,? ┣┓ ┃┣ 477 Reduce /transformer/attn_layers/layers.15.0.0/ReduceMean ┃┃ ━━━ 1,300,1,? ┣━┓┃ ┃┣┻ 478 Sub /transformer/attn_layers/layers.15.0.0/Sub ┃┣┓ ┃┃┣┻ 479 Pow /transformer/attn_layers/layers.15.0.0/Pow ┃┃┣ 480 Reduce /transformer/attn_layers/layers.15.0.0/ReduceMean_1 ┃┃┃ ━━━ 1,300,1,? ┃┃┣┻ 481 Add /transformer/attn_layers/layers.15.0.0/Add ┃┃┣ 482 Sqrt /transformer/attn_layers/layers.15.0.0/Sqrt ┃┣┻ 483 Div /transformer/attn_layers/layers.15.0.0/Div ┃┣┻ 484 Mul /transformer/attn_layers/layers.15.0.0/Mul ┃┣┻ 485 Add /transformer/attn_layers/layers.15.0.0/Add_1 ┃┃ ━━━ 1,300,144,F32 ┃┣┻ 486 MatMulInference /transformer/attn_layers/layers.15.1/ff/ff.0/ff.0.0/MatMul ┃┃ ━━━ 1,300,576,F32 ┃┣┻ 487 Add /transformer/attn_layers/layers.15.1/ff/ff.0/ff.0.0/Add ┃┃ ━━━ 1,300,576,F32 ┃┣┓ ┃┃┣┻ 488 Div /transformer/attn_layers/layers.15.1/ff/ff.0/ff.0.1/Div ┃┃┣ 489 Erf /transformer/attn_layers/layers.15.1/ff/ff.0/ff.0.1/Erf ┃┃┣┻ 490 Add /transformer/attn_layers/layers.15.1/ff/ff.0/ff.0.1/Add ┃┣┻ 491 Mul /transformer/attn_layers/layers.15.1/ff/ff.0/ff.0.1/Mul ┃┣┻ 492 Mul /transformer/attn_layers/layers.15.1/ff/ff.0/ff.0.1/Mul_1 ┃┣┻ 493 MatMulInference /transformer/attn_layers/layers.15.1/ff/ff.3/MatMul ┃┃ ━━━ 1,300,144,F32 ┃┣┻ 494 Add /transformer/attn_layers/layers.15.1/ff/ff.3/Add ┃┃ ━━━ 1,300,144,F32 ┗┓ ┣┻ 495 Add /transformer/attn_layers/layers.15.2/Add ┃ ━━━ 1,300,144,? ┣┓ ┃┣ 496 Reduce /transformer/attn_layers/final_norm/ReduceMean ┃┃ ━━━ 1,300,1,? ┣┻ 497 Sub /transformer/attn_layers/final_norm/Sub ┣┓ ┃┣┻ 498 Pow /transformer/attn_layers/final_norm/Pow ┃┣ 499 Reduce /transformer/attn_layers/final_norm/ReduceMean_1 ┃┃ ━━━ 1,300,1,? ┃┣┻ 500 Add /transformer/attn_layers/final_norm/Add ┃┣ 501 Sqrt /transformer/attn_layers/final_norm/Sqrt ┣┻ 502 Div /transformer/attn_layers/final_norm/Div ┣┻ 503 Mul /transformer/attn_layers/final_norm/Mul ┣┻ 504 Add /transformer/attn_layers/final_norm/Add_1 ┃ ━━━ 1,300,144,F32 ┣┻ 505 MatMulInference /transformer/to_logits/MatMul ┃ ━━━ 1,300,162,F32 ┣┻ 506 Add /transformer/to_logits/Add ┃ ━━━ 1,300,162,F32 ┣┻ 507 MatMulInference /fc_out/MatMul ┃ ━━━ 1,300,18,F32 ┣┻ 508 Add /fc_out/Add ┃ ━━━ 1,300,18,F32 ┣ 509 Softmax /Softmax ┣┓ ┃┣ 510 Reduce /ArgMax ┃┃ ━━━ 1,300,I64 ┃┣┻ 512 Squeeze13 /Squeeze_1 ┃┃ ━━━ 300,I64 ┃┣ 513 onnx.Cast /Cast_3 ┃ ━━━ 300,U8 ┣ 511 Reduce /ReduceMax ┃ ━━━ 1,300,F32 ┣┻ 514 Squeeze13 /Squeeze_2 ━━━ 300,F32 [2023-08-24T18:41:12.118238100Z ERROR tract] Error at stage analyse Caused by: 0: Failed analyse for node #111 "/transformer/token_emb/emb/Gather" Gather 1: Failed analyse for node #111 "/transformer/token_emb/emb/Gather" Gather 2: Infering facts 3: Applying rule inputs[0].rank + -1*{1} + inputs[1].rank == outputs[0].rank 4: Impossible to unify 7 with 3.

I can send the model privately if required.

Best Musharraf

kali commented 1 year ago

I think your input spec is broken. Do you mean -i 1,300,i64 -i 1,1,i64 ? You may also need --onnx-ignore-output-shapes because ONNX semantics around symbols are more lax than tract's.

mush42 commented 1 year ago

Thanks @kali that works perfectly. Yes you're correct, I didn't specify input shape correctly.

But now I faced the following issue (on Windows) when running the command.

tract --onnx-ignore-output-shapes --nnef-tract-onnx --nnef-tract-core  -i 300,i64 -i 1,i64 model.onnx dump --nnef model.tgz

Command output:

[2023-08-25T14:27:32.744023700Z ERROR tract] Writting model to tgz

    Caused by:
        0: Appending tensor "/transformer/attn_layers/layers.10.0.0/Mul.fix-rank-1-2.dat"
        1: paths in archives must be relative when setting path for
mush42 commented 1 year ago

@kali

Tried to use the converted nnef model with the following:

let model = nnef()
    .with_tract_core()
    .with_tract_resource()
    .model_for_path("./data/model.tar")?
    .into_optimized()?
    // .into_compact()?
    .into_runnable()?;
Ok(model

gives the following error. am I doing something wrong?!

In ModelBuilder::translate: Wiring root graph body: Plugging in assignement for \"/Constant_2_output_0.0\": Resolving invocation Identifier(\"variable\"): Interrogating registry Identifier(\"tract_nnef\"): Deserializing op `variable': No data for tensor Identifier(\"/Constant_2_output_0.0\")")'

Best

kali commented 1 year ago

It smells strongly of path delimiter problems. Can you check if the model generated under windows can be opened on linux (or mac) ? and vice-versa ?

mush42 commented 1 year ago

@kali thanks! I will.

To create the .tar file I exported the model using --nnef-dir and compressed it using the builtin tar utility that comes with Windows 11.

I'll dump the model in a Linux box and try run it on Windows.

Best

kali commented 1 year ago

@mush42 you can also give a shot to dump --nnef-tar or dump --nnef (actually dumping to .nnef.tgz).

mush42 commented 1 year ago

@kali exporting to tar or tgz does not work on Windows. Also a path sep issue, I think.

mush42 commented 1 year ago

@kali Would you mind looking into this.

I tried running the export command on a linux box. I faced the same problem.

$ tract --onnx-ignore-output-shapes --nnef-tract-onnx --nnef-tract-core  --nnef-extended-identifier -i 300,i64 -i 1,i64 model.onnx dump --nnef model.tgz

Command output:

[2023-08-28T14:25:59.540478400Z ERROR tract] Writting model to tgz

    Caused by:
        0: Appending tensor "/transformer/attn_layers/layers.13.1/ff/ff.3/Add.fix-rank-0-2.dat"
        1: paths in archives must be relative when setting path for
kali commented 1 year ago

Ok this is a problem that appear after pytorch-to-onnx started using leading / in the node names. The main branch has been fixed, but no release have been made since. I am going to cut one today or early this week, as soon as I get all my ducks in a row.

kali commented 1 year ago

0.20.18 is out. Please give it a shot, let's see where we are after the leading slash fix.

mush42 commented 1 year ago

@kali the model is dumped successfully after upgrading to v0.20.18.

But now I face the following error when running the exported model:

     Running `target\debug\app.exe`
thread 'main' panicked at 'Unary("-", Identifier(Identifier("inf")))', C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\deser.rs:534:18
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library\std\src\panicking.rs:593
   1: core::panicking::panic_fmt
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library\core\src\panicking.rs:67
   2: enum2$<tract_nnef::ast::RValue>::resolve
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\deser.rs:534
   3: tract_nnef::deser::impl$5::resolve::closure$2
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\deser.rs:486
   4: core::iter::adapters::map::map_try_fold::closure$0<tuple$<ref$<enum2$<tract_nnef::ast::RValue> >,ref$<enum2$<core::option::Option<enum2$<tract_data::datum::DatumType> > > > >,enum2$<core::result::Result<enum2$<tract_nnef::deser::Value>,anyhow::Error> >,tu
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\map.rs:91
   5: core::iter::traits::iterator::Iterator::try_fold<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::DatumType> > > > > >,t
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\traits\iterator.rs:2303
   6: core::iter::adapters::map::impl$2::try_fold<enum2$<core::result::Result<enum2$<tract_nnef::deser::Value>,anyhow::Error> >,core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enu
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\map.rs:117
   7: core::iter::adapters::impl$0::try_fold<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::D
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\mod.rs:195
   8: core::iter::adapters::impl$0::next<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::Datum
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\mod.rs:178
   9: alloc::vec::spec_from_iter_nested::impl$0::from_iter<enum2$<tract_nnef::deser::Value>,core::iter::adapters::GenericShunt<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::so
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\alloc\src\vec\spec_from_iter_nested.rs:26
  10: alloc::vec::spec_from_iter::impl$0::from_iter<enum2$<tract_nnef::deser::Value>,core::iter::adapters::GenericShunt<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\alloc\src\vec\spec_from_iter.rs:33
  11: alloc::vec::impl$14::from_iter<enum2$<tract_nnef::deser::Value>,core::iter::adapters::GenericShunt<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\alloc\src\vec\mod.rs:2696
  12: core::iter::traits::iterator::Iterator::collect
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\traits\iterator.rs:1895
  13: core::result::impl$25::from_iter::closure$0<enum2$<tract_nnef::deser::Value>,anyhow::Error,alloc::vec::Vec<enum2$<tract_nnef::deser::Value>,alloc::alloc::Global>,core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\result.rs:1932
  14: core::iter::adapters::try_process<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::DatumT
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\mod.rs:164
  15: core::result::impl$25::from_iter<enum2$<tract_nnef::deser::Value>,anyhow::Error,alloc::vec::Vec<enum2$<tract_nnef::deser::Value>,alloc::alloc::Global>,core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef:
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\result.rs:1932
  16: core::iter::traits::iterator::Iterator::collect<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data:
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\traits\iterator.rs:1895
  17: enum2$<tract_nnef::ast::RValue>::resolve
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\deser.rs:483
  18: tract_nnef::deser::impl$5::resolve::closure$2
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\deser.rs:486
  19: core::iter::adapters::map::map_try_fold::closure$0<tuple$<ref$<enum2$<tract_nnef::ast::RValue> >,ref$<enum2$<core::option::Option<enum2$<tract_data::datum::DatumType> > > > >,enum2$<core::result::Result<enum2$<tract_nnef::deser::Value>,anyhow::Error> >,tu
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\map.rs:91
  20: core::iter::traits::iterator::Iterator::try_fold<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::DatumType> > > > > >,t
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\traits\iterator.rs:2303
  21: core::iter::adapters::map::impl$2::try_fold<enum2$<core::result::Result<enum2$<tract_nnef::deser::Value>,anyhow::Error> >,core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enu
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\map.rs:117
  22: core::iter::adapters::impl$0::try_fold<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::D
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\mod.rs:195
  23: core::iter::adapters::impl$0::next<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::Datum
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\mod.rs:178
  24: alloc::vec::spec_from_iter_nested::impl$0::from_iter<enum2$<tract_nnef::deser::Value>,core::iter::adapters::GenericShunt<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::so
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\alloc\src\vec\spec_from_iter_nested.rs:26
  25: alloc::vec::spec_from_iter::impl$0::from_iter<enum2$<tract_nnef::deser::Value>,core::iter::adapters::GenericShunt<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\alloc\src\vec\spec_from_iter.rs:33
  26: alloc::vec::impl$14::from_iter<enum2$<tract_nnef::deser::Value>,core::iter::adapters::GenericShunt<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\alloc\src\vec\mod.rs:2696
  27: core::iter::traits::iterator::Iterator::collect
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\traits\iterator.rs:1895
  28: core::result::impl$25::from_iter::closure$0<enum2$<tract_nnef::deser::Value>,anyhow::Error,alloc::vec::Vec<enum2$<tract_nnef::deser::Value>,alloc::alloc::Global>,core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\result.rs:1932
  29: core::iter::adapters::try_process<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::DatumT
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\mod.rs:164
  30: core::result::impl$25::from_iter<enum2$<tract_nnef::deser::Value>,anyhow::Error,alloc::vec::Vec<enum2$<tract_nnef::deser::Value>,alloc::alloc::Global>,core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef:
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\result.rs:1932
  31: core::iter::traits::iterator::Iterator::collect<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data:
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\traits\iterator.rs:1895
  32: enum2$<tract_nnef::ast::RValue>::resolve
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\deser.rs:483
  33: tract_nnef::deser::impl$5::resolve::closure$2
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\deser.rs:486
  34: core::iter::adapters::map::map_try_fold::closure$0<tuple$<ref$<enum2$<tract_nnef::ast::RValue> >,ref$<enum2$<core::option::Option<enum2$<tract_data::datum::DatumType> > > > >,enum2$<core::result::Result<enum2$<tract_nnef::deser::Value>,anyhow::Error> >,tu
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\map.rs:91
  35: core::iter::traits::iterator::Iterator::try_fold<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::DatumType> > > > > >,t
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\traits\iterator.rs:2303
  36: core::iter::adapters::map::impl$2::try_fold<enum2$<core::result::Result<enum2$<tract_nnef::deser::Value>,anyhow::Error> >,core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enu
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\map.rs:117
  37: core::iter::adapters::impl$0::try_fold<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::D
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\mod.rs:195
  38: core::iter::adapters::impl$0::next<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::Datum
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\mod.rs:178
  39: alloc::vec::spec_from_iter_nested::impl$0::from_iter<enum2$<tract_nnef::deser::Value>,core::iter::adapters::GenericShunt<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::so
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\alloc\src\vec\spec_from_iter_nested.rs:26
  40: alloc::vec::spec_from_iter::impl$0::from_iter<enum2$<tract_nnef::deser::Value>,core::iter::adapters::GenericShunt<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\alloc\src\vec\spec_from_iter.rs:33
  41: alloc::vec::impl$14::from_iter<enum2$<tract_nnef::deser::Value>,core::iter::adapters::GenericShunt<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\alloc\src\vec\mod.rs:2696
  42: core::iter::traits::iterator::Iterator::collect
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\traits\iterator.rs:1895
  43: core::result::impl$25::from_iter::closure$0<enum2$<tract_nnef::deser::Value>,anyhow::Error,alloc::vec::Vec<enum2$<tract_nnef::deser::Value>,alloc::alloc::Global>,core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\result.rs:1932
  44: core::iter::adapters::try_process<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::DatumT
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\mod.rs:164
  45: core::result::impl$25::from_iter<enum2$<tract_nnef::deser::Value>,anyhow::Error,alloc::vec::Vec<enum2$<tract_nnef::deser::Value>,alloc::alloc::Global>,core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef:
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\result.rs:1932
  46: core::iter::traits::iterator::Iterator::collect<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data:
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\traits\iterator.rs:1895
  47: enum2$<tract_nnef::ast::RValue>::resolve
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\deser.rs:483
  48: tract_nnef::deser::impl$5::resolve::closure$2
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\deser.rs:486
  49: core::iter::adapters::map::map_try_fold::closure$0<tuple$<ref$<enum2$<tract_nnef::ast::RValue> >,ref$<enum2$<core::option::Option<enum2$<tract_data::datum::DatumType> > > > >,enum2$<core::result::Result<enum2$<tract_nnef::deser::Value>,anyhow::Error> >,tu
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\map.rs:91
  50: core::iter::traits::iterator::Iterator::try_fold<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::DatumType> > > > > >,t
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\traits\iterator.rs:2303
  51: core::iter::adapters::map::impl$2::try_fold<enum2$<core::result::Result<enum2$<tract_nnef::deser::Value>,anyhow::Error> >,core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enu
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\map.rs:117
  52: core::iter::adapters::impl$0::try_fold<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::D
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\mod.rs:195
  53: core::iter::adapters::impl$0::next<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::Datum
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\mod.rs:178
  54: alloc::vec::spec_from_iter_nested::impl$0::from_iter<enum2$<tract_nnef::deser::Value>,core::iter::adapters::GenericShunt<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::so
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\alloc\src\vec\spec_from_iter_nested.rs:26
  55: alloc::vec::spec_from_iter::impl$0::from_iter<enum2$<tract_nnef::deser::Value>,core::iter::adapters::GenericShunt<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\alloc\src\vec\spec_from_iter.rs:33
  56: alloc::vec::impl$14::from_iter<enum2$<tract_nnef::deser::Value>,core::iter::adapters::GenericShunt<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\alloc\src\vec\mod.rs:2696
  57: core::iter::traits::iterator::Iterator::collect
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\traits\iterator.rs:1895
  58: core::result::impl$25::from_iter::closure$0<enum2$<tract_nnef::deser::Value>,anyhow::Error,alloc::vec::Vec<enum2$<tract_nnef::deser::Value>,alloc::alloc::Global>,core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\result.rs:1932
  59: core::iter::adapters::try_process<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data::datum::DatumT
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\adapters\mod.rs:164
  60: core::result::impl$25::from_iter<enum2$<tract_nnef::deser::Value>,anyhow::Error,alloc::vec::Vec<enum2$<tract_nnef::deser::Value>,alloc::alloc::Global>,core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef:
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\result.rs:1932
  61: core::iter::traits::iterator::Iterator::collect<core::iter::adapters::map::Map<core::iter::adapters::zip::Zip<core::slice::iter::Iter<enum2$<tract_nnef::ast::RValue> >,core::iter::sources::repeat::Repeat<ref$<enum2$<core::option::Option<enum2$<tract_data:
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\iter\traits\iterator.rs:1895
  62: enum2$<tract_nnef::ast::RValue>::resolve
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\deser.rs:483
  63: tract_nnef::deser::ModelBuilder::wire_body
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\deser.rs:150
  64: tract_nnef::deser::ModelBuilder::translate
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\deser.rs:81
  65: tract_nnef::deser::ModelBuilder::into_typed_model
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\deser.rs:112
  66: tract_nnef::framework::Nnef::translate
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\framework.rs:78
  67: tract_nnef::framework::impl$2::model_for_proto_model_with_symbols
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\framework.rs:284
  68: tract_core::framework::Framework::model_for_proto_model<tract_nnef::framework::Nnef,tract_nnef::ast::ProtoModel,tract_core::model::graph::Graph<tract_core::model::fact::TypedFact,alloc::boxed::Box<dyn$<tract_core::ops::TypedOp>,alloc::alloc::Global> > >
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-core-0.20.18\src\framework.rs:20
  69: tract_nnef::framework::impl$2::model_for_path<ref$<str$> >
             at C:\Users\user\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tract-nnef-0.20.18\src\framework.rs:221
  70: tapp::main
             at .\src\main.rs:5
  71: core::ops::function::FnOnce::call_once<enum2$<core::result::Result<tuple$<>,anyhow::Error> > (*)(),tuple$<> >
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\ops\function.rs:250
kali commented 1 year ago

All right, progress.

I don't think this one is windows-specific. Can you send me the model so I can have a look ? (first_name.last_name@sonos.com)

mush42 commented 1 year ago

@kali the model is opensource. Find it here:

https://github.com/mush42/libtashkeel/blob/main/libtashkeel_base/data/tract/model.onnx

mush42 commented 1 year ago

Converted using the command

tract --onnx-ignore-output-shapes --nnef-tract-onnx --nnef-tract-core  -i 300,i64 -i 1,i64 model.onnx dump --nnef "model.tgz"
kali commented 1 year ago

Can you give a shot to the branch above ? I fixed parsing for -inf and inf, tell me if there are other issues.

mush42 commented 1 year ago

@kali it works perfectly now. All tests are passing with tract-nnef. Thanks!