Closed Lime-Cakes closed 1 year ago
Based on this error message:
[libprotobuf ERROR google/protobuf/message_lite.cc:447] onnx.ModelProto exceeded maximum protobuf size of 2GB: 3222639541
I think the problem is that your model doesn't fit inside the ONNX protobuf representation and as a result gets truncated.
You could try to keep the weights outside the model to see if it helps:
opts = poptorch.Options()
opts._Popart.set("saveInitializersToFile", "my_weights.onnx")
Thanks, that kinda helps, but compiling still fails
I tried porting a pytorch model to run and train on IPU, but ran into the following problem. Is there a solution? It seemed to be an issue with model size.
I have the loss defined as following: