How to see the parameters of a model

[davidw0311]

If I have loaded in a model using DynamicGraph.read(), how can I print the weights of the model for debugging purposes?

For example, I loaded a textModel and when I print

print(textModel.parameters)

I get NNC.Model.Parameters

But is there a way to see the actual tensor values for each layer in the model? or something similar to PyTorch where I can get a dictionary of key: values for each parameter in the model?

Thank you!

[liuliu]

One of these things we are far-away from PyTorch on debuggability.

You can get parameters in roundabout way: https://liuliu.github.io/s4nnc/documentation/nnc/model/parameters-swift.class/copied(_:)

Basically, you need to use https://liuliu.github.io/s4nnc/documentation/nnc/model/parameters(for:) to get a single parameter, and then copy it out into a tensor to print, something like:

print(textModel.parameters(for: .index(0)).copied(Float32.self))

Note that to print out any parameters associated with convolution / dense, you'd better to keep track of that yourself, that is why I always put an additional block with it when building models. That block can be used to load parameters, but also in cases can be used to print parameters etc.

In the future we might write a more informed debugDescription so these information can be easily print like you just said.

[liuliu]

Added a few more methods for this. In particular, if you use tensorboard, you can do this:

import TensorBoard

let summaryWriter = xxxx
summaryWriter.addParameters("something", model.parameters, step: 0)

Or you can only write some parameters to there:

summaryWriter.addParameters("something", model.parameters.filter { $0.contains("_proj") }, step: 0)

See example in: https://github.com/liuliu/s4nnc/blob/main/examples/minrf/main.swift