search

ezyang / pytorch-unattached

Tensors and Dynamic neural networks in Python with strong GPU acceleration
http://pytorch.org
Other
20 stars 8 forks source link

Use state_dict so that parameters get named #85

Open ezyang opened 7 years ago

ezyang commented 7 years ago

Right now, parameters are completely unnamed (just a big pile of numbers) because they are retrieved using parameters(). However, we actually have names for them from the state_dict interface from modules. It would be great to use these names to make the traces more interpretable.

We'll need to adjust name allocation to make use of name hints, and it's probably simplest if we unconditionally include the unique number even with the name hint.

apaszke commented 7 years ago

While it might be good to use state_dict, it's important to remember that using modules is not the only way people write models in PyTorch, and we'll need a way to export functional models as well

ezyang commented 7 years ago

This was prototyped, and it was decided that this just made the networks more unreadable (because the long names got repeatedly renamed). But a useful refinement is to name the parameters on input, but then use numbers internally.

fmassa commented 7 years ago

Could you expand on the "unreadable" part, that sounded counter-intuitive to me... Is it harder to debug because of longer names? If that's the case, couldn't we have a mapping structure that convert the numbers to their names just before exporting to ONNX?