Open cowwoc opened 3 years ago
Looks good, are you planning on making a PR for this?
If so, some initial comments:
It looks like the session
is leftover from a previous version?
I think a toString(Tensor t)
(or TType, but Tensor is better) w/ resolution done based on the runtime type would be useful, too, since the concrete type isn't always known. Perhaps make this the only method.
It would also be good to have this as a default method in Operand
too, that only does something in an eager session (like asTensor()
). Probably return a N/A message rather than throwing, since it's for debugging.
@rnett Good suggestions. I'll try to formulate a PR.
Question though, since this method is meant strictly for debugging, couldn't we implement it for non-eager sessions as well? We could spin up a graph runner, evaluate the operand, and return the String representation.
You could, but if that tensor depends on anything non-constant (i.e. placeholders or variables), you won't be able to get it, since the session has no way of knowing about those inputs. And I would think most things you want to debug would have dependencies like that. Plus I'm not sure sessions support adding things to the graph after the session is created, and you'd have to re-run the whole graph each time you called asString.
Once we finish functions and eager gradients, most debugging should be done in eager mode anyways. You'd almost always use functions instead of graphs, and there would be a global "execute functions in eager mode" like in Python. It's not necessarily impossible to have asString in graph mode, but it's not easy and won't ever fit very well, and since we have this coming I don't think it's worth it. Feel free to come up with an implementation and make a PR, but maybe PR just the eager version first.
Per our discussion on Gitter, here is a possible implementation for converting Tensors to a String representation. It is still missing some important features, like collapsing long arrays using ellipses, but this can serve as a stepping stone. The functionality is meant to ease troubleshooting/debugging so performance should not be an issue.