Failing on large digraph

[IanQS]

Hey there!

I'm a huge fan of the library! I'm running into an issue where by using a ConvLSTM I'm getting a gigantic digraph. I was wondering if you had dealt with this issue before? You'll first need to . To reproduce this:

1) run ! wget -q https://www.cs.toronto.edu/~nitish/unsupervised_video/mnist_test_seq.npy to get the data 2) run git clone git@github.com:IanQS/torchviz_convlstm_issue.git to get the repo 3) check out line 66 and 67 in debug_clstm.py where you'll see the issue.

            bla = make_dot(
                output, params=dict(model.named_parameters()), show_attrs=True, show_saved=True
            )
            bla.render("/tmp/model_y_hat_ob", format="png")

(you might want to put a breakpoint on bla.render("/tmp/model_y_hat_ob", format="png") but basically, you should see a digraph of size 2914

Might you have any solutions in mind?

[albanD]

Hey! Is the problem that the graph is that the library fails to generate such a graph? Or that the final graph is so big it is not really usable anymore?

If it is the second case, then an approach like the one from https://github.com/ahgamut/torchrecorder might help as it will allow you to collapse nodes at a higher level (nn.Module level).

[IanQS]

It's the first :( i waited for a very long time on my custom code that builds off of the code in that repo. It was half an hour in and the image still wasn't generated if I remember correctly.

But that repo you shared looks like it is exactly what I need

[albanD]

Ho. For the first one, if the "render" part is the slow one then I'm not sure there is much we can do about this here as that function is provided by the "dot" library that renders the graph. And I've definitely seen it struggle before with very large graph (and the final image is not very readable anyways).