Open l8l opened 1 year ago
The d3viz functionality is not being tested or really used by the developers, so I am not surprised it is broken.
Hopefully someone will be able to take a look to give feedback
Thanks for the post! I hope so too. It seems that one only needs to make sure some template html file is copied to the d3viz directory upon installation to fix the issue.
You're saying that developers don't really use this feature? But isn't the graph visualization quite useful? I think it is a really cool feature, distinguishing it from other computational graph frameworks that is worth maintaining!
Yeah, I think it could be useful, I just haven't seen much use (or any at all) since we forked from Theano/Aesara.
Other libraries usually rely on a condensed type of graph, for instance just showing relationship between named nodes, such as this utility in PyMC: https://www.pymc.io/projects/docs/en/stable/api/generated/pymc.model_to_graphviz.html
Some example outputs in: https://www.pymc.io/projects/examples/en/latest/case_studies/multilevel_modeling.html
For pure PyTensor graphs I usually just use dprint
, I find a graphical form of all operations too much to digest in practice.
Anyway, in the codebase, it seems we have just these tests: https://github.com/pymc-devs/pytensor/blob/main/tests/d3viz/test_d3viz.py
However it doesn't seem the dependency is being installed according to the CI logs: https://github.com/pymc-devs/pytensor/actions/runs/5212573519/jobs/9406421099#step:6:1730
Even though we request it be installed: https://github.com/pymc-devs/pytensor/blob/ec6a3153fbf5aaa023368f36aac58212c9bedbd4/.github/workflows/test.yml#L142
So a first step, would be to try and run those tests locally and see if they pass or are broken.
I think this is a setuptools issue. One probably needs to add the correct files to MANIFEST.in
. Or even better, switch to Hatch so that we don't have to deal with setuptools weirdness.
[Deleted wrong comment]
I think this is a setuptools issue. One probably needs to add the correct files to
MANIFEST.in
. Or even better, switch to Hatch so that we don't have to deal with setuptools weirdness.
You mean the specific template.html
issue?
For the tests it seems we actually have to install pydot
and not just graphviz
:
https://github.com/pymc-devs/pytensor/blob/ec6a3153fbf5aaa023368f36aac58212c9bedbd4/pytensor/printing.py#L54-L61
However I am not sure this package is still maintaned? Last release was February 2021: https://pypi.org/project/pydot/#history
Ah, I meant the original issue, not the CI tests
Okay, so we should fix the setup to handle those path issues correctly and decide whether we want to test pydot/graphviz support or not.
If not, I guess there's no other reason to install graphviz either
The pydot_ng
recommendation should definitely be dropped and the code logic as well, since that project is officially archived in favor of pydot
: https://github.com/pydot/pydot-ng
Thanks everyone for trying to solve the problem!
@ricardoV94 : I tried model_to_graphviz()
and dprint()
that you recommended and they seem nice and useful but the first one has much less information about the computational graph than I need and the second one is not very visual (and only for single tensors). So I strongly vote in favor of maintaining the pydot-d3viz implementation.
Yeah, it sounds reasonable to replace the pydot_ng
recommendation by pydot
.
Sure thanks for the feedback.
dprint
allows multiple tensors. Just pass a list with multiple variables, or you can pass a compiled function as well.
But I agree it's a different use case.
Would you have availability to update the docs regarding the ng
library?
Also @maresb do you have more specific recommendations for what the fix could look like?
I committed a pull request to fix the -ng
-recommendation: https://github.com/pymc-devs/pytensor/pull/336
It seems this file used to exist in Theano, and no longer exists in PyTensor: https://github.com/Theano/Theano/blob/master/theano/d3viz/html/template.html
It was lost in the rename to PyTensor: https://github.com/pymc-devs/pytensor/commit/19e1a982e5a2e9ba2bd9d708133abf102c6fb835#diff-976e71f8933be25c4bd13d49f8ebf4c8b9f524585a85b1303409ffc44182d577
Describe the issue:
I followed the example described here: https://pytensor.readthedocs.io/en/latest/library/d3viz/index.html
I did not change any line of the example code provided and everything executes fine (including the installation of
pydot-ng
) until the lined3v.d3viz(predict, 'examples/mlp.html')
, where an error is thrown, even though the import statementimport pytensor.d3viz as d3v
returns no error.The error is:
No such file or directory: '<HOME>/.local/lib/python3.11/site-packages/pytensor/d3viz/html/template.html'
. When checking the directory, I can see that the directory'<HOME>/.local/lib/python3.11/site-packages/pytensor/d3viz/
actually does exist but there is no/html/
folder inside.I also made sure that
graphviz
is installed on the system.What to do about this? Thanks!
Reproducable code example:
Error message:
PyTensor version information:
Pytensor Version 2.12.1, installed with pip Python Version 3.11.3 Linux
Context for the issue:
The graph visualization feature would be very important for my work.