Graphviz seem to miss PNG support.

Carreau commented 8 years ago

See first report blaze/dask#788

When using dask.visualize() the visualization fails if graphviz (actually dotexecutable) is installed via conda.

Format: "png" not recognized. Use one of: canon cmap cmapx cmapx_np dot eps fig gv imap imap_np ismap pic plain plain-ext pov ps ps2 svg svgz tk vml vmlz xdot xdot1.2 xdot1.4

On OS X a simple workaround is to brew-install graphiz which then works great.

Thanks,

OS X 10.10.3, conda 3.18.3 in python 3.4 root environment.

bryevdv commented 8 years ago

I can confirm that I have run into this issue before as well.

mrocklin commented 8 years ago

This has come up several times from dask users. Example here https://github.com/blaze/dask/issues/835

I'm telling people that this packages is broken and that they should uninstall it. Is this incorrect? Given how prevalent this library is within system installers (brew, apt-get) it seems that this package might cause more pain than its worth.

ccordoba12 commented 8 years ago

I think there's a fix in conda-recipes for this, right @ahmadia?

fonnesbeck commented 8 years ago

Removing conda's graphviz and using Homebrew's cask binary has resolved these issues for me. I was getting output files with zero size and no error messages (other than an dyld warning) under the conda distribution.

joejacobs commented 8 years ago

Having the same issue on Debian. Solved by uninstalling graphviz from anaconda and then re-installing graphviz with apt-get.

groutr commented 8 years ago

Looks like our graphviz is missing libgd. I'll update that as soon as I can.

caseyclements commented 8 years ago

Thanks Ryan!

michaelaye commented 8 years ago

same problem here. To the guys that installed brew's graphviz: How do I get that to be known by conda`s python?

WangZhwei commented 8 years ago

thank you! @joejacobs, same problem solved on Ubuntu.

kalefranz commented 8 years ago

@groutr Did we get this graphviz issue fixed?

ozancaglayan commented 8 years ago

I just encountered this issue.

bscipio commented 8 years ago

Same problem here.

bryevdv commented 8 years ago

ping @groutr

groutr commented 8 years ago

Graphviz on linux now has png support (via pangocairo). Due to some problems getting the dependencies of pango to compile on OSX 10.7, that platform doesn't have support for png yet.

eriknw commented 8 years ago

@groutr, Graphviz on linux does not work if libtool (libltdl) is not installed on the system.

ijstokes commented 8 years ago

@groutr Any love on the OS X front? Just ran into this myself. Thank you @mrocklin for the nice error message that pointed me here! Yes, homebrew to the rescue. That solved it for me.

mrocklin commented 8 years ago

I'm going to again recommend that we just remove this package entirely until we have a proper fix for it. A broken package feels far worse than a missing package.

bryevdv commented 8 years ago

Agreed.

On May 6, 2016, at 05:46, Matthew Rocklin notifications@github.com wrote:

I'm going to again recommend that we just remove this package entirely until we have a proper fix for it. A broken package feels far worse than a missing package.

— You are receiving this because you commented. Reply to this email directly or view it on GitHub

ccordoba12 commented 8 years ago

All people interested in seeing this one solved, please follow conda-forge/staged-recipes#568 :-)

mcg1969 commented 8 years ago

Ping. @csoja this is still busted. Ryan was assigned to it, so now it's not assigned to anyone. Note that conda install -c conda-forge graphviz solves the problem for me, which involves updating jpeg and libtiff from conda-forge as well.

jni commented 7 years ago

Update: oops: conda update conda fixed this issue!

@mcg1969 your suggestion still seems to be pulling graphviz from defaults for me... Anyone have any idea how to force conda to download the conda-forge version?

 ~ $ conda search -c conda-forge graphviz
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata: ....
graphviz                     2.38.0                        0  conda-forge
                             2.38.0                        1  conda-forge
                          *  2.38.0                        2  http://repo.continuum.io/pkgs/free/osx-64/

ccordoba12 commented 7 years ago

@mcg1969, I was the one who built graphviz for conda-forge. Unfortunately I haven't had time to move it to defaults, and also to add python-graphviz to them.

mrocklin commented 7 years ago

This issue continues to affect Dask users. It would be great if we could remove this faulty package.

http://stackoverflow.com/questions/42014458/dask-not-isntalling-graphviz-dependency

abalter commented 7 years ago

@mrocklin I'm just confused because the dask.dot graph creating capability seems like a major component of dask. Seems like if you aren't going to support being able to conda install dask and have dask.dot import properly, you should remove the dask.dot feature. Better yet, fix it :)

mrocklin commented 7 years ago

We tell people to install graphviz with their system installer and to avoid conda. Then things work fine. This would be a user's default behavior if this errant conda package was removed.

Carreau commented 7 years ago

Just out of precision for @abalter , it is not because @mrocklin worked/is working for/with continuum that he have the authority or the influence to have the libpng package fixed or removed. Dasked dot is a feature of Dask, on which @mrocklin have most control, and it is a useful feature for many of the non-conda user, as well as conda user that manually set it to use the system libpng.

So unfortunately that's where we can't do much except maybe loby higher in the food chain.

Would tweeting at continuum/pwang/toliphant help to push this forward ?

abalter commented 7 years ago

Oh golly. Sorry if I stepped on toes. Thanks for pointing this out @Carreau. @mrocklin -- if you say it's not working right now, I'm guessing there is a very good reason for it. I've asked our sysadmin to install graphviz natively.

bryevdv commented 7 years ago

I also asked recently @csoja offline about removing this broken package, but I will second the request again here. It has never fully worked in four years, AFAIK.

mcg1969 commented 7 years ago

My understanding is that, technically, it does work. It just doesn't do what people logically expect it should: install the base graphviz package and its python bindings. It just does the former. Nevertheless I agree with the conclusion that it would reduce confusion to remove it.

mrocklin commented 7 years ago

It's not that it's missing the python bindings, it's that it won't emit png files.

On Fri, Feb 3, 2017 at 1:02 PM, Michael C. Grant notifications@github.com wrote:

My understanding is that, technically, it does work. It just doesn't do what people logically expect it should: install the base graphviz package and its python bindings. Nevertheless I agree with the conclusion that it would reduce confusion to remove it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ContinuumIO/anaconda-issues/issues/485#issuecomment-277317668, or mute the thread https://github.com/notifications/unsubscribe-auth/AASszA7gyw6b9AGQhwic-8x_y6lGwx3gks5rY2udgaJpZM4GSY1i .

mrocklin commented 7 years ago

Even if you pip install graphviz afterwards the issues still persist.

On Fri, Feb 3, 2017 at 1:10 PM, Matthew Rocklin mrocklin@continuum.io wrote:

It's not that it's missing the python bindings, it's that it won't emit png files.

On Fri, Feb 3, 2017 at 1:02 PM, Michael C. Grant <notifications@github.com

wrote:

My understanding is that, technically, it does work. It just doesn't do what people logically expect it should: install the base graphviz package and its python bindings. Nevertheless I agree with the conclusion that it would reduce confusion to remove it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ContinuumIO/anaconda-issues/issues/485#issuecomment-277317668, or mute the thread https://github.com/notifications/unsubscribe-auth/AASszA7gyw6b9AGQhwic-8x_y6lGwx3gks5rY2udgaJpZM4GSY1i .

mcg1969 commented 7 years ago

Ah, thanks for the education!

bryevdv commented 7 years ago

yes, it's definitely just plain broken. I've never even wanted it for the python bindings, I wanted it for a sphinx integration (which uses dot directly, not the python bindings). Graphviz has an unusual and baroque build system and I gather this has been part of the problem. But if we can't prioritize fixing it, it should just be removed at this point.

Carreau commented 7 years ago

Oh golly. Sorry if I stepped on toes.

No I don't think you did, don't worry. The distinction Conda/Anaconda/Continuum and various packages/packager can be hard to grasp often. So I was just offering clarifications as to why things are as they are now.

DavidMertz commented 7 years ago

Just to chime in, this bit me again today; having done a rm -rf anaconda3 and fresh install recently. I keep forgetting this broken corner of the Anaconda ecosystem. A working graphviz is a very important tool for lots of purposes, and what conda install graphviz gets you is definitely not a working graphviz.

I have to teach this in a week, and have a special new dread of getting things working for students.

On my system, I have graphviz itself installed with brew, then use pip to install the Python bindings. The "attractive nuisance" of using conda for what it obviously should do (and appears to do) is a problem.

ccordoba12 commented 7 years ago

Graphviz has an unusual and baroque build system and I gather this has been part of the problem

Not now, i.e. after I created a good recipe for conda-forge.

The real problem is that the Anaconda team is always understaffed for the amount of packages we manage.

I have to teach this in a week, and have a special new dread of getting things working for students.

Ok, I'll create packages for graphviz and python-graphviz during the weekend.

I can't promise when they'll be uploaded though because that doesn't depend on me.

Carreau commented 7 years ago

Thanks @ccordoba12 for taking some time to do it.

DavidMertz commented 7 years ago

I think the pip install pygraphviz was really just irrelevant. But I was distracted by the fact I knew I had the brew install working system wide, yet the message said “You need both graphviz and the Python bindings.” If I understand correctly pygraphviz and graphviz are two separate Python packages (on PyPI) that serve mostly the same purpose, but with slightly different APIs. That is, either one provides Python functions/classes to create graphs, then passes rendering to the actual graphviz library (or maybe to the executables—dot, neato, etc).

In particular, Dask chose to use the Python module called graphviz rather than the one called pygraphviz. There are also packages graphviz-python and pydot that also “compete” with those two. I have no opinion on the choice of Python library, I trust Matt’s judgement (or whoever made the choice).

Whatever it is that conda install graphviz installs is not a Python binding, but is intended to be the system library/utilities. However, it is a non-functional version of that library/utilities. As Matt noted in the GH issue, it would be FAR better for that not to exist at all than to install something non-functional.

Actually, I guess the breakage is slightly more subtle. Maybe in libgvplugin_dot_layout.dylib or somewhere similar. I can do this:

% conda create -n graphviz python graphviz
% which dot
/usr/local/bin/dot
activate graphviz
(graphviz) % which dot
//anaconda/envs/graphviz/bin/dot
(graphviz) % dot atoms.dot -Tpng -oatoms.png
(graphviz) % deactivate
% dot atoms.dot -Tpng -oatoms.png

Both my conda graphviz and my system graphviz create a PNG from the command line.

ccordoba12 commented 7 years ago

There are new graphviz packages in our default channels that have PNG support since the beginning of 2017. I didn't know that, please try them!

@DavidMertz, our graphviz package contains only C libraries and binaries (e.g. the dot executable). We decided to call python-graphviz to the Python bindings that on PyPi are called simply graphviz.

That's an unfortunate clash of names, but after a very long discussion in conda-forge, both Continuum and conda-forge decided that the best way to solve this conflict was to use the graphviz name for the original (i.e. C) project, and python-graphviz for its bindings (I know there are several other Python bindings, but none of them is just called graphviz).

This is just another proof that conda is a general package manager and not just a pip replacement of sorts. So

conda install graphviz

and

pip install graphviz

will always give different results, and we have to learn to live with that (and make users aware of that too).

DavidMertz commented 7 years ago

My training customer has a firewall, and a private repo. They frequently mirror the main channel, but I'm pretty sure they will not mirror conda-forge. Is it possible to get python-graphviz into the main channel?

ccordoba12 commented 7 years ago

Is it possible to get python-graphviz into the main channel?

Yep, I added a recipe for it to our internal repo. It should be available shortly.

Carreau commented 7 years ago

Thank you Carlos, much appreciated !

On Feb 13, 2017 15:42, "Carlos Cordoba" notifications@github.com wrote:

Is it possible to get python-graphviz into the main channel?

Yep, I added a recipe for it to our internal repo. It should be available shortly.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ContinuumIO/anaconda-issues/issues/485#issuecomment-279560366, or mute the thread https://github.com/notifications/unsubscribe-auth/AAUez4chCZvU_WQBLsxV2zRU2KNpyObfks5rcOpXgaJpZM4GSY1i .

mrocklin commented 7 years ago

Windows error reported here: http://stackoverflow.com/questions/43983089/dask-dot-runtimeerror-visualizing

They seem to have updated graphviz already.

Joshuaalbert commented 7 years ago

To clarify, the setup for the SO issue that @mrocklin just refered to is: pip install graphviz for the python bindings which installs 0.7 conda install graphviz for the system executables which installs 2.38.0

and I just now got it to work by installing graphviz from http://www.graphviz.org/ and then putting this before the conda graphviz path (I should really just remove the conda graphviz). So it seems the conda graphviz is the issue.

ccordoba12 commented 7 years ago

Windows error reported here: http://stackoverflow.com/questions/43983089/dask-dot-runtimeerror-visualizing

I solved this problem already and asked the Anaconda team to create packages for python-graphviz (which is the name for the package you get when you run pip install graphviz).

This package was created for Mac and Linux, but (I don't know why) not for Windows. I'll ask the team again.

@Joshuaalbert, sorry for the inconvenience this caused you.

@mrocklin, please remember that conda install graphviz gives you the C libraries and executables. The Python bindings that dask use are obtained with conda install python-graphviz.

ccordoba12 commented 7 years ago

Packages for python-graphviz on Windows are now available in our channels.

Thanks for your patience.

poleguy commented 7 years ago

I can't get this to work on windows. Depending on which of the various pip/conda/path commands listed here I run and in what order, I get various error messages or no output with no error message. Can someone check that this should work on windows?

ccordoba12 commented 7 years ago

It is working on Windows. You need to run

conda install graphviz python-graphviz

and remove the pip graphviz package to make it work.

poleguy commented 7 years ago

I just tried again using that command on a different Windows 7 box than the one I tried the first time. It doesn't work... but now I see that the problem seems to be the handling of a space in the path to dot.py

I guess that's not really a problem with graphviz... I'm not sure where to report this as a separate bug, but it manifests itself as a link to this thread, so it's probably not bad to leave a clue here for the next guy. I'll reinstall anaconda in a path with no spaces and try again.

'C:\Program' is not recognized as an internal or external command, operable program or batch file. Traceback (most recent call last): File "G:/data/Dropbox/SVN/ppprsGui/dask_test.py", line 26, in dot_graph(b2_min.dask) File "C:\Program Files\Anaconda3\lib\site-packages\dask\dot.py", line 227, in dot_graph raise RuntimeError("Graphviz failed to properly produce an image. " RuntimeError: Graphviz failed to properly produce an image. This probably means your installation of graphviz is missing png support. See: https://github.com/ContinuumIO/anaconda-issues/issues/485 for more information.

poleguy commented 7 years ago

No dice. I reinstalled the latest anaconda3 on windows 7 in a path with no space. It now gives no error, but does not produce any image.

It returns only: dask.array<amin-aggregate, shape=(), dtype=float64, chunksize=()>

import numpy as np from dask.dot import dot_graph import dask.array as da a = np.random.randn(1000) a2 = da.from_array(a, chunks=200) b2 = a2 * 4 b2_min = b2.min() dot_graph(b2_min.dask) print(b2_min)

ContinuumIO / anaconda-issues

Graphviz seem to miss PNG support. #485