XifengGuo / CapsNet-Keras

A Keras implementation of CapsNet in NIPS2017 paper "Dynamic Routing Between Capsules". Now test error = 0.34%.
MIT License
2.47k stars 651 forks source link

pydot issue #7

Closed qihongl closed 7 years ago

qihongl commented 7 years ago

Hi, I got the following error message (copied below) when I tried the code, although I do have both pydot and graphviz. I don't think this is an issue with your code. It seems to be a general long-standing issue with pydot. The solutions are discussed here: 1: https://github.com/Theano/Theano/issues/1801 2: https://github.com/erocarrera/pydot/issues/126

Traceback (most recent call last): File "/Users/Qihong/anaconda/envs/brainiak/lib/python3.6/site-packages/keras/utils/vis_utils.py", line 23, in _check_pydot pydot.Dot.create(pydot.Dot()) File "/Users/Qihong/anaconda/envs/brainiak/lib/python3.6/site-packages/pydot_ng/init.py", line 1890, in create 'GraphViz\'s executables not found') pydot_ng.InvocationException: GraphViz's executables not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "capsulenet.py", line 195, in plot_model(model, to_file=args.save_dir+'/model.png', show_shapes=True) File "/Users/Qihong/anaconda/envs/brainiak/lib/python3.6/site-packages/keras/utils/vis_utils.py", line 131, in plot_model dot = model_to_dot(model, show_shapes, show_layer_names, rankdir) File "/Users/Qihong/anaconda/envs/brainiak/lib/python3.6/site-packages/keras/utils/vis_utils.py", line 52, in model_to_dot _check_pydot() File "/Users/Qihong/anaconda/envs/brainiak/lib/python3.6/site-packages/keras/utils/vis_utils.py", line 27, in _check_pydot raise ImportError('Failed to import pydot. You must install pydot' ImportError: Failed to import pydot. You must install pydot and graphviz for pydotprint to work.

XifengGuo commented 7 years ago

This is a bug of Keras. You maybe solve this by removing the plot_model function and from keras.utils.vis_utils import plot_model.

Or you can install pydot 1.2.3 by pip. pip install pydot==1.2.3

I have not re-tested these solutions, if they work, please let me know. Thanks. @QihongL

qihongl commented 7 years ago

Yes! I commented those lines since I don't need that model diagram. By the way, thanks for the implementation!

dlazares commented 6 years ago

FWIW pip install pydot==1.2.3 didn't work for me. Commenting out the lines worked.

Codeguyross commented 6 years ago

Im running python 3.5. pydot 1.2.3 and i have installed graphviz but plot_model doesn't work. Does anyone have a fix for the plot_model function that corrects whatever is wrong with it? If it helps this is the function in keras.utils.vis_utils.

def plot_model(model,
               to_file='model.png',
               show_shapes=False,
               show_layer_names=True,
               rankdir='TB'):
    """Converts a Keras model to dot format and save to a file.

    # Arguments
        model: A Keras model instance
        to_file: File name of the plot image.
        show_shapes: whether to display shape information.
        show_layer_names: whether to display layer names.
        rankdir: `rankdir` argument passed to PyDot,
            a string specifying the format of the plot:
            'TB' creates a vertical plot;
            'LR' creates a horizontal plot.
    """
    dot = model_to_dot(model, show_shapes, show_layer_names, rankdir)
    _, extension = os.path.splitext(to_file)
    if not extension:
        extension = 'png'
    else:
        extension = extension[1:]
    dot.write(to_file, format=extension)
kostistsaprailis commented 6 years ago

I'm writing this here in case anyone finds this again.

Checking the code for keras/utils/vis_utils.py:

"""Utilities related to model visualization."""
import os

try:
    # pydot-ng is a fork of pydot that is better maintained.
    import pydot_ng as pydot
except ImportError:
    # pydotplus is an improved version of pydot
    try:
        import pydotplus as pydot
    except ImportError:
        # Fall back on pydot if necessary.
        try:
            import pydot
        except ImportError:
            pydot = None

def _check_pydot():
    try:
        # Attempt to create an image of a blank graph
        # to check the pydot/graphviz installation.
        pydot.Dot.create(pydot.Dot())
    except Exception:
        # pydot raises a generic Exception here,
        # so no specific class can be caught.
        raise ImportError('Failed to import pydot. You must install pydot'
                          ' and graphviz for `pydotprint` to work.')

we can see the error raised by the try except clause above. This tries to run a dummy pydot command in to confirm that pydot is installed correctly. However the first lines how that the code tries to install initially the package pydot_ng then pydotplus and finally if it's not found the pydot package.

So I installed pydot_ng and it worked for me. However I had another issue. The system couldn't find the GraphViz executables:

InvocationException: GraphViz's executables not found

Under Ubuntu 16.04 I ran:

sudo apt-get install graphviz

and it worked.

AFAgarap commented 6 years ago

Installing graphviz as mentioned by @kostistsaprailis worked for me too.

Behrad3d commented 6 years ago

For future reference: If you're using a Mac using Pip to install graphviz is not enough. You need to use brew brew install graphviz

HemersonTacon commented 6 years ago

On Windows I had to install de binaries (https://graphviz.gitlab.io/_pages/Download/Download_windows.html) and then set the bin folder on PATH environment variable

sakimilo commented 6 years ago

yup, by install using apt-get works for me as well! sudo apt-get install graphviz

abhishek95 commented 6 years ago

I did the following for Mac:-

  1. Installed pydot_ng (pip install pydot_ng)
  2. Go to keras/utils/vis_util.py (this is the file showing error)

You will see something like this. try:

import pydot except ImportError: pydot = None

  1. Change import pydot to import pydot_ng as pydot

And voila!

manuelblancovalentin commented 6 years ago

None of the above worked for me as none of them were "immediately" applied to my jupyter-notebook session. As I was running a very long code on jupyter and COULD NOT LOSE MY DATA, BY ANY CHANCE, installing these modules was not sufficient, as even reloading the modules would not make the trick. Instead I needed to restart the kernel, or spyder-ide (which would make me lose my data).

So I simply opened a terminal (ubuntu here) and edited the /keras/utils/vis_util.py file (for me, using anaconda, it was under: /home//miniconda2/lib/python2.7/site-packages/keras/utils/vis_util.py) and commented the "check_pydot()" call inside the "model_to_dot" function definition.

After that, reloading the code inside jupyter-notebook was enough to render the net architecture WITHOUT HAVING TO RESTART THE JUPYTER KERNEL, AND WITHOUT LOSING THE DATA. No other errors appeared, btw, and the net was rendered properly.

hamk3010 commented 6 years ago

Importing pydot-ng wasn't working for me since I was using python 3.6

UnsatisfiableError: The following specifications were found to be in conflict: - pydot-ng -> python=2.7 - python=3.6 Importing pydotplus worked for me

rgkimball commented 6 years ago

I also ran into kostistsaprailis's issue but I'm not on a *nix so the previous answers didn't apply.

If you're on a Windows machine, obtaining the GraphViz library is slightly less convenient without a native package manager. Looking at pydot_ng's find_graphviz() , it first checks your registry, then your path via os.environ but these may not be configured if you downloaded the source instead. If you install GraphViz to: C:\Program Files\att\GraphViz then it will locate the executables within the bin subdirectory there.

dmpierre commented 6 years ago

For those still struggling:

First:

$ pip install pydot-ng

As explained, pydot-ng is actually better maintained.

Second, go to where your keras repo is set up. E.g. mine is a virtualenv at:

$ cd ~/.virtualenvs/venv-name/lib/python3.6/site-packages/keras/utils/

Third, at line 11 of vis_utils.py, change to:

try:
    import pydot_ng as pydot
except ImportError:
    pydot = None

Ensure graphviz is installed (using brew, apt-get, yum or whatever your OS is), this should work :)

JialieY commented 6 years ago

Download the dll from https://graphviz.gitlab.io/_pages/Download/Download_windows.html

and then append the bin folder path in your system path, it will work.

The problem is they only check the dot.exe in the system path or machine registration.

JialieY commented 6 years ago

This is a bug, keras should fix it.

Hippyu commented 5 years ago

Ensure you have already installed graphviz, pydot, pydotplus. pydot-ngis not satisfied with python=3.6.8, but python=3.4, where I found by conda info pydot-ng So what I did is followed.

import pydotplus from keras.utils.vis_utils import model_to_dot import keras keras.utils.vis_utils.pydot = pydot plot_model(autoencoder, to_file='model.png')

that's worked for me. On mac Anaconda. python=3.6.8

Jeff-Winchell commented 5 years ago

What a cluster$Q$5

Testing code, encapsulation, robustness to change are all software engineering problems solved decades ago. Data science has thrown them all into the garbage can in order to push out partially working code faster. No thank you.

yijiejiang commented 5 years ago

I got this error recently, I install the pydot and pydot-ng , and I brew install grapgviz in my env, I can see these packages in my interpreter, but it do not work even if I try edit vis_utils.py, I solve the problem by using sudo apt-get install grapgviz.

matheushent commented 5 years ago

I am using windows 10 and had the same issue. First, I commented the "_check_pydot()" as @manuelblancovalentin mentioned above. After that, the code kept doesn't run. So I did what @JialieY said and violà, everything right.

Furjoza commented 5 years ago

This is a bug of Keras. You maybe solve this by removing the plot_model function and from keras.utils.vis_utils import plot_model.

Or you can install pydot 1.2.3 by pip. pip install pydot==1.2.3

I have not re-tested these solutions, if they work, please let me know. Thanks. @QihongL

Thanks. I have googled half of the internet to find this comment to downgrade the package. Thank you very much.

squallssck commented 5 years ago

For those still struggling:

First:

$ pip install pydot-ng

As explained, pydot-ng is actually better maintained.

Second, go to where your keras repo is set up. E.g. mine is a virtualenv at:

$ cd ~/.virtualenvs/venv-name/lib/python3.6/site-packages/keras/utils/

Third, at line 11 of vis_utils.py, change to:

try:
    import pydot_ng as pydot
except ImportError:
    pydot = None

Ensure graphviz is installed (using brew, apt-get, yum or whatever your OS is), this should work :)

install pydot_ng solves my problem :) ... again , on my win10 env i have to install graphvis.msi , set bin to path and then pip install pydot_ng to make keras plot_model to work.

tzamalisp commented 4 years ago

the solution that @kostistsaprailis suggested worked fine for me!

hessam94 commented 4 years ago

install graphviz on windows and add bin to env var wokred for me,

micuevisa commented 3 years ago

Tried everything and nothing works. uninstalled all pydot related modules and graphiviz, then reinstalled in the following order:

pydot_ng

    #pydot
    #pydotplus
    #graphviz
    #On Windows I had to install de binaries (https://graphviz.gitlab.io/_pages/Download/Download_windows.html) which also sets the PATH variable

It's broken.

Calandiel commented 3 years ago

Four years later and it's still broken

Daniel-Chin commented 2 years ago

I am on Windows and I finally got it working.

The problem was that restarting the kernal in Jupyter Notebook does not restart the system shell, so %PATH% was outdated!!!

So the complete solution on Windows:

(What happened: Windows caches %PATH% at the launch of the shell. Restarting the kernal only starts a new python session in the same shell. So even if you start a new kernal and import os again, os will still contain the old %PATH%.)

usmandroid commented 2 years ago

Ensure you have already installed graphviz, pydot, pydotplus. pydot-ngis not satisfied with python=3.6.8, but python=3.4, where I found by conda info pydot-ng So what I did is followed.

import pydotplus from keras.utils.vis_utils import model_to_dot import keras keras.utils.vis_utils.pydot = pydot plot_model(autoencoder, to_file='model.png')

that's worked for me. On mac Anaconda. python=3.6.8

This worked for me on Python 3.10 on Mac M1 minoforge3 environment.! Kudos to you! Before running this I made sure that all the packages are installed. No need to edit the vis_utils file! pydot — 1.4.2 pydot-ng — 2.0.0 pydotplus — 2.0.2 keras — 2.9.0 python — 3.10.5

hadiloghman commented 2 years ago

I am on Windows and I finally got it working.

The problem was that restarting the kernal in Jupyter Notebook does not restart the system shell, so %PATH% was outdated!!!

So the complete solution on Windows:

  • install graphviz binaries
  • add the binaries to %PATH%
  • install pydot_ng
  • restart the Jupyter Notebook server!

(What happened: Windows caches %PATH% at the launch of the shell. Restarting the kernal only starts a new python session in the same shell. So even if you start a new kernal and import os again, os will still contain the old %PATH%.)

Great!!! worked for me. TNX

maurlco commented 1 year ago

Ensure you have already installed graphviz, pydot, pydotplus. pydot-ngis not satisfied with python=3.6.8, but python=3.4, where I found by conda info pydot-ng So what I did is followed.

import pydotplus from keras.utils.vis_utils import model_to_dot import keras keras.utils.vis_utils.pydot = pydot plot_model(autoencoder, to_file='model.png')

that's worked for me. On mac Anaconda. python=3.6.8

This worked for me on Python 3.10 on Mac M1 minoforge3 environment.! Kudos to you! Before running this I made sure that all the packages are installed. No need to edit the vis_utils file! pydot — 1.4.2 pydot-ng — 2.0.0 pydotplus — 2.0.2 keras — 2.9.0 python — 3.10.5

This worked for me - I installed the packages as above, the pydotplus was missing for me. It works even with the file vis_utils edited. Then import the above package and "keras.utils.vis_utils.pydot = pydot" on my Jupiter notebook. Thank you @usmandroid