Diagram generation appears to be broken

[matt3o]

• Python version: Python 3.12.3 inside a venv
• Operating System: Ubuntu 24 inside WSL

Description

Diagram generation does not work. I first tried it on my project and thought it was my bad, however even with one of the toy examples I could not get it to run. I'll paste the file below and the commands. I did some testing and found out the that the issue is the fontsize=10pt;, to be more precise the pt part. I wrote a little function that removes all the pts after the font and it generates perfectly well again.

What I Did

import random

from statemachine import State
from statemachine import StateMachine

class GuessTheNumberMachine(StateMachine):
    start = State(initial=True)
    low = State()
    high = State()
    won = State(final=True)
    lose = State(final=True)

    guess = (
        lose.from_(low, high, cond="max_guesses_reached")
        | won.from_(low, high, start, cond="guess_is_equal")
        | low.from_(low, high, start, cond="guess_is_lower")
        | high.from_(low, high, start, cond="guess_is_higher")
    )

    def __init__(self, max_attempts=5, lower=1, higher=5, seed=42):
        self.max_attempts = max_attempts
        self.lower = lower
        self.higher = higher
        self.guesses = 0

        # lets play a not so random game, or our tests will be crazy
        random.seed(seed)
        self.number = random.randint(self.lower, self.higher)
        super().__init__()

    def max_guesses_reached(self):
        return self.guesses >= self.max_attempts

    def before_guess(self, number):
        self.guesses += 1
        print(f"Your guess is {number}...")

    def guess_is_lower(self, number):
        return number < self.number

    def guess_is_higher(self, number):
        return number > self.number

    def guess_is_equal(self, number):
        return self.number == number

    def on_enter_start(self):
        print(f"(psss.. don't tell anyone the number is {self.number})")
        print(
            f"I'm thinking of a number between {self.lower} and {self.higher}. "
            f"Can you guess what it is?"
        )

    def on_enter_low(self):
        print("Too low. Try again.")

    def on_enter_high(self):
        print("Too high. Try again.")

    def on_enter_won(self):
        print(f"Congratulations, you guessed the number in {self.guesses} guesses!")

    def on_enter_lose(self):
        print(f"Oh, no! You've spent all your {self.guesses} attempts!")

Log output (filenames and module censored):

 python -m statemachine.contrib.diagram test.GuessTheNumberMachine state_machine.png
"dot" with args ['-Tpng', '/tmp/tmph5zx2pi2'] returned code: 1

stdout, stderr:
 b''
b"Warning: syntax ambiguity - badly delimited number '10p' in line 4 of /tmp/tmph5zx2pi2 splits into two tokens\nWarning: syntax ambiguity - badly delimited number '1p' in line 6 of /tmp/tmph5zx2pi2 splits into two tokens\nError: /tmp/tmph5zx2pi2: syntax error in line 6 near ','\n"

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/mnt/c/Users//code//.lvenv/lib/python3.12/site-packages/statemachine/contrib/diagram.py", line 243, in <module>
    sys.exit(main())
             ^^^^^^
  File "/mnt/c/Users//code//.lvenv/lib/python3.12/site-packages/statemachine/contrib/diagram.py", line 239, in main
    write_image(qualname=args.class_path, out=args.out)
  File "/mnt/c/Users//code//.lvenv/lib/python3.12/site-packages/statemachine/contrib/diagram.py", line 220, in write_image
    graph.write(out, format=out_extension)
  File "/mnt/c/Users//code//.lvenv/lib/python3.12/site-packages/pydot/core.py", line 1708, in write
    s = self.create(prog, format, encoding=encoding)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/c/Users//code//.lvenv/lib/python3.12/site-packages/pydot/core.py", line 1825, in create
    assert process.returncode == 0, (
           ^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: "dot" with args ['-Tpng', '/tmp/tmph5zx2pi2'] returned code: 1

[fgmacedo]

Hi @matt3o , how are you?

I was not able to reproduce the issue locally:

python -m statemachine.contrib.diagram tests.examples.guess_the_number_machine.GuessTheNumberMachine state_machine.png

Produced this, without warnings.

Can you please share the versions of the involved libs in your system?

Mine are:

• Libs:

  python-statemachine=2.3.4
  pydot==2.0.0

• Graphviz

  ❯ dot -V
  dot - graphviz version 2.43.0 (0)

Ubuntu inside WSL

❯ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.4 LTS
Release:        22.04
Codename:       jammy

[fgmacedo]

I was using the graphviz from the default Ubuntu repositories, installed using sudo apt install graphviz.

After manually upgrading to the version 12.0.0 (latest release available at https://gitlab.com/graphviz/graphviz/-/releases), I was able to reproduce the issue.

[matt3o]

Ah sweet, I was about to send you the libs :D Nice to meet you Fernando, hope you are doing good as well.

At least for the problem above I think remove the font size pt part would be enough. For my bigger state machine that appears to be true as well.

Thanks a lot for looking into that, if you need any more information, just ping me!

[fgmacedo]

Hi, thanks !

I've found that my attempt to install the latest graphviz is failing. It installs but without plugins.

Can you share your output of dot -v?

[matt3o]

Sure!

dot - graphviz version 2.43.0 (0)
libdir = "/usr/lib/x86_64-linux-gnu/graphviz"
Activated plugin library: libgvplugin_dot_layout.so.6
Using layout: dot:dot_layout
Activated plugin library: libgvplugin_core.so.6
Using render: dot:core
Using device: dot:dot:core
The plugin configuration file:
        /usr/lib/x86_64-linux-gnu/graphviz/config6a
                was successfully loaded.
    render      :  cairo dot dot_json fig gd json json0 map mp pic pov ps svg tk visio vml vrml xdot xdot_json
    layout      :  circo dot fdp neato nop nop1 nop2 osage patchwork sfdp twopi
    textlayout  :  textlayout
    device      :  bmp canon cmap cmapx cmapx_np dot dot_json eps fig gd gd2 gif gtk gv ico imap imap_np ismap jpe jpeg jpg json json0 mp pdf pic plain plain-ext png pov ps ps2 svg svgz tif tiff tk vdx vml vmlz vrml wbmp webp x11 xdot xdot1.2 xdot1.4 xdot_json xlib
    loadimage   :  (lib) bmp eps gd gd2 gif ico jpe jpeg jpg png ps svg webp xbm

[fgmacedo]

I was not able to reproduce the issue :(

The problem I've found was related to a broken attempt to install graphviz latest version from binaries, that resulted on this error:

 ❯ dot -v
dot - graphviz version 12.0.0 (20240704.0754)
There is no layout engine support for "dot"
Perhaps "dot -c" needs to be run (with installer's privileges) to register the plugins?

When I tried to write the diagram, the output was:

"dot" with args ['-Tdot', '/tmp/tmpoxpcac_q'] returned code: 1

stdout, stderr:
 b''
b'Format: "dot" not recognized. No formats found.\nPerhaps "dot -c" needs to be run (with installer\'s privileges) to register the plugins?\n'

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/macedo/projects/python-statemachine/.venv/lib/python3.12/site-packages/pydot/core.py", line 1587, in new_method
    self.write(path, format=f, prog=prog, encoding=encoding)
  File "/home/macedo/projects/python-statemachine/.venv/lib/python3.12/site-packages/pydot/core.py", line 1662, in write
    s = self.create(prog, format, encoding=encoding)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/macedo/projects/python-statemachine/.venv/lib/python3.12/site-packages/pydot/core.py", line 1785, in create
    assert (
AssertionError: "dot" with args ['-Tdot', '/tmp/tmpoxpcac_q'] returned code: 1

But it's not similar to the problem you're encountering:

b"Warning: syntax ambiguity - badly delimited number '10p' in line 4 of /tmp/tmph5zx2pi2 splits into two tokens\nWarning: syntax ambiguity - badly delimited number '1p' in line 6 of /tmp/tmph5zx2pi2 splits into two tokens\nError: /tmp/tmph5zx2pi2: syntax error in line 6 near ','\n"

After this, I installed Graphviz from the source, which worked as expected.

Can you share a dot file resulting from your state machine?

Assuming that the file test.py contains the class GuessTheNumberMachine, this will generate an sm.dot.txt file:

python -c  "from tests import GuessTheNumberMachine as SM; m=SM(); open('sm.dot.txt','w').write(str(m._graph()))"

Mine was: sm.dot.txt

That can be turned into a sm.svg diagram using graphviz given this command:

dot -Tsvg -o sm.svg sm.dot.txt

This file works locally with graphviz 2.3 (old versions distributed on default Ubuntu repositories), the latest version 12.0, and the online version: https://dreampuf.github.io/GraphvizOnline.

[matt3o]

Thanks for the comamnds, same error, here is the output.

The dot file, as instructed with python -c "from guess_the_number_machine import GuessTheNumberMachine as SM; m=SM(); open('sm.dot.txt','w').write(str(m._graph()))"

digraph list {
label=GuessTheNumberMachine;
fontname=Arial;
fontsize=10pt;
rankdir=LR;
i [shape=circle, style=filled, fontsize=1pt, fixedsize=true, width=0.2, height=0.2, fillcolor=black];
i -> start [label="", color=blue, fontname=Arial, fontsize=9pt];
high [label="High\nentry / on_enter_high", shape=rectangle, style="rounded, filled", fontname=Arial, fontsize=10pt, peripheries=1, fillcolor=white];
high -> lose [label="guess\n[max_guesses_reached]", color=blue, fontname=Arial, fontsize=9pt];
high -> won [label="guess\n[guess_is_equal]", color=blue, fontname=Arial, fontsize=9pt];
high -> low [label="guess\n[guess_is_lower]", color=blue, fontname=Arial, fontsize=9pt];
high -> high [label="guess\n[guess_is_higher]", color=blue, fontname=Arial, fontsize=9pt];
lose [label="Lose\nentry / on_enter_lose", shape=rectangle, style="rounded, filled", fontname=Arial, fontsize=10pt, peripheries=2, fillcolor=white];
low [label="Low\nentry / on_enter_low", shape=rectangle, style="rounded, filled", fontname=Arial, fontsize=10pt, peripheries=1, fillcolor=white];
low -> lose [label="guess\n[max_guesses_reached]", color=blue, fontname=Arial, fontsize=9pt];
low -> won [label="guess\n[guess_is_equal]", color=blue, fontname=Arial, fontsize=9pt];
low -> low [label="guess\n[guess_is_lower]", color=blue, fontname=Arial, fontsize=9pt];
low -> high [label="guess\n[guess_is_higher]", color=blue, fontname=Arial, fontsize=9pt];
start [label="Start\nentry / on_enter_start", shape=rectangle, style="rounded, filled", fontname=Arial, fontsize=10pt, peripheries=1, penwidth=2, fillcolor=turquoise];
start -> won [label="guess\n[guess_is_equal]", color=blue, fontname=Arial, fontsize=9pt];
start -> low [label="guess\n[guess_is_lower]", color=blue, fontname=Arial, fontsize=9pt];
start -> high [label="guess\n[guess_is_higher]", color=blue, fontname=Arial, fontsize=9pt];
won [label="Won\nentry / on_enter_won", shape=rectangle, style="rounded, filled", fontname=Arial, fontsize=10pt, peripheries=2, fillcolor=white];
}

dot -Tsvg -o sm.svg sm.dot.txt
Warning: syntax ambiguity - badly delimited number '10p' in line 4 of sm.dot.txt splits into two tokens
Warning: syntax ambiguity - badly delimited number '1p' in line 6 of sm.dot.txt splits into two tokens
Error: sm.dot.txt: syntax error in line 6 near ','

If I remove the pt in line 6, then this error occurs:

dot -Tsvg -o sm.svg sm.dot.txt
Warning: syntax ambiguity - badly delimited number '10p' in line 4 of sm.dot.txt splits into two tokens
Warning: syntax ambiguity - badly delimited number '9p' in line 7 of sm.dot.txt splits into two tokens
Error: sm.dot.txt: syntax error in line 7 near ']'

[fgmacedo]

Can you please share your version of pydot? (You can get from the output of pip freeze)

If you see my example dot output, these fontsize appears quoted. Yours are without quotes. Maybe is an older version of pydot already installed. I didn't pinned a minimal version.

[SebastianSchiefer]

Not OP but i am having the same issue and it seems like the used pydot version makes the difference.

I installed pydot with pip install pydot which gave me version 3.0.1. This version has the issue. After seeing this comment chain i downgraded pydot to the version you used(2.0.0) and tried to print a graph again. Now the graph has the fontsizes in quotes and i can generate a png. Graphviz is the same version as OP(2.4.3)

See console output for reference:

### SKIPPING STATEMACHINE DECLARATION
>>> print(SingleAgentStateController()._graph())
digraph list {
label=SingleAgentStateController;
fontname=Arial;
fontsize=10pt;
rankdir=LR;
i [shape=circle, style=filled, fontsize=1pt, fixedsize=true, width=0.2, height=0.2, fillcolor=black];
i -> create_task [label="", color=blue, fontname=Arial, fontsize=9pt];
create_task [label="Create task", shape=rectangle, style="rounded, filled", fontname=Arial, fontsize=10pt, peripheries=1, penwidth=2, fillcolor=turquoise];
create_task -> generate_tests [label=cycle, color=blue, fontname=Arial, fontsize=9pt];
execute_environment [label="Execute environment\nentry / execute_environment_action", shape=rectangle, style="rounded, filled", fontname=Arial, fontsize=10pt, peripheries=1, fillcolor=white];
execute_environment -> fix_code_error [label="cycle\n[fix_errors_condition]", color=blue, fontname=Arial, fontsize=9pt];
execute_environment -> finished [label=cycle, color=blue, fontname=Arial, fontsize=9pt];
finished [label=Finished, shape=rectangle, style="rounded, filled", fontname=Arial, fontsize=10pt, peripheries=2, fillcolor=white];
fix_code_error [label="Fix code error\nentry / fix_code_error_action", shape=rectangle, style="rounded, filled", fontname=Arial, fontsize=10pt, peripheries=1, fillcolor=white];
fix_code_error -> execute_environment [label=cycle, color=blue, fontname=Arial, fontsize=9pt];
generate_code [label="Generate code", shape=rectangle, style="rounded, filled", fontname=Arial, fontsize=10pt, peripheries=1, fillcolor=white];
generate_code -> execute_environment [label=cycle, color=blue, fontname=Arial, fontsize=9pt];
generate_tests [label="Generate tests", shape=rectangle, style="rounded, filled", fontname=Arial, fontsize=10pt, peripheries=1, fillcolor=white];
generate_tests -> generate_code [label=cycle, color=blue, fontname=Arial, fontsize=9pt];
}

>>> exit()
(aigen_playground) sebastian@sebastian-MS-7C35:~/Projects/my-multi-agent$ pip freeze | grep pydot
pydot==3.0.1
(aigen_playground) sebastian@sebastian-MS-7C35:~/Projects/my-multi-agent$ pip uninstall pydot
Found existing installation: pydot 3.0.1
Uninstalling pydot-3.0.1:
  Would remove:
    /home/sebastian/.pyenv/versions/3.11.6/envs/aigen_playground/lib/python3.11/site-packages/pydot-3.0.1.dist-info/*
    /home/sebastian/.pyenv/versions/3.11.6/envs/aigen_playground/lib/python3.11/site-packages/pydot/*
Proceed (Y/n)? y
  Successfully uninstalled pydot-3.0.1
(aigen_playground) sebastian@sebastian-MS-7C35:~/Projects/my-multi-agent$ pip install pydot==2.0.0

### SKIPPING STATEMACHINE DECLARATION
>>> print(SingleAgentStateController()._graph())

digraph list {
fontname=Arial;
fontsize="10pt";
label=SingleAgentStateController;
rankdir=LR;
i [fillcolor=black, fixedsize=true, fontsize="1pt", height=0.2, shape=circle, style=filled, width=0.2];
i -> create_task  [color=blue, fontname=Arial, fontsize="9pt", label=""];
create_task [fillcolor=turquoise, fontname=Arial, fontsize="10pt", label="Create task", penwidth=2, peripheries=1, shape=rectangle, style="rounded, filled"];
create_task -> generate_tests  [color=blue, fontname=Arial, fontsize="9pt", label=cycle];
execute_environment [fillcolor=white, fontname=Arial, fontsize="10pt", label="Execute environment\nentry / execute_environment_action", peripheries=1, shape=rectangle, style="rounded, filled"];
execute_environment -> fix_code_error  [color=blue, fontname=Arial, fontsize="9pt", label="cycle\n[fix_errors_condition]"];
execute_environment -> finished  [color=blue, fontname=Arial, fontsize="9pt", label=cycle];
finished [fillcolor=white, fontname=Arial, fontsize="10pt", label=Finished, peripheries=2, shape=rectangle, style="rounded, filled"];
fix_code_error [fillcolor=white, fontname=Arial, fontsize="10pt", label="Fix code error\nentry / fix_code_error_action", peripheries=1, shape=rectangle, style="rounded, filled"];
fix_code_error -> execute_environment  [color=blue, fontname=Arial, fontsize="9pt", label=cycle];
generate_code [fillcolor=white, fontname=Arial, fontsize="10pt", label="Generate code", peripheries=1, shape=rectangle, style="rounded, filled"];
generate_code -> execute_environment  [color=blue, fontname=Arial, fontsize="9pt", label=cycle];
generate_tests [fillcolor=white, fontname=Arial, fontsize="10pt", label="Generate tests", peripheries=1, shape=rectangle, style="rounded, filled"];
generate_tests -> generate_code  [color=blue, fontname=Arial, fontsize="9pt", label=cycle];
}

I think at least on my machine the issue was that the recommended install step python3 -m pip install "python-statemachine[diagrams]" did not work. When i installed with this command, i had to install pydot manually afterwards and i got version 3.0.1 when i did not specify the exact version you used:

(aigen_playground) sebastian@sebastian-MS-7C35:~/Projects/my-multi-agent$ pip install python-statemachine[diagrams]
Collecting python-statemachine[diagrams]
  Obtaining dependency information for python-statemachine[diagrams] from https://files.pythonhosted.org/packages/75/e0/2f021c830d0cb3f78e9d372f7ecceebde48f9d1b11ef68c87149e931fd77/python_statemachine-2.3.4-py3-none-any.whl.metadata
  Downloading python_statemachine-2.3.4-py3-none-any.whl.metadata (13 kB)
Downloading python_statemachine-2.3.4-py3-none-any.whl (41 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.3/41.3 kB 4.4 MB/s eta 0:00:00
Installing collected packages: python-statemachine
Successfully installed python-statemachine-2.3.4

[notice] A new release of pip is available: 23.2.1 -> 24.1.2
[notice] To update, run: pip install --upgrade pip
(aigen_playground) sebastian@sebastian-MS-7C35:~/Projects/my-multi-agent$ pip install pydot
Collecting pydot
  Obtaining dependency information for pydot from https://files.pythonhosted.org/packages/9a/fd/df3932340498a8f38c6107c95b0eb1d9ac406c5ea1307c8f43408977378e/pydot-3.0.1-py3-none-any.whl.metadata
  Downloading pydot-3.0.1-py3-none-any.whl.metadata (9.9 kB)
Collecting pyparsing>=3.0.9 (from pydot)
  Obtaining dependency information for pyparsing>=3.0.9 from https://files.pythonhosted.org/packages/9d/ea/6d76df31432a0e6fdf81681a895f009a4bb47b3c39036db3e1b528191d52/pyparsing-3.1.2-py3-none-any.whl.metadata
  Downloading pyparsing-3.1.2-py3-none-any.whl.metadata (5.1 kB)
Downloading pydot-3.0.1-py3-none-any.whl (22 kB)
Downloading pyparsing-3.1.2-py3-none-any.whl (103 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 103.2/103.2 kB 6.9 MB/s eta 0:00:00
Installing collected packages: pyparsing, pydot
Successfully installed pydot-3.0.1 pyparsing-3.1.2

[matt3o]

Nice catch, thanks @SebastianSchiefer! Same on my machine, installation yielded pydot==3.0.1, with a manual downgrade I get quoted font sizes and thus a working diagram. I am not sure, I think at first I installed pydot manually, if I remember correctly there was no warning in the text to only use python-statemachine[diagrams] for the installation.

[fgmacedo]

Manually upgrading pydot, I was able to reproduce the issue. Thanks!

Inspecting the new version of pydot, they have refactored to automatically quote only necessary attributes, and fontsize was expecting a number that doesn't need quotes. I've figured out that the default unit for fontsize is also pt (https://graphviz.org/docs/attrs/fontsize/). so no need to pass 10pt like I was doing. Changed to 10 and it worked as expected.

Working to get a PR.

[matt3o]

Sweet, thanks for the quick help!