Generate a schematic starting from dump produced by GHDL (XML or others)

[pidgeon777]

Hi everyone, I would like to write a small tool which, given a VHDL file describing an entity plus its architecture, would allow me to draw its corresponding schematic. It should at least include:

1. Entity input/output ports as pins.
2. Boxes representing the instanced sub-components.
3. Boxes representing the internal processes.
4. Boxes representing the concurrent assignments (thus outside processes).
5. Signal paths between 1, 2, 3 and 4.

Apparently, this would seem something not too much hard to do. It would involve:

1) Identify which source should be used for the initial parsing. XML generated with --file-to-xml? The VHDL file itself? Some tags-based file? Etc. 2) Identify the fastest, easiest and most optimized way to parse the obtained dump. Some specific language/library to parse the XML? Etc. 3) Translate the parsed structures (entities, ports, signals etc.) into the final representation format. Dot? ELK? Etc.

So, I would like to know your opinion about this idea. For example:

• Is someone already trying to do something similar to this?
• Would the GHDL generated XML be the perfect starting point, or something better could be done?
• What could be the best and fastest way to parse the XML file?

I thought about the XML dump, but maybe better solutions exist. Maybe the netlist obtained by GHDL synthesis? My dream one day would be to parse the VHDL file and obtain a schematic using some advanced routing algorithms which would automatically produce the layout.

This kind of algorithms already exist, here are some great examples using ELK, one of the best automatic layout and routing algorithm currently available:

• Execution time comparison to Java ELK
• Schematics
  • netlistsvg - electronic and gate level schematics as SVG
  • d3-hwschematic - interactive digital circuit analysis and documentation, particularly for FPGA-based designs
• Visualizing npm dependencies
• How to specify node label placement
• How to enforce certain node positioning (topological)
• Wrapping elkjs into a docker-based microservice
• Cytoscape adapter
• reaflow - Node-based editor built on elkjs in React

[pidgeon777]

The generated schematic could be something like this:

• Main entity ports are placed externally as physical pins.
• Sub-components are displayed as boxes with their input/output ports.
• Processes are represented as boxes. Inside a process, a signal could be read or written. Thus, inputs of these processes will be associated with the signals read, outputs will be associated with the signals written inside the process.
• Each concurrent assignment (assignments outside a process) will be represented as a box. Each of these will generally involve a signal written, and one or more signals read.

Thus, for example, if Process 1 write a signal that will be read by COMP2, a wire will connect one of the Process 1 outputs (the one associated to the signal) to COMP2 input (the port which has been mapped to that input signal).

As a result, the resulting schematic would greatly improve the readability of code in terms of:

• Entity ports
• Instantiated components
• Declared processes
• Concurrent statements

also helping to visually debug some problems, such as:

• Inputs driven by more than one signal.
• Outputs not connected.
• Wrong signal connections (clocks, etc.).
• Signals written by Process X at a certain clock frequency, read in Process Y working at a different clock frequency (possible metastability, draw a red line from Process X associated signal output to Process Y input).

and so on.

[tgingold]

That's indeed a very interesting and useful tool.

Starting from the synthesized netlist would be much easier, but you won't be able to display process or concurrent assignments.

If you start from analyzed entity/architecture, you have to decide what to do with generate statements, and vector signals whose length is not known.

So I suppose you'd like to start from an elaborated architecture. But then you have to decide how to give the value of the generics.

Unfortunately, there is currently no easy way with GHDL to get a full access to an elaborated design. The API for that (VHPI) is not fully enough implemented.

So maybe you should start from analyzed designs and deal with generate laters.

Using the xml file is not recommended. It's huge and not stable. I would start with libghdl (a thin python interface to the internal of ghdl).

[Nic30]

I am using grouping of the "gates by process" in hwtGraph and the examples in d3-hwschematic are generated from it. I do have positive feedback from colleagues and it seems to me as a must have. However nearly no other person is using that library.

example

I have never found a clear answer about what should be elaborated for a visualization and what should have a custom element. I remember a discussion with @drom and @xkotek07 . Drom suggested to start with the specification of shapes and behaviour of the gui.

[pidgeon777]

@tgingold

Starting from the synthesized netlist would be much easier, but you won't be able to display process or concurrent assignments.

I didn't follow the latest GHDL developments, so now it is possible to obtain a complete synthesized netlist of the input source? If yes, then would it be currently possible to generate a schematic from that? With Yosis something like that was already possible.

It would be an interesting thing to do. But first, I would like to try a higher-level approach, for making the code easily readable during its development. Thus, visual high-level representation of instanced components, process, ports etc. to find the possible source of bugs I described above, or for quickly understand the involved subcomponents and their connections.

If you start from analyzed entity/architecture, you have to decide what to do with generate statements, and vector signals whose length is not known. So I suppose you'd like to start from an elaborated architecture. But then you have to decide how to give the value of the generics.

You're right. For the vector signals I could start with declaring them as buses of unknown size on that case. For the generate statements, it would obviously be harder.

The generate statements produce their effects on source compilation/elaboration (thinking for example about generics), so we could still instantiate all of the code included in the generate statement for example by putting them inside a rectangle with dashed edges as if to say, all of this could be instantiated or not after compilation. But at least someone would know those might actually be there.

Unfortunately, there is currently no easy way with GHDL to get a full access to an elaborated design. The API for that (VHPI) is not fully enough implemented.

Well, that would be useful. Let's hope, one day.

So maybe you should start from analyzed designs and deal with generate laters.

That was the idea, for now. Instantiate everything in the code eventually detecting those generates and placing the parsed elements accordingly (boxes with dashed edges etc.).

Using the xml file is not recommended. It's huge and not stable. I would start with libghdl (a thin python interface to the internal of ghdl).

Thanks for the tip, I'll look for the documentation of that interface.

I would like to report this HDL specific schematic diagram tool, which also seems to be very good for the kind of schematic I had in mind:

https://davidthings.github.io/hdelk/

https://github.com/davidthings/hdelk

Some examples:

https://davidthings.github.io/spokefpga/pipelines

https://davidthings.github.io/spokefpga/i2c

[pidgeon777]

@Nic30

I am using grouping of the "gates by process" in hwtGraph and the examples in d3-hwschematic are generated from it. I do have positive feedback from colleagues and it seems to me as a must have. However nearly no other person is using that library. example

That seems to be a great project, congratulations. I wanted to do something very similar to your schematic, but for VHDL. I see that you're representing a language somehow different from VHDL.

Considering your example, when I expand those "processes", I can see their equivalent RTL representation? Correct? If yes, that is very cool.

Well, in the schematic that I had in mind I would be happy enough to expand the "process box" and simply see the code of the actual process, for example. Still, this "process box" should have as inputs all of the signals read by the process and the written signals as outputs. Exactly as you already did, except for that great RTL preview.

But, one day using the power of GHDL synthesis, maybe it could be possible to expand this box and actually show the GHDL synthesized RTL schematic, as you already did with your custom library.

It seems that your diagramming library could be a good candidate to display the resulting GHDL schematic, though. I like the fact that those rectangles can be expanded, maybe in a hierarchical way.

Anyway, I use Vivado and I love its RTL schematic viewer. Dynamically showing and hiding signal paths, and expanding subcomponents and so on. It would be good to obtain something similar for basic VHDL sources, without following all of the operations needed by Vivado, which is not a very light tool. Something fast to quickly display the HDL code would be nice to have, in my opinion.

[Nic30]

when I expand those "processes", I can see their equivalent RTL representation?

yes, it should expand on-click . It creates a complex problem however. As parts of an expressions can be shared with multiple processes we need to take an extra care while moving gates in to "process box".

[pidgeon777]

This is not strictly related to GHDL, but it could be of interest for many of you. I'm currently helping @masatake with completing the CTAGS-based VHDL parser. Something which could facilitate the schematic project I previously discussed with you:

https://github.com/universal-ctags/ctags/issues/2678

Anyway, those with a good knowledge of VHDL have now a great opportunity to participate in this discussion and help us with improving this CTAGS VHDL parser (I'm not a Universal Ctags developer, just someone using it extensively).

Some of the discussion points which are near to bo solved:

• How to classify ports/generics.
• How to identify instanced components inside an architecture.
• How to define the scope of each instanced element? Ports, generics, components, packages, etc.

All of this could be very useful for quickly parsing multiple architectures, for example, for fast code navigation, or to build hierarchical component trees, and so on.

Those willing to share some opinions or do some tests are more than welcome.

[Nic30]

Is ctags a real parser/parser generator or is it just source code indexer?

[pidgeon777]

Ctags is a programming tool that generates an index (or tag) file of names found in source and header files of various programming languages to aid code comprehension. Depending on the language, functions, variables, class members, macros and so on may be indexed. These tags allow definitions to be quickly and easily located by a text editor, a code search engine, or other utility. Alternatively, there is also an output mode that generates a cross-reference file, listing information about various names found in a set of language files in human-readable form.

Thus, it mainly used to index a source code, but theoretically, it could perform a complete language parsing. At the current form, it is still very useful to index elements such as:

• Entities
• Ports
• Generics
• Constants
• Types
• Signals
• Variables
• Components
• Labels
• Packages
• Processes
• Architecture

almost every VHDL element.

It provides a list of tags where those keywords are defined, one for each row, with the corresponding line number of the source file where they are defined. But the most useful aspect is the scope. For example, given two identically named ports of two distinct entities, thanks to the scope I could differentiate those port when parsing the code, correctly associating them to the proper entity.

CTAGS is used in many editors: Eclipse, Visual Studio Code, Vim...

For more info:

http://docs.ctags.io/en/latest/

https://buildmedia.readthedocs.org/media/pdf/ctags/latest/ctags.pdf

I was thinking about helping out CTAGS developers with adding some features so that processes and especially component instances could be parsed, useful for example to build a tool which could generate a tree of the component dependencies. Starting from that, a compile script could be generated, with the source files properly sorted. The resulting tree could be also used as a starting point to produce a diagram showing the relationship between those components.

[umarcor]

Hi @pidgeon777! It's nice to see interest in this topic!

Since purely technical discussions about graphical libraries, routing algorithms, etc. have been long discussed, I will skip that.

I think that the main point is that the abstraction level you are trying to achieve does not exist. Tristan and Nic already pointed it, but I think they didn't do it explicit enough. Don't take me wrong. I do think that's the only usable objective for us, hardware designers writing VHDL code. We want to see a representation of the sources we are describing, and we want them to update in "real-time" as we create proceses or add concurrent statements. It's a nice dream that Nic, me and many others had before, but which is very difficult to obtain from HDL sources. That's why most tools are HDL generators (labview, simulink, icestudio...) or represent HDL as manually created black boxes, but don't try the other way. I've seen periodically asking Tristan about this for several years now. It was really challenging to explain him my purpose initially, because it does make absolutely no sense from a code perspective. It's only useful for humans. And it's the kind of usefulness that feels so intuitively stupid for humans but is awfully difficult for machines.

Overall, you can obtain the input/output ports of entities, the boxes for sub-components, blocks, processes, etc. If there are multiple concurrent statements and you don't want to parse all the details, you can create a "virtual" box for those in the diagram and represent the content in text. All of that is relatively easy, as long as you have a good parser and abstract model (see pyVHDLModel). However, and this is a huge however, there is no tool for providing your 5th point: "Signal paths between 1, 2, 3 and 4".

Simulators do not need to know about that. All concurrent statements are implicitly or explicitly converted to sequential processes (there is an LRM compliant sequential equivalent for each concurrent VHDL statement), and all signals are just variables (which typically contain structs). There is no notion of "a connection" or "a wire" per se. There is an execution model that ensures the scheduling of all the processes. And many things are resolved at runtime.

On the other hand, synthesizers do know about all the connections, because that is their main purpose (building the whole circuit/netlist), but they don't care about blocks, processes or statements (not to mention generates, which you already talked about). Synthesizers produce RTL netlists, which are typically lower level than the HDL descriptions and it's expanded (unless it's a very structural and close-to-the-hardware style of HDL description).

As a result, you need a simulator which does something more than a regular simulator; or a synthesizer that does slightly less than a regular synthesizer. In some sense, you need half of the elaboration for one thing, and half for the other. Therefore, tools do not provide that type of elaboration, unless the purpose is generation of diagrams explicitly. You might find that some vendor synthesizers have several settings for preserving the hierarchy after synthesis. That can help visualisation, but the representation is still RTL.

Fortunately, GHDL is a weird and brilliant tool. It allows simulation and synthesis. Moreover, it allows generation of unoptimised netlists (ghdl --synth) or optimised results (ghdl-yosys-plugin). Incidentally, the output of ghdl --synth is probably the closest we can get to a non-RTL representation that is actually drawable. It does try to preserve the naming and it does elaborate and transform the sources, but it should do many less transformations than any other synthesis solution. Moreover, the output is VHDL, so GHDL can be used as a parser for GHDL's synth output. That is, pyGHDL can be potentially used for running synthesis and for getting a pyVHDLModel representation both before and after synthesis, where entities and signal names should match; and there should be a predictable pattern for expansion of generics (which pyGHDL can be aware of due to the shared sources with the synthesiser). You can use the first model for generating blocks and ports matching the original sources, and you can extract connection information from the second model. By the same token, you can identify FSMs in the first model, and hide them for the synthesised result.

There is umarcor/hwstudio, where I'd like to prototype these ideas. There is a currently useless web version, which provides a visual demo umarcor.github.io/hwstudio. There are also many thoughts and references gathered in https://umarcor.github.io/hwstudio/doc/. Godot, the 2D/3D game engine used there, is written in C++. However, it has a GDScript language similar to Python. It is also possible to bind any language. Therefore, my strategy is to use pyGHDL for achieving points 1-4 of your list. I will forget about having the connections drawn automatically for now. The first usable target is the following: let users "import" a directory of dozens of VHDL cores and provide an interface for manually adding them and drawing connections (preferredly using VHDL 2019 interfaces). Then generate the top-level HDL source for multiple boards. This is similar to Vivado's Block Design, but without requiring each core to be IP-XACT, just pick the blocks from VHDL entities.

Some months ago, I did a successful test with libghdl for getting the entity, ports, names, modes and types (std_logic or std_logic_vector) from a VHDL source: #1449. However, GHDL's Python interface was completely reworked very few weeks ago, and that example needs to be done again with pyVHDLModel. Hence, I consider the GUI/visualisation part of the project to be negligible for now. I think the priority is to make pyGHDL.dom usable for getting a JSON/YAML representation of your points 1-4 (both before and after synthesis).

EDIT

In case you didn't see it: #111.

[nobodywasishere]

@pidgeon777 While not 100% what you're looking for, it is possible to use GHDL / ghdl-yosys-plugin / Yosys / netlistsvg to generate a block diagram of VHDL. It's as simple as:

ghdl -a --std=08 top.vhdl
yosys -p "ghdl --std=08 top; prep -top top; write_json -compat-int svg.json"
netlistsvg svg.json -o top.svg

[svenn71]

@pidgeon777 While not 100% what you're looking for, it is possible to use GHDL / ghdl-yosys-plugin / Yosys / netlistsvg to generate a block diagram of VHDL. It's as simple as: yosys -p "ghdl --std=08 top; prep -top top; write_json -compat-int svg.json"

I had to add -m ghdl otherwise yosys did not know the command 'ghdl'

[svenn71]

Regarding CTAGS: I have used this for some years now : https://github.com/PAntoine/VTAGS

[umarcor]

@svenn71 using -m ghdl when calling Yosys depends on how was ghdl-yosys-plugin built. Either embedded in Yosys, or as a module (shared lib).

[umarcor]

FTR, https://github.com/SymbiFlow/sphinxcontrib-hdl-diagrams

[umarcor]

FTR, there is work in progress in https://umarcor.github.io/osvb/apis/project.html for extracting the hierarchy of a design using pyGHDL.dom. See Open Source VHDL Design Explorer (OSVDE).