Add a Verilog / SystemVerilog support to Sphinx

[mithro]

Add a Verilog / SystemVerilog support to Sphinx

Brief explanation

We use Sphinx for documentation heavily. Sphinx currently doesn't support Verilog or SystemVerilog natively. It would be good if it did.

Expected results

Sphinx has native support for Verilog and SystemVerilog.

Detailed Explanation

While Sphinx was originally created for Python, it does supports the concept of "domains". Domains allow Sphinx to support multiple languages.

It might be useful to use the Python module hdlparse which is used by Symbolator to generate nice diagrams. Symbolator already has some Sphinx integration;

.. symbolator::

  component foo is
    ...
  end component;

The current planned process for getting Verilog into Sphinx is the following;

1. Use doxygen-verilog to generate Doxygen output from the Verilog
2. Use breathe and/or exhale to read the Doxygen output into Sphinx.

[Dhiraj240]

@mithro I am interested in this project for GSOC 2019. I am third year undergraduate student doing Bachelor of engineering in Electronics and Telecommunication in India and have good grasp over python.I have been contributing to opensource a lot.Please see my github profile https://github.com/Dhiraj240 and also in summers 2018 i had participated in an opensource contest where i built this project https://github.com/Dhiraj240/GitBot-v1 and you can checkout my blog on my github profile page.

While communicating on irc i got to know that doxygen-verilog will use symbiflow-arch-defs to generate Doxygen output from the Verilog and i have to use either breathe or exhale to read it and output it to sphinx. But i was curious as what type of verilog code i have to write? can i get an examples ? is verilog code not present in https://github.com/SymbiFlow/symbiflow-arch-defs

GitHub

Dhiraj240 - Overview

Coder || Microsoft Student Partner || Python Developer @wtfpython-web || Core Developer, Google Code-in Mentor @coala || JS Developer and Mentor @publiclab - Dhiraj240

GitHub

Dhiraj240/GitBot-v1

Slack bot for github needs more feature , originally developed during opensource contribution.People looking to contribute(or need help) directly comment on issue. - Dhiraj240/GitBot-v1

GitHub

SymbiFlow/symbiflow-arch-defs

FOSS architecture definitions of FPGA hardware useful for doing PnR device generation. - SymbiFlow/symbiflow-arch-defs

[mithro]

@Dhiraj240 This aim of this project is about adding the Verilog language support directly to the Sphinx project.

Sphinx calls these domains.

The currently supported domains in Spinx are Python, C, C++ and Javascript (plus special reStructuredText and Math). There is also a sphinx-contrib repository which adds support for Ada, CoffeeScript, Erlang, HTTP, Lasso, MATLAB, PHP, and Ruby domains. There are also available are domains for Chapel, Common Lisp, dqn, Go, Jinja, Operation, and Scala.)

The idea would be to add Verilog to that list.

I suggest looking at existing projects which parse Verilog, these include;

• PyVerilog
• verilogparser
• hdlparse

This would then allow us to write Verilog code like the following;

//% @file xor2.v
//% @brief xor with two inputs

//% Xor Module brief description
//% Detailed description of this module 

module XOR2 ( OUT, I0, I1 );
    input I0; //% first input
    input I1; //% second input

    //% output
    output OUT;

    assign OUT = I0 ^ I1;
endmodule

And it would be converted into documentation.

[Dhiraj240]

@mithro Alright sir, i got the point as what all is expected but do you have list of verilog codes which has to be parsed.Basically where does the verilog code comes from, should i work randomly based on my knowledge like implementation of full adder, half adder etc. And then create the documentation in sphinx.

[Dhiraj240]

Or https://github.com/SymbiFlow/vtr-verilog-to-routing https://github.com/SymbiFlow/symbiflow-arch-defs

Both the above projects are the source for the verilog codes and i have to create sphinx documentation for all the .v extension verilog codes. is there any section in above repositories which hold the greater priority.

GitHub

SymbiFlow/vtr-verilog-to-routing

SymbiFlow WIP changes for Verilog to Routing -- Open Source CAD Flow for FPGA Research - SymbiFlow/vtr-verilog-to-routing

GitHub

SymbiFlow/symbiflow-arch-defs

FOSS architecture definitions of FPGA hardware useful for doing PnR device generation. - SymbiFlow/symbiflow-arch-defs

[mithro]

@Dhiraj240 This project is not about writing documentation for existing Verilog code, it is about creating the tools needed for process the documentation.

[Dhiraj240]

Sorry for the mistake i meant the same to process the documentation.

[mithro]

The primary user of the sphinx documentation would be the https://github.com/SymbiFlow/symbiflow-arch-defs repository.

You would be expected to write a whole bunch of examples as part of the project.

You will also need strong Python skills.

GitHub

SymbiFlow/symbiflow-arch-defs

FOSS architecture definitions of FPGA hardware useful for doing PnR device generation. - SymbiFlow/symbiflow-arch-defs

[Dhiraj240]

Oh alright sir. I will start working on proposal then and hopefully by 29 March i will try to share my draft.I will put my best efforts in python to get the positive outcome. But before that do i need any contribution in SymbiFlow or should i focus on proposal ? Also i am confident and does have strong skills n python. Should i provide resume ?

[mithro]

@Dhiraj240 You should aim to demonstrate your skills with Python and understanding of what is needed for this task. This should include your proposal but also examples of working with Python and understanding how sphinx works.

[Dhiraj240]

Alright sir, just count me in for GSOC 2019. I will try to implement a sample of it in my GSOC proposal. Thank You.

[Dhiraj240]

@mithro is this the repo https://github.com/SymbiFlow/symbiflow-docs which needs this extension to make this tool. Or a new repositry should be created in Symbiflow which first parse the verilog code, can't we fork one of the existing opensource projects which is used for parsing verilog code and then port it to sphinx. Need some clear view.

GitHub

SymbiFlow/symbiflow-docs

Documentation for SymbiFlow. Contribute to SymbiFlow/symbiflow-docs development by creating an account on GitHub.

[mithro]

You are adding a plugin to sphinx. Please look at how the other new languages are added via the sphinx-contrib repository.

[Dhiraj240]

Okay, first i parse the verilog code

module top
  (
   input CLK, 
   input RST,
   input enable,
   input [31:0] value,
   output [7:0] led
  );
  reg [31:0] count;
  reg [7:0] state;
  assign led = count[23:16];
  always @(posedge CLK) begin
    if(RST) begin
      count <= 0;
      state <= 0;
    end else begin
      if(state == 0) begin
        if(enable) state <= 1;
      end else if(state == 1) begin
        state <= 2;
      end else if(state == 2) begin
        count <= count + value;
        state <= 0;
      end
    end
  end
endmodule

which will generate

Source:  (at 1)
  Description:  (at 1)
    ModuleDef: top (at 1)
      Paramlist:  (at 0)
      Portlist:  (at 2)
        Ioport:  (at 3)
          Input: CLK, False (at 3)
        Ioport:  (at 4)
          Input: RST, False (at 4)
        Ioport:  (at 5)
          Input: enable, False (at 5)
        Ioport:  (at 6)
          Input: value, False (at 6)
            Width:  (at 6)
              IntConst: 31 (at 6)
              IntConst: 0 (at 6)
        Ioport:  (at 7)
          Output: led, False (at 7)
            Width:  (at 7)
              IntConst: 7 (at 7)
              IntConst: 0 (at 7)
      Decl:  (at 9)
        Reg: count, False (at 9)
          Width:  (at 9)
            IntConst: 31 (at 9)
            IntConst: 0 (at 9)
      Decl:  (at 10)
        Reg: state, False (at 10)
          Width:  (at 10)
            IntConst: 7 (at 10)
            IntConst: 0 (at 10)
      Assign:  (at 11)
        Lvalue:  (at 11)
          Identifier: led (at 11)
        Rvalue:  (at 11)
          Partselect:  (at 11)
            Identifier: count (at 11)
            IntConst: 23 (at 11)
            IntConst: 16 (at 11)
      Always:  (at 12)
        SensList:  (at 12)
          Sens: posedge (at 12)
            Identifier: CLK (at 12)
        Block: None (at 12)
          IfStatement:  (at 13)
            Identifier: RST (at 13)
            Block: None (at 13)
              NonblockingSubstitution:  (at 14)
                Lvalue:  (at 14)
                  Identifier: count (at 14)
                Rvalue:  (at 14)
                  IntConst: 0 (at 14)
              NonblockingSubstitution:  (at 15)
                Lvalue:  (at 15)
                  Identifier: state (at 15)
                Rvalue:  (at 15)
                  IntConst: 0 (at 15)
            Block: None (at 16)
              IfStatement:  (at 17)
                Eq:  (at 17)
                  Identifier: state (at 17)
                  IntConst: 0 (at 17)
                Block: None (at 17)
                  IfStatement:  (at 18)
                    Identifier: enable (at 18)
                    NonblockingSubstitution:  (at 18)
                      Lvalue:  (at 18)
                        Identifier: state (at 18)
                      Rvalue:  (at 18)
                        IntConst: 1 (at 18)
                IfStatement:  (at 19)
                  Eq:  (at 19)
                    Identifier: state (at 19)
                    IntConst: 1 (at 19)
                  Block: None (at 19)
                    NonblockingSubstitution:  (at 20)
                      Lvalue:  (at 20)
                        Identifier: state (at 20)
                      Rvalue:  (at 20)
                        IntConst: 2 (at 20)
                  IfStatement:  (at 21)
                    Eq:  (at 21)
                      Identifier: state (at 21)
                      IntConst: 2 (at 21)
                    Block: None (at 21)
                      NonblockingSubstitution:  (at 22)
                        Lvalue:  (at 22)
                          Identifier: count (at 22)
                        Rvalue:  (at 22)
                          Plus:  (at 22)
                            Identifier: count (at 22)
                            Identifier: value (at 22)
                      NonblockingSubstitution:  (at 23)
                        Lvalue:  (at 23)
                          Identifier: state (at 23)
                        Rvalue:  (at 23)
                          IntConst: 0 (at 23)

and then after this.

[mithro]

Your verilog code is missing the special comments which have the documentation in them.

Have you used doxygen or javadoc before? If not, please read those wikipedia pages.

Once you are familiar with the concept of documentation generation tools, take a look at some Sphinx + Python examples. For example, look at https://pythonhosted.org/an_example_pypi_project/sphinx.html#full-code-example and https://sphinxcontrib-napoleon.readthedocs.io/en/latest/#getting-started

A good example of a project which uses documentation generation heavily is the numpy project. Take a look at https://numpydoc.readthedocs.io/en/latest/format.html#sections

The idea is that instead of parsing Python to get the API documentation you would be parsing Verilog.

Documenting Your Project Using Sphinx — an_example_pypi_project v0.0.5 documentation

Napoleon - Marching toward legible docstrings — napoleon 0.7 documentation

numpydoc docstring guide — numpydoc v0.9.dev0 Manual

[Dhiraj240]

@mithro Sorry but I was unable to submit it. But I really want to work on this project in the future. Do you apply in any other program or can sponsor a student with less stipend, I desperately want to work on it?

[Dhiraj240]

@mithro ^^^

[mpkopec]

@mithro Are you still up for the task of integrating Verilog/SystemVerilog into Sphinx? I have tested almost all the tools mentioned in the discussion but for now, the least bad solution I have tested is the doxygen-verilog. Which is bad basically because it is not maintained. The rest of the solutions are either incomplete or have bugs that make it impossible to parse verilog code from my project. I would be more than happy to reopen the discussion and maybe help in the development.

[mithro]

@mpkopec Yes, I would still love to see it happening as we are using Sphinx heavily for hardware related projects which use Verilog.

(Hence we also developing http://sphinxcontrib-verilog-diagrams.rtfd.io/)

Sphinx Verilog Diagrams — Sphinx Verilog Diagrams 0.0-86-g8c16b8f documentation