Extraction of netlist graph database and HDL parser/serializer modules

[Nic30]

The circuit database has to be somehow implemented in every project related to HDL/FPGA etc. This leads to ridiculous amount of work for developers of such a libraries and also makes this libraries incompatible because their internal representation of circuit is different and exporting/importing of Verilog/VHDL/FIRRTL is not an effective solution.

In fact this implies that very few of just HW optimization libraries exists.

Plan is to search for use cases https://github.com/HardwareIR/hardwareIr/wiki, https://github.com/SymbiFlow/ideas/issues/19 and who would like to use such a library.

Expected library contains:

• implementation: C++, Python/Java wrapper
• graph database for netlist with AST/IR view available
• HWT like serializers for Verilog-2005, VHDL 2002, 2008, SystemC, FIRRTL
• SystemVerilog, VHDL parserers from hdlConvertor (without python dependency) + FIRRTL parser

Similar functionality is implemented in many projects including de facto all HDL/FPGA related project.

I would gladly just extract the implementation from yosys but it is far more complicated than this and there is very strong will of many other projects to use their current solutions and simply do not make the projects compatible because the authors do not see the profit and it is quite lot of work.

For me it is hard to explain them that they should not lose years on developing of own custom half working HDL parser/generator which will die together with their project because 4K men-hours is likely the time required to develop proposed library and even if it may seems as an easy task in the beginning. Also own custom solution means incompatibility.

Also it is extremely important to finis this library in reasonable time, that means in Q2 2019, I mean I need more committers and testers.

[Nic30]

Serializer best practices:

• separated ConstCache -> userfull
• separated NameSpope -> life saving feature
• state restricted only to used names -> better code reuse

Problems:

• Serialization may alter the circuit: for example it can add tmp variable because target language requires it.
  • This is not expected behaviour because it breaks incremental serialization while debugging, serialization to multiple target languages etc.
  • It is required to copy only part of the circuit
    • How to represent the replacement of part of the circuit?
    • How to represent how to alter call stack of serialization to use new circuit?
    • How to use NameScope for the two sets of objects?
  • Is it possible to perform prior check and then optionally perform copy of whole circuit if required and serialize it?

[Nic30]

First step is to move all VHDL/Verilog/.... generating logic to a independent library. https://github.com/Nic30/hdlConvertor/tree/master/hdlConvertor/hdlAst https://github.com/Nic30/hdlConvertor/blob/master/hdlConvertor/toVhdl.py https://github.com/Nic30/hdlConvertor/blob/master/hdlConvertor/toVerilog.py should be also extracted.

The goal is to create library which contains simple containers for HDL objects and functions which can convert this object to a desired HDL.

HWT currently does have such a objects in https://github.com/Nic30/hwt/tree/master/hwt/hdl, however they are polluting this library and causing code duplication between the projects as the functionality is required elsewhere, but there is unnecessary dependency on HWT.

[Nic30]

Functionality extracted to https://github.com/Nic30/hdlConvertor/tree/verilog_pp_line_directive now it is required to split hdlConvertor to python/C++ part so we can use it without C++ dep. from this lib.

[Nic30]

done in v3.1