Improve VHDL parser

[LarsAsplund]

This is an issue to collect various issues related to the current regexp-based VHDL parser which can be solved by introducing a more proper parser

• See #144
• See #74

[joshrsmith]

I will say that this is a pretty low priority for me. The VHDL parser today is very good, and I have encountered very few "code style" issues that cause it to get confused.

[Paebbels]

I started a Python based streaming-oriented VHDL parser: pyVHDLParser. I'm open for contributors :)

A first prototype to evaluate the concept is already online.

[cjlockett]

I have been using VUnit to establish the compile order and determine the required source files for a particular top level. However, it is failing to detect one of the file dependencies, which is a package. Looking at the VHDL parser, its seems that the package is not being recognised as a design unit because it does not end with the usual ‘end package’ statement. The package is declared using the construct: package my_package_name is new existing_package_name generic map (… );

Could the parser be updated to allow this type of package declaration?

[gsorrenti]

Sorry for the OT reply. I'm also interested in using Vunit also to generate a list of source file for a particular top, but I failed because he doesn't propagate dependency through entity instantiated by component declaration. How are you managing this problem?

I'm not a fan of component declaration but it's not acceptable to unsupported this coding style.

At this time my orrible workarounds was to assume as dependent all the file in the top level working library.

Thanks for any help.

Il gio 16 feb 2017 17:13 cjlockett notifications@github.com ha scritto:

I have been using VUnit to establish the compile order and determine the required source files for a particular top level. However, it is failing to detect one of the file dependencies, which is a package. Looking at the VHDL parser, its seems that the package is not being recognised as a design unit because it does not end with the usual ‘end package’ statement. The package is declared using the construct: package my_package_name is new existing_package_name generic map (… );

Could the parser be updated to allow this type of package declaration?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/VUnit/vunit/issues/174#issuecomment-280376538, or mute the thread https://github.com/notifications/unsubscribe-auth/AUXvvrMqAUOBr5py96EIoavRM2Oqg2inks5rdHWwgaJpZM4IyusF .

[kraigher]

@gsorrenti Yes VUnit supports compile order detection on component instantiation (There is none). You have to first understand that the problem to determine compile order dependency and the problem of determining a subset of files required to build a top level are slightly different problems. VUnit focuses only on the first.

This difference is clear when it comes to component instantiation. There is no need to compile the entity intended to map to the component before compiling the architecture instantiating the component. When the entity intended to map to the component is changed there is no need to recompile the architecture instantiating it and vice versa. Thus there is no compile order dependency between the two.

The only requirement is that both files have been compiled before starting elaboration which is when component binding occurs. VUnit will always compile all files before starting simulation and thus ensures this requirement is met. VUnit thus does not need to know the component binding to compile files in the correct order or to recompile the correct affected files. VUnit only needs to care if one file needs to be compiled before another file, never what subset of files are required to successfully elaborate a top level.

Thus I am saying that you are trying to use VUnit to solve a problem that is does not need to solve to perform its officially supported function. That is not to say that it is not possible to extend VUnit to support other use cases, a user implemented component dependency detection based on a simple heuristic in one of our internal classes: https://github.com/VUnit/vunit/pull/30. This is not something that we publicly support though.

I can say though that we try to avoid creating the need to do a full VHDL parser with semantic analysis (https://en.wikipedia.org/wiki/Semantic_analysis_(compilers)) since it is a lot of work. More work than the rest of VUnit combined. Precise component binding analysis requires semantic analysis such as the knowledge of all data types in the scope of the instantiation to select the match. People might theoretically define their own std_logic type in their own package, simple regex wont do the job.

[gsorrenti]

@kraigher I already had your same opinion, I understand this feature is out of topic of Vunit. Indeed this feature is a must-have for script toolchain because it makes no sens to free my self from the boring and error-prone work to maintein compile script for verification while I need to do the same work on a subset of the same source file for implementation.

I want to share my experience with @cjlockett because maybe he is tryng to do something I already did. My current solution is not elegant and it had overload to sythesis tool, but it's works.

And I am also interested to know if any body else solved the same need is some other way.

[kraigher]

@cjlockett can you provide more complete example code to illustrate your use case?

[gsorrenti]

I just read #30 and now I'm a bit confused. Is this feature already implented? I can get the list of dependency of an entity including components? Do it also support this feature across mixed language VHDL-Verilog? Thank you

[Paebbels]

@gsorrenti We solve the compile order issue with one *.files file per IP cores, testbench or design. We can recursivly include other *.files files. Additionally, these files support conditionals so you can switch source files based on the VHDL version number, the used tool chain or if you need files for synthesis or simulation.

The parser is written in Python 3 and part of PoC. If this is in your interest, please contact me.

[kraigher]

@gsorrenti It is implemented as an optional setting on a private class that is not reachable through the public interface. Maybe we can expose it on the public interface as an alternative method to the get_compiler_order method, maybe it can be named get_implementation_subset(source_files).

[kraigher]

At work I hardcode what folders to add to the Vivado project not individual files. Each folder corresponds to a VHDL library. A folder contains either a reusable subsystem with a single top level or a set of related reusable utility modules. The test subfolder in each folder containing test benches and verification code is never added. Only when adding folders with many related modules do I run the risk of adding unused files. However Vivado has a TCL command to remove unused files from the project so I always run that after adding the hardcoded folder in the build script. This works ok for me.

[gsorrenti]

Sounds good for me! I think this could open a new road to the creation of, let me say, implementation plugin for Vunit for different vendors. I have to check with my boss but maybe I can share the simple one I developed for Lattice.

I also understand that exposing a new function adds work to you to be maintained.

If you prefer I can start using it as is. I cannot figure how because I'm not an expert in python.

[cjlockett]

My code example: scoreboard_cabs_pkg.vhd:

library osvvm; context osvvm.osvvmContext;

library dsp_lib; use dsp_lib.cabs_types_pkg.all; -- contains definitions of types used in generic map

package scoreboard_cabs_pkg is new osvvm.ScoreboardGenericPkg generic map ( ExpectedType => cabs_test_vector_type, ActualType => cabs_test_result_type, Match => cabs_check_result, expected_to_string => cabs_test_vector_to_string, actual_to_string => cabs_test_result_to_string);

tb_cabs.vhd: library dsp_lib; use dsp_lib.scoreboard_cabs_pkg.all; entity tb_cabs is generic (runner_cfg : string := ""); end entity; architecture tb of tb_cabs is … end architecture; Running VUnit: WARNING - …\dsp_lib\tb_cabs.vhd: failed to find a primary design unit 'scoreboard_cabs_pkg' in library 'dsp_lib' And: COMP96 ERROR COMP96_0055: "Cannot find referenced context element "dsp_lib.scoreboard_cabs_pkg"." "…\dsp_lib\tb_cabs.vhd"

[kraigher]

@cjlockett thanks that helps us solve your problem

[kraigher]

@cjlockett in the meanwhile I hope you know we have a way to add manual dependencies between files

[kraigher]

@gsorrenti For implementation dependencies there is also things like synthesis translate off/on pragmas to consider which people might add around instantiations of verification modules added inside a block during simulation. There is maybe some overlap between the dependency scanning VUnit has but as you see there are may differences.

To be honest the way I see it there is no major benefit to have implementation plug-ins in VUnit since the overlap with that functionality and what VUnit does is so small, much smaller than the non overlapping parts. An implementation tool needs to handle constraint files. ip-files, block designs, tool settings, part settings etc. It is better to view them as fully separate tools which only share some common list of files and libraries which is only the small subset of information that is relevant to both tools.

At work my implementation automation scripts shares no code with VUnit, I feed them both the same list of hard-coded re-usable folders for each top level project. The list of folders does not become long and is not hard to maintain. Since each folder has a fixed structure for simulation and synthesis files I never have to specify specific files.

Based on this I think that implementation issues might be better to be considered out of scope of VUnit.

[LarsAsplund]

@cjlockett @kraigher I have code locally to handle package instantiations but I haven't pushed this since my use case of verifying the IEEE packages for the upcoming VHDL-2017 runs in to another problem. They have local package instantiations with the same name as global packages and that caused other problems. This has to be revisited. Meanwhile I use the manual dependencies @kraigher mentioned.

[kraigher]

@LarsAsplund Could you post a minimal example of this here? This should really be a new issue.

[LarsAsplund]

@kraigher That I can do. Sitting on a train now so I'll do that tomorrow.

[kraigher]

@gsorrenti I am sure lattice users would benefit from sharing open source build automation tools such as yours. Probably lots of people reinvented the same thing in private. We created VUnit to stop people reinventing the simulation automation.

[gsorrenti]

Anyway @kraigher , if possible I really appreciate the creation of a get_implementation_subset public method. Thank you also, @Paebbels, at this time I manage well my toolchain with dependencies graph generated by Vunit. Should be notice that the compile order is not a need, the only real need is the minimun complete set of dependecies of an entity.

[cjlockett]

@kraigher @LarsAsplund Thanks for looking into my issue. I’d like to avoid adding manual dependencies if possible. I am creating a temporary VUnit project, searching my repository and adding all source files to their appropriate libraries. Then getting the compile order for the top level entity (identified from the test case, variable between runs as my code base covers several units) and creating a new VUnit project and adding only the necessary files rather than everything from my file search. That way I only compile the necessary files for a particular test. This works just fine except for identifying ‘new’ packages, as described in my first post. I got round it by adding a condition in the VHDL parser, and posted here so that you could add it to your code base for all users, or in case you have a preferable solution.

[kraigher]

@gsorrenti I will add the get_implementation_subset method but make it clear in the documentation that it is a best effort feature where we will not necessarily prioritize lack of functionality and bugs in it as much as the normal features of VUnit.

[gsorrenti]

@kraigher Thank you. It will be usefull.

[kraigher]

@gsorrenti I added the get_implementation_subset feature on master now. it also adds package bodies as dependencies if the simulator does not already require it. This was never a problem unless you separate the package header and body in different files.

[gsorrenti]

@kraigher Thank you, I hope I can test and add this function to my toolchain this week.