request switch to turn off preprocessor in verible-verilog-syntax

[fpgauserdude]

Describe the request

I want to use Verible to (finally) provide reliable dependency analysis in SystemVerilog. Using the verible-verilog-syntax command, I am able to very effectively extract dependent modules, packages and includes. Using a regex for this is...problematic. Verible is 95% able to do exactly what I need. But unfortunately, there is a flaw.

I have great hopes that eventually the preprocessor will do everything it should, but right now, I can't control it. Using the verible-verilog-syntax command, I cannot control the include directories or the `defines that are meant to be configured externally. But despite not having this control, by default the preprocessor is still being run. See line 309 of "verilog_syntax.cc".

// TODO(hzeller): is there ever a situation in which we do not want
    // to use the preprocessor ?
    const verilog::VerilogPreprocess::Config preprocess_config{
        .filter_branches = true,
    };

If I cannot control the preprocessor, I would much rather have the option to not run the preprocessor at all (and use the "rawtoken") output to essentially do my own basic preprocessing. But I can't do that with the code compiled as it is. The answer to the question in the comment is a definite "yes". I was able to recompile this code setting the "filter_branches" switch to false, and the verible-verilog-syntax command does exactly what I need.

When the preprocessor is more developed I would be happy to turn that back on.

To Reproduce

// Test case (preferably reduced) that illustrates the problem.
`ifndef RSD_DISABLE_PERFORMANCE_COUNTER
   should_be_present test1
     (
      .clk (port.clk));
`else
      opposite_of_should test3
        (.clk(port.clk));
`endif
`ifdef RSD_DUMMY
   should_not_be_present test2
     (.clk (port.clk));
`endif

All the symbols are removed from the parse tree for test2 and test3 by the preprocessor before it makes it to my code. I cannot control the "RSD_DISABLE_PERFORMANCE_COUNTER" macro. Note that this change is simple enough that I will probably just add a command line switch for my own benefit. But I wanted to see if you folks were open to this change.

[pTanik]

Hi @fpgauserdude, is there any solution available to get all the data in the syntaxtree for the latest version?

[pTanik]

Hi @hzeller I see that there is rawtokens available for the latest releases, can the same be done for the syntaxtree so that the syntaxtree is not preprocessed? And I also see that the tokens are filtered with preprocessor. As the syntaxtree is more accessible with the python wrapper (verible_verilog_syntax.py). If you kindly help on this.