Incorrect line numbers with some preprocessor directives

[atyrberg]

Some preprocessor directives seem to give wrong line numbers.

`default_nettype none
module Mod();
endmodule

The default_nettype get the correct line number, but everything after have line number offset by 1.

SourceText
   DefaultNettypeValue
    Keyword
     Token: 'none' @ line:1
 Description
  ModuleDeclaration
   ModuleDeclarationAnsi
    ModuleAnsiHeader
     ModuleKeyword
      Keyword
       Token: 'module' @ line:3
     ModuleIdentifier
      Identifier
       SimpleIdentifier
        Token: 'Mod' @ line:3
     ListOfPortDeclarations
      Symbol
       Token: '(' @ line:3
      Symbol
       Token: ')' @ line:3
     Symbol
      Token: ';' @ line:3
    Keyword
     Token: 'endmodule' @ line:4

Using include seems to offset the line number with the number of lines in the included file. inc.vh:

// Line 1
// Line 2
// Line 3
// Line 4
// Line 5
// Line 6

`include "inc.vh"
module Mod();
endmodule

SourceText
 Description
  ModuleDeclaration
   ModuleDeclarationAnsi
    ModuleAnsiHeader
     ModuleKeyword
      Keyword
       Token: 'module' @ line:8
     ModuleIdentifier
      Identifier
       SimpleIdentifier
        Token: 'Mod' @ line:8
     ListOfPortDeclarations
      Symbol
       Token: '(' @ line:8
      Symbol
       Token: ')' @ line:8
     Symbol
      Token: ';' @ line:8
    Keyword
     Token: 'endmodule' @ line:9

Is there some why to get the line number of the source-file (and in case of includes which source/include-file) and not (what I guess) the preprocessed file? Or is this a bug?

[DaveMcEwan]

This looks like a bug to me. See #58.

[atyrberg]

Thanks, Dave, this seems to fix the default_nettype case (and probably many other preprocessor directives).

The include case seems still to be wrong, but I guess that is a harder problem to solve since multiple files are involved.

[DaveMcEwan]

@atyrberg Would you mind making a draft PR or branch with a testcase? Perhaps by adding some checks to test2: https://github.com/dalance/sv-parser/blob/master/sv-parser-pp/src/preprocess.rs#L919

[atyrberg]

Thanks for the help to try to solve this.

I have added a commit atyrberg/sv-parser@ed2eac6d583640da3af0fe11f7c639aabdc517ed with a test showing the line number issue for include files. The line numbers are incorrect both for statements in the include file and in the source file. Also, I have not found any way to see if a statement is from an include file or from the source file (see out-comment assert in test).

I didn't manage to add the test to preprocessor.rs since I'm not sure if the problem is visible at this stage of the parsing. There are probably better ways to write this test, but I hope it shows the problem.

[DaveMcEwan]

Thanks, I see what you mean now, but I'm not sure of an immediate fix.

[DaveMcEwan]

@atyrberg Are your cases helped by #61? If not, would you consider adding some tests in sv-parser-pp/src/preprocess.rs? I think that adding tests is a bit easier now.

[atyrberg]

@DaveMcEwan After some simple testing, #61 doesn't seem to fix the problem.

I have looked quickly in sv-parser-pp/src/preprocess.rs but what I can see the problem is not there. In the preprocessor stage, it looks like the information is there (and probably correct, the origins in PreprocessedText have different PathBuf for the source file and the include file). I think the information about the included files is lost in some later stage.