Changing how PRAGMA and END_PRAGMA tokens are created

kaarthikalagappan commented 3 years ago

P4 grammar specifies that a pragma statement needs a token to specify the beginning (PRAGMA) and end of a pragma statement (END_PRAGMA). P4's lexer creates the PRAGMA token when it encounters the "@pragma" symbol and changes the lexer state to indicate it's inside a pragma statement. The END_PRAGMA token is created when the lexer is in the pragma state and encounters a new line. Following is the extracted code block that I translated from P4's lexer file to SuperP4's p4lexer file where both the tokens are created (to be copied to P4Lexer.java):

"@pragma"       { saveState = PRAGMA_LINE; 
                  yybegin(YYINITIAL);
                  Language<P4Tag> syntax = new Language<P4Tag>(P4Tag.PRAGMA);                                                                
                  syntax.setLocation(new Location(fileName, yyline+1, yycolumn+1));                                                 
                  return syntax; 
                }

{whitespace}*({newline}|{linecomment})+{whitespace}*
{
  if (saveState == PRAGMA_LINE) {
    for (int i = 0; i < yytext().length(); i++) {
        if (yytext().charAt(i) == '\n') {
          saveState = 0 ; 
          Language<P4Tag> syntax = new Language<P4Tag>(P4Tag.END_PRAGMA);                                                                
          syntax.setLocation(new Location(fileName, yyline+1, yycolumn+1));                                                        
          return syntax;
        }
    }
  } 
  return new Layout(yytext(), true); 
}

And to create the respective tags (to be copied to P4Tag.java):

END_PRAGMA(getID("END_PRAGMA"), null, PreprocessorTag.END_PRAGMA),
PRAGMA(getID("PRAGMA"), null, PreprocessorTag.PRAGMA)

I incorrectly assumed that pragma was a preprocessor statement, hence I specified PRAGMA and END_PRAGMA to be a PreprocessorTag. Specifying it as part of the PreprocessorTag made me add those two values as part of the PreprocessorTag enum in Syntax.java:

  public static enum PreprocessorTag {
    NONE,
    OPEN_PAREN,
    CLOSE_PAREN,
    COMMA,
    ELLIPSIS,
    HASH,
    DOUBLE_HASH,
    NUMBER,
    // Currently using those two tags in P4's lexer to mark beginning 
    // and end of pragma statements according to the original P4 lexer logic
    PRAGMA,
    END_PRAGMA
  }

Since pragmas are not preprocessor statements, it wouldn't be appropriate to add them under PreprocessorTag, so working to find an alternative way that doesn't require them to be added to the PreprocessorTag enum

kaarthikalagappan commented 3 years ago

Commit d6b8df2 changes how PRAGMA and END_PRAGMA tokens are defined. Rather than as PreprocessorTag enum values, they are not defined as regular language tags/tokens in P4Tag.java. This also doesn't require us to modify Syntax.java

kaarthikalagappan commented 3 years ago

Pull request should have been to add superp4

appleseedlab / superc

Changing how PRAGMA and END_PRAGMA tokens are created #140