modelop / hadrian

Implementations of the Portable Format for Analytics (PFA)
Apache License 2.0
129 stars 49 forks source link

PFA function requires reserved keyword "to" in record name #34

Open bmwilly opened 7 years ago

bmwilly commented 7 years ago

Several PFA library functions (e.g., interp.linear) require records with the field name to, but trying to define a record with a field to leads to an error.

Minimal reproducible example:

In [1]: from titus import prettypfa
      :
      : doc = """
      : types: MyRecord = record(x: int, to: double);
      : input: int
      : output: int
      : action: 1
      : """
      :
      : prettypfa.jsonNode(doc)
      :
      :
---------------------------------------------------------------------------
PrettyPfaException                        Traceback (most recent call last)
<ipython-input-1-0a736b5ef5a3> in <module>()
      8 """
      9
---> 10 prettypfa.jsonNode(doc)
     11

/Users/brandon/miniconda2/lib/python2.7/site-packages/titus/prettypfa.pyc in jsonNode(text, lineNumbers, check, version, subs, **subs2)
   2214     :return: PFA in Pythonized JSON
   2215     """
-> 2216     return ast(text, check, version, subs, **subs2).jsonNode(lineNumbers, set())
   2217
   2218 def json(text, lineNumbers=True, check=True, version=None, subs={}, **subs2):

/Users/brandon/miniconda2/lib/python2.7/site-packages/titus/prettypfa.pyc in ast(text, check, version, subs, **subs2)
   2187             parser.initialize(lex, yacc)
   2188
-> 2189     out = parser.parse(text, subs2)
   2190
   2191     anysubs = lambda x: x

/Users/brandon/miniconda2/lib/python2.7/site-packages/titus/prettypfa.pyc in parse(self, text, subs)
   2122         self.text = text
   2123         self.subs = subs
-> 2124         out = self.yacc.parse(text, lexer=self.lexer)
   2125         if self.wholeDocument:
   2126             return out

/Users/brandon/miniconda2/lib/python2.7/site-packages/ply/yacc.pyc in parse(self, input, lexer, debug, tracking, tokenfunc)
    329             return self.parseopt(input, lexer, debug, tracking, tokenfunc)
    330         else:
--> 331             return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
    332
    333

/Users/brandon/miniconda2/lib/python2.7/site-packages/ply/yacc.pyc in parseopt_notrack(self, input, lexer, debug, tracking, tokenfunc)
   1197                             errtoken.lexer = lexer
   1198                         self.state = state
-> 1199                         tok = call_errorfunc(self.errorfunc, errtoken, self)
   1200                         if self.errorok:
   1201                             # User must have done some kind of panic

/Users/brandon/miniconda2/lib/python2.7/site-packages/ply/yacc.pyc in call_errorfunc(errorfunc, token, parser)
    191     _token = parser.token
    192     _restart = parser.restart
--> 193     r = errorfunc(token)
    194     try:
    195         del _errok, _token, _restart

/Users/brandon/miniconda2/lib/python2.7/site-packages/titus/prettypfa.pyc in p_error(p)
   2103                 else:
   2104                     offendingLine = "\n".join(insertArrow(lines[(lineno - 1):(lineno + 2)], 1))
-> 2105                     raise PrettyPfaException("Parsing syntax error on line {0}:\n{1}".format(p.lineno, offendingLine))
   2106
   2107         self.yacc = yacc.yacc(debug=False, write_tables=False)

PrettyPfaException: Parsing syntax error on line 2:
types: MyRecord = record(x: int, to: double);
input: int     <----
output: int
jpivarski commented 7 years ago

That's a PrettyPFA limitation— which developed separately from the PFA library. "to" is a reserved word in the PrettyPFA parser, but it's a required field in records that passed into interpolation functions.

Since the PFA specification is more robust than PrettyPFA (which is just an implementation), I think the best solution to this is to modify PrettyPFA, un-reserving "to" as a keyword and replacing it with some punctuation, such as "-->".