Inline comments with double quotes are not removed correctly

[reuterbal]

Hi,

again a very exotic problem... Consider the following (incomplete) example:

>>> fcode = """
... MODULE MOD
... IMPLICIT NONE
... TYPE(SOME_TYPE), PARAMETER :: TVAR(NVAR) = (/&
...   & SOME_TYPE('u         ',2),&! Horizontal wind component
...   & SOME_TYPE('v         ',2),&! "
...   & SOME_TYPE('t         ',2),&! Temperature
...   & SOME_TYPE('x         ',2)/)! Etc...
... END MODULE MOD
... """
>>> from fparser.common.readfortran import FortranStringReader
>>> from fparser.two.parser import ParserFactory
>>> reader = FortranStringReader(fcode)
>>> parser = ParserFactory().create(std='f2008')
>>> ast = parser(reader)
Traceback (most recent call last):
  File "/.../src/fparser/two/Fortran2003.py", line 237, in __new__
    return Base.__new__(cls, string)
  File "/.../src/fparser/two/utils.py", line 407, in __new__
    raise NoMatchError(errmsg)
fparser.two.utils.NoMatchError: at line 7
>>>  & SOME_TYPE('t         ',2),&! Temperature

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/.../src/fparser/two/Fortran2003.py", line 241, in __new__
    raise FortranSyntaxError(string, "")
fparser.two.utils.FortranSyntaxError: at line 7
>>>  & SOME_TYPE('t         ',2),&! Temperature

Note the double quotes in the inline-comment. Removing those parses successfully:

>>> fcode = fcode.replace('"', '')
>>> reader = FortranStringReader(fcode)
>>> ast = parser(reader)
>>> ast
Program(Module(Module_Stmt('MODULE', Name('MOD')), Specification_Part(Implicit_Part(Implicit_Stmt('NONE')), Type_Declaration_Stmt(Declaration_Type_Spec('TYPE', Type_Name('SOME_TYPE')), Attr_Spec_List(',', (Attr_Spec('PARAMETER'),)), Entity_Decl_List(',', (Entity_Decl(Name('TVAR'), Explicit_Shape_Spec_List(',', (Explicit_Shape_Spec(None, Name('NVAR')),)), None, Initialization('=', Array_Constructor('(/', Ac_Value_List(',', (Part_Ref(Name('SOME_TYPE'), Section_Subscript_List(',', (Char_Literal_Constant("'u         '", None), Int_Literal_Constant('2', None)))), Part_Ref(Name('SOME_TYPE'), Section_Subscript_List(',', (Char_Literal_Constant("'v         '", None), Int_Literal_Constant('2', None)))), Part_Ref(Name('SOME_TYPE'), Section_Subscript_List(',', (Char_Literal_Constant("'t         '", None), Int_Literal_Constant('2', None)))), Part_Ref(Name('SOME_TYPE'), Section_Subscript_List(',', (Char_Literal_Constant("'x         '", None), Int_Literal_Constant('2', None)))))), '/)'))),)))), End_Module_Stmt('MODULE', Name('MOD'))))

The problem seems to be already in the reader, my guess would be a problem with the detection (and elimination) of inline comments:

>>> reader = FortranStringReader(fcode)
>>> reader.next()
Line('MODULE MOD',(2, 2),None,None,<reader>)
>>> reader.next()
Line('IMPLICIT NONE',(3, 3),None,None,<reader>)
>>> reader.next()
Line("TYPE(SOME_TYPE), PARAMETER :: TVAR(NVAR) = (/ SOME_TYPE('u         ',2), SOME_TYPE('v         ',2), SOME_TYPE('t         ',2),&! Temperature",(4, 7),None,None,<reader>)

[reuterbal]

I have just hit this again. Since it's marked as "in progress", has there been any update? Or anything I can do to help?

[arporter]

Hmm, that's an embarrassingly long time ago. I'll take a look and see whether I did in fact do anything...

[arporter]

I can't find any trace of a branch for this. I must have got distracted...

[arporter]

Wow, that was hard but I've cleaned up some old code. @reuterbal would you mind confirming that this branch fixes your issue?

[reuterbal]

Wow, thanks a lot! I can confirm the branch works for me. The specific file causing the issue parses fine now.