fparser2 cannot handle preprocessor statements

[TeranIvy]

LFRic code uses preprocessing statements (Physics code and unit testing for now but there are likely to be more) and fparser2 is not able to parse them. For example,

#ifdef UM_PHYSICS
  ! UM BL scheme
  use bl_alg_mod,  only: bl_alg_step
#endif

raises the following error

  File "Fparser2_LFRicDirs.py", line 61, in <module>
    kernel_obj = FPARSER2(READER)
  File "fparser/src/fparser/two/Fortran2003.py", line 212, in __new__
    raise FortranSyntaxError(error)
fparser.two.utils.FortranSyntaxError: at line 104
>>>#ifdef UM_PHYSICS

[rupertford]

My general view is that pre-processing should be applied before parsing the code. The issue is that code before pre-processing is a potential set of different codes, not a single code.

Code before pre-processing may not be valid and it may cause PSyclone to generate incorrect code. For example ...

  call invoke(fred(a,b,c), &
#ifdef something
                    kate(b,a,c) &
#else
                    john(a,d,e) &
#endif
                    )

In the above example there is a missing comma after kate() and before john() that the parser would fail on (hopefully) if we don't pre-process. Essentially all fortran rules go out of the window when you have ifdefs, as the content between them is treated as text.

Even if we had a valid code when taking both idef paths into consideration we would then try to generate a PSy-layer for an invalid invoke as PSyclone would try to generate something including kernels kate, fred and john.

Perhaps the PSyclone script should include the option of calling a pre-processor and being given the associated keys to make life easier, but I don't think it is a big deal to require someone to run cpp on it before running psyclone.

So, in summary, I don't think it is a bug!

[TeranIvy]

I agree, I think cpp should take on that job. I will close the issue!