Strange behevior related to include statement. No Include class yet in nclock_statement.py

[GoogleCodeExporter]

Hi pearu,
I looked into r40. It works OK.But two point if you can modify.

1. When I parsed a code, I can not find the include statement.
It seems that there is no class for the include statement.
I think the include statement should make depth one-rank deeper
(as an attribute of the "Include" Class, included file name. 
 Maybe relative path from the source is good from 
 the view of protability).
---Now we see strange results; span gives a line number in the included file. 

2.Then I found that the include statement in a fortran code is automatically
  classified to a comment. I think this is wrong procedure.
  It makes difficult to use fparser as a rewriter.
  If you introduce a class for include, then its instance 
  of included file is just empty, and no content.

So two problems above is systematically solved by adding a "Include" Class
in block_statement.py, I think

Original issue reported on code.google.com by TakaoKot...@gmail.com on 27 Mar 2010 at 11:36

[GoogleCodeExporter]

Note that INCLUDE line is not a Fortran statement according to Fortran standard.
And hence there should not be an Include class derived from Statement.

Hmm, it seems that the issue of resolving INCLUDE lines is similar
to the issue of resolving preprocessor directives.

The "Source form" section in Fortran standard gives an idea how to resolve these
issues for rewriter type of applications. It says that 
"""
A Fortran program unit is a sequence of one or more lines, organized as Fortran
statements, comments, and INCLUDE lines.
"""
For fparser we could extend this definition to
"""
A Fortran program unit is a sequence of one or more lines, organized as Fortran
statements, comments, and INCLUDE lines, and preprocessor directives.
"""
and let the fparser reader to return three types of instances:

  Line - contains a Fortran statement
  Comment - contains a comment
  Unresolved - contains an unresolved INCLUDE line or an preprocessor directive.

And there should be an option whether INCLUDE lines should be resolved or not.
Similarly, there should be an option for applying preprocessor or not.

Btw, the fact that statements in included files start from 1, is expected.
The corresponding Line instances have different reader instance.

Original comment by pearu.peterson on 27 Mar 2010 at 1:07

• Changed state: Started

[GoogleCodeExporter]

I think "include" is a part of fortran standard (a standard extention). 
Within my knowledge, I don't know a fortran compilar which do not allow include
statement. So I think it is reasonable to treat it as a Statement.
(When F77, complex*8 or so are not in fortran regulation, but it was
taken as a standard extension).

----
I don't think it is so meaningful for the fortran reader to classify "input 
lines" so
much. Reader should be just a reader. So we have

  Comment - contains comment line. 
            Not affects to any functions of obtained binaries.

  Line - Except Comment. It recognizes line continuation, and line division by ;. 
         (thus Line is meaningful unit in codes).

  IncludedLine - These are Line from included files. 
                 This can be derived from Line class.

FortranFileReader should allow the to specify location of *.inc files.
I think *.inc need not allow include statement in it; but I don't know.

---
When fparser can recognize preprocessor in future, I think a line starting from 
'#'
is simply classified to a Line. Then parser identify it as a preprocessor 
directive.
Or maybe reader can recognize it as PreprocesserDirective or so.

---
So, maybe, I guess that the introduction Unresolved class do not simplify 
fparser and
its concept. But, anyway, I have little knowledge on fparser. I have to respect 
your
decision.

Original comment by TakaoKot...@gmail.com on 27 Mar 2010 at 3:59

[GoogleCodeExporter]

See at the end of
  http://www.f2py.com/home/references
where you will find Fortran language standards.
Take, for example, Fortran 2003 where you'll find that
INCLUDE line is indeed part of the Fortran standard
but it is not a Fortran statement.
F77 complex*8 has another story.

FortranFileReader has include_dirs kw argument that
can be used to specify the location of included files.

In fparser, Line is interpreted as a line that
contains single and complete Fortran statement, that is,
line continuations are resolved by the reader.
It is already a difficult task and I don't want the
parser deal with it.

Note that initially the main application of fparser
was not to rewrite Fortran codes but to get useful
information from it for wrapper generation. Later it
turned out that fparser could be used for rewriting
Fortran codes as well.

Using IncludeLine is a good idea. For preprocessor lines
we could have PPLine or similar.

I don't think we would need any more different classes.
These four should cover all cases.

Original comment by pearu.peterson on 27 Mar 2010 at 4:39

[GoogleCodeExporter]

>Using IncludeLine is a good idea. For preprocessor lines
>we could have PPLine or similar.
>
>I don't think we would need any more different classes.
>These four should cover all cases.
>
I also think so. 

In anyway, current revision r41 change "include foo" to "! include foo" as 
Comment.
I think this is not good  (I can not distingush whether ! exist from
the begining or just the FortranFileReader add it). I think
it is reasonable to leave it as a Line. And parset just do "nothing" for 
Include class.

Original comment by TakaoKot...@gmail.com on 28 Mar 2010 at 4:31