Encountered unrecognized line while parsing Module

[koshMer]

Hi, I am not familiar with github, so I don't know whether this is the right place to ask:

• I am currently trying to compile Fortran code using fmodpy and ran into the following error:

Encountered unrecognized line while parsing Module:

'PURE FUNCTION MATINV2 ( A ) RESULT ( B )'

are pure functions not supported by your module?

• When I delete the attribute pure i run into the following exception - could you tell me why?:

fmodpy.exceptions.ParseError: Finished parsing FUNCTION MATINV2, but never declared A, B.

Attached you can find the minimum working example which i ran with python3 fmodpy minimum_working_example.f90 name=mwe: minimum_working_example.txt

• Also: I accidentally forgot the end module tmm statement the first time i ran python3 fmodpy minimum_working_example.f90 name=mwe and i had to CTRL-C the terminal because it wouldn't stop ^^ :)

I really like this project of yours, keep it going! 🥇

[tchlux]

@koshMer this is definitely the right spot! There's no support for PURE nor COMPLEX yet (because you're the first user to ask), but it looks like there are corresponding numpy types. So I should be able to add complex support with relatively little effort and give slightly more informative error messages. I'll try and do that in the next few days.

Also: I accidentally forgot the end module tmm statement the first time i ran python3 fmodpy minimum_working_example.f90 name=mwe and i had to CTRL-C the terminal because it wouldn't stop ^^ :)

Oof, I'll look into this!

[koshMer]

@tchlux thanks a lot! I'd really appreciate this.

[tchlux]

An update, I've got a sample test program with complex128 (REAL64 real and imaginary components in Fortran) running successfully. My next steps are to use that to model the correct parsing logic and generalize to the complex64 and complex256 numpy types.

A few more changes than I expected were required, so probably still a week away from finishing it.

[koshMer]

Thanks for the update! Tell me if there is anything i can help you with.

[tchlux]

@koshMer the new version 1.2.2 is now up. It supports complex numbers composed of 32 and 64 bit real numbers. Notably, I have not gotten it to work for 128 bit real numbers (corresponding to numpy.complex256). Please let me know if the new code works when you get a chance!

Your minimum_working_example file should now be successfully wrapped when the real128 complex numbers are replaced with real64 types. Unfortunately the Python ctypes module does not have native support for 128 bit floats, and when numpy.complex256 arrays are passed to Fortran their values appear to be garbled.

If you're able to play with the generated wrapper (specifically *_c_wrapper.f90 and *_python_wrapper.py in the generated directory) and get the 128 bit float transfer to work, please share! Otherwise, I'm going to consider this done for now.

[tchlux]

If you're able to play with the generated wrapper (specifically *_c_wrapper.f90 and *_python_wrapper.py in the generated directory) and get the 128 bit float transfer to work, please share!

For more information, it appears the trouble with 128-bit real numbers is architecture related (https://gcc.gnu.org/pipermail/fortran/2007-September/021958.html), and that the internal compiler representation of REAL128 might be mismatched (there's a difference between that and LONG DOUBLE?).

I tried making anything work by following a post (https://stackoverflow.com/a/28994452), but all I've really verified is that numpy.complex256, a ctypes.c_longdouble structure, and COMPLEX(KIND=REAL128) are not directly compatible (when gfortran is the compiler).

If you mainly care about the extra precision for doing Fortran computations, you can always cast up and down in the Fortran BIND(C) wrapper that is generated by making some minor modifications of your own.

fmodpy is built so that once it generates a wrapper, you no longer have it as a dependency. The goal is to enable people to make Fortran packages accessible from Python with as few dependencies as possible!

Also: I accidentally forgot the end module tmm statement the first time i ran python3 fmodpy minimum_working_example.f90 name=mwe and i had to CTRL-C the terminal because it wouldn't stop ^^ :)

☝️ I wasn't able to reproduce this. When I remove the end module, it just raises an error saying the module was never ended.

[koshMer]

The extra precision is not really important for me. Real64 will work great for me. I modified the MWE slightly -- removed the pure statement and changed Real128 to Real64 -- but unfortunately it wouldn't compile. Error is:

Finished parsing SUBROUTINE MATINV2, but never declared A, OUT.

You will get the same error when you change REAL to COMPLEX in the example in your readme.md. (EDIT: Without the change I could run your code just fine) I checked that I have the newest version installed (ran python3 -c "import fmodpy; fmodpy.configure()"). The output is in output.txt

Here you can find my modified MWE minimum_working_example.txt

And here is the modified example of your readme.md tchlux_example.txt

[tchlux]

Okay, something strange is happening here. I renamed those .txt files you sent to .f90 and it works on my macOS 11.3.1 with Python 3.9.5 computer and my Windows Subsystem for Linux (Ubuntu 20.04.2) with Python 3.8.5 environment. Can you add the option verbose=True to your execution and share the output of that? Your python version might be helpful too. So (assuming the minimum_working_example.txt file is renamed to minimum_working_example.f90 locally) paste a code block here with the output of these two:

python3 --version
python3 -m fmodpy minimum_working_example.f90 name=mwe verbose=True

For example, this is the output of my fmodpy compilation:

______________________________________________________________________
fimport

fmodpy configuration:
  autocompile = True
  blas = False
  config_file = '.fmodpy.py'
  debug_line_numbers = False
  end_is_named = True
  f_compiler = 'gfortran'
  f_compiler_args = ['-fPIC', '-shared', '-O3']
  home_directory = '/home/thomas'
  implicit_typing = False
  lapack = False
  link_blas = ['-lblas']
  link_lapack = ['-lblas', '-llapack']
  link_omp = ['-fopenmp']
  log_file = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
  omp = False
  rebuild = False
  show_warnings = True
  verbose = True
  wrap = True

=========================================================================
Input file directory:  /home/thomas/fmodpy_issue
Input file name:       minimum_working_example-2.f90
Base module name:      mwe2
Using build dir:       temporary directory
  fortran wrapper:     mwe2_c_wrapper.f90
  python wrapper:      mwe2_python_wrapper.py
Output module path:    /home/thomas/fmodpy_issue
=========================================================================

Using temporary build directory at '/tmp/tmpf7ff_erb'.
Build directory is different from source directory..
 sym-linking '/home/thomas/fmodpy_issue/minimum_working_example-2.f90' to '/tmp/tmpf7ff_erb/minimum_working_exam
ple-2.f90'
 sym-linking '/home/thomas/fmodpy_issue/mwe.f90' to '/tmp/tmpf7ff_erb/mwe.f90'
 sym-linking '/home/thomas/fmodpy_issue/output.txt' to '/tmp/tmpf7ff_erb/output.txt'
 sym-linking '/home/thomas/fmodpy_issue/tchlux_example.f90' to '/tmp/tmpf7ff_erb/tchlux_example.f90'

Attempting to autocompile..

 reading 'minimum_working_example-2.f90' to check if it can be autocompiled.. yes.
 reading 'tchlux_example.f90' to check if it can be autocompiled.. yes.
 skipping '/tmp/tmpf7ff_erb/output.txt'
 reading 'mwe.f90' to check if it can be autocompiled.. yes.

Compiling 'minimum_working_example-2.f90'..
 gfortran -fPIC -shared -O3 minimum_working_example-2.f90
  success.

Identified dependencies:
  minimum_working_example-2.f90

File.parse 
Subroutine.parse MATINV2
 Subroutine.parse done.
 File.parse done.
----------------------------------------------------------------------
FILE
  SUBROUTINE MATINV2(A, OUT)
    ! This subroutine computes the inverse of matrix A.
    !
    ! INPUT:
    !   A(M,N)  --  A 2D matrix of 64 bit complex floats.
    !
    ! OUTPUT:
    !   out(M,N)  --  The matrix that is the result of matinv2(A).
    !
    USE ISO_FORTRAN_ENV , ONLY : REAL64
    IMPLICIT NONE
    COMPLEX(KIND=REAL64), INTENT(IN), DIMENSION(2,2) :: A
    COMPLEX(KIND=REAL64), INTENT(OUT), DIMENSION(2,2) :: OUT
  END SUBROUTINE MATINV2

----------------------------------------------------------------------

Making symlink from '__init__.py' to 'mwe2_python_wrapper.py'

Moving from:
  /tmp/tmpf7ff_erb
to
  /home/thomas/fmodpy_issue/mwe2

Finished making module 'mwe2'.

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Running system command with arguments
   gfortran -fPIC -shared -O3 -o mwe2.so minimum_working_example-2.f90 mwe2_c_wrapper.f90

[koshMer]

I will check. But it may also very well be that i screwed something up ^^. When installing the new version of fmodpy, is it okay that i just ran python3 setup.py install on top of the existing installation of fmodpy?

[koshMer]

python3 --version yields Python 3.8.5. I am running anaconda, if that helps.

Output of python3 -m fmodpy minimum_working_example.f90 name=mwe verbose=True is: fmodpy_verbose.txt

[tchlux]

When installing the new version of fmodpy, is it okay that i just ran python3 setup.py install on top of the existing installation of fmodpy?

I'm not sure. I've only ever used python3 -m pip install --user fmodpy for testing. You could try using pip, which should automatically upgrade, or I think you can use python3 setup.py install --force if you're installing from the cloned source repository and want to overwrite an existing install.

I modified the MWE slightly -- removed the pure statement and

Also, I updated the code and it should handle the PURE statement fine now. You can leave that in your code if you want it. It's actually not relevant for the wrapper and should be ignored by fmodpy (same as with RECURSIVE).

Output of python3 -m fmodpy minimum_working_example.f90 name=mwe verbose=True is: fmodpy_verbose.txt

That looks like it worked! Any errors when you try to import mwe afterwards?

[koshMer]

Seems like user error to me ^^. I was under the impression that if i used fmodpy.fimport() in my python file i didn't need to compile it first using the command line ...

But as it seems i need to compile it first with python3 -m fmodpy minimum_working_example.f90 and then use the fimport() statement?

Anyways, if i do it in this order, the code works!

[tchlux]

I was under the impression that if i used fmodpy.fimport() in my python file i didn't need to compile it first using the command line

☝️ this is how it should work. It's possible that a "corrupt" incomplete build was causing the error? If you delete the directory created by compiling from the command line, then just use your python code it should all be handled automatically.

Another thing you can try, is modify your source Fortran code and run your python (without rebuilding from the command line), it should automatically generate a new wrapper and overwrite the old one.

[koshMer]

yup, that's what seems to be wrong. If i change the .f90 file or delete the directory created by compiling from the command line it doesn't work anymore.

I tried running pip uninstall and reinstalled the package but that didn't help, unfortunately. I'll try to fix this and maybe look into building a conda package from your setup.py if that's okay with you. I've never done that so it may take a while :)

I think we consider the original Issue as closed. Thanks a lot! Complex numbers are the bread and butter for a lot of physics-related stuff so that helped a lot.

On a side note: Is there any reason why dashes - are not allowed in a module name? :D (tried to compile a module with one it and it threw this error at me: ... must match the regexp "^[a-zA-Z_]+[a-zA-Z0-9_]*"

[tchlux]

any reason why dashes - are not allowed in a module name?

This is a python restriction, modules cannot contain - because the import <name> statements don't have enclosing quotes around <name> and so - gets interpreted as an infix minus operator. You could probably hack it into working by using importlib, but in general that's not recommended.

I tried running pip uninstall and reinstalled the package but that didn't help, unfortunately.

Yeah, my first guess would be that there's a different version of fmodpy being resolved based on PATH and PYTHONPATH in your shell environments for the two scenarios. You could check by printing fmodpy.__version__ in your python code to see if it is different from what you see at the bottom of the help page when running python3 -m fmodpy from the command line.

maybe look into building a conda package from your setup.py

Go for it! I've never done that, so I'm not sure what it involves. Just let me know if there's anything I can do to help.

[koshMer]

Thanks for the explanation! I guess I'll just rename my fortran file ^^ Have a good day :)