search

mesonbuild / meson

The Meson Build System
http://mesonbuild.com
Apache License 2.0
5.4k stars 1.55k forks source link

BUG: Compilation of F77/Fixed Format Fortran breaks while trying to verify compiler (F2PY) #13381

Open ClarkMyWords opened 1 week ago

ClarkMyWords commented 1 week ago

Describe the bug This previous issue correctly addresses a problem with .f90/.f77 sanity checks and compilation issues. I am using meson 1.5.0.rc2 in order to take advantage of this commit which addresses that. However, I am running into a more complex issue once that has been addressed, as follows.

The build now attempts to create a file called testfile.f90 and compile it. I have grep'd for this name across the meson source and the numpy/f2py source, and I cannot find where it is created. However, this file subsequently gets clobbered by first plain text and then C code, which causes the compiler to fail and thus the build to not continue. The testfile.f90 file is cleaned up somewhere in the build, even as it fails, and I've not been able to actually find and open it to inspect it, so I haven't included it. I've included my most-recent meson-log.txt which shows the error, but, in summary:

Running compile:
Working directory:  /tmp/tmpm5k5u2ml/bbdir/meson-private/tmpmzw9ln71
Code:
stop; end program
-----------
Command line: `gfortran /tmp/tmpm5k5u2ml/bbdir/meson-private/tmpmzw9ln71/testfile.f90 -o /tmp/tmpm5k5u2ml/bbdir/meson-private/tmpmzw9ln71/output.exe -D_FILE_OFFSET_BITS=64 -fcray-pointer -cpp -fPIC -O3 -funroll-loops -ffixed-form -O0 -lquadmath -Wl,--allow-shlib-undefined -fcray-pointer -cpp -fPIC -O3 -funroll-loops -ffixed-form` -> 1
stderr:
/tmp/tmpm5k5u2ml/bbdir/meson-private/tmpmzw9ln71/testfile.f90:1:2:

    1 | stop; end program
      |  1
Error: Non-numeric character in statement label at (1)
-----------
Running compile:
Working directory:  /tmp/tmpm5k5u2ml/bbdir/meson-private/tmpcrh15p74
Code:

        #include<stddef.h>
        #include<stdio.h>
        int main(void) {
            printf("%ld\n", (long)(sizeof(void *)));
            return 0;
        }
-----------
Command line: `gfortran /tmp/tmpm5k5u2ml/bbdir/meson-private/tmpcrh15p74/testfile.f90 -o /tmp/tmpm5k5u2ml/bbdir/meson-private/tmpcrh15p74/output.exe -D_FILE_OFFSET_BITS=64 -fcray-pointer -cpp -fPIC -O3 -funroll-loops -ffixed-form -O0 -fcray-pointer -cpp -fPIC -O3 -funroll-loops -ffixed-form` -> 1
stderr:
/tmp/tmpm5k5u2ml/bbdir/meson-private/tmpcrh15p74/testfile.f90:2:10:

    2 |         #include<stddef.h>
      |          1
Error: Invalid character in name at (1)
/tmp/tmpm5k5u2ml/bbdir/meson-private/tmpcrh15p74/testfile.f90:3:10:

    3 |         #include<stdio.h>
      |          1
Error: Invalid character in name at (1)
/tmp/tmpm5k5u2ml/bbdir/meson-private/tmpcrh15p74/testfile.f90:5:18:

    5 |             printf("%ld\n", (long)(sizeof(void *)));
      |                  1
Error: Syntax error in PRINT statement at (1)
/tmp/tmpm5k5u2ml/bbdir/meson-private/tmpcrh15p74/testfile.f90:6:19:

    6 |             return 0;
      |                   1
Error: Alternate RETURN statement at (1) is only allowed within a SUBROUTINE
/tmp/tmpm5k5u2ml/bbdir/meson-private/tmpcrh15p74/testfile.f90:7:10:

    7 |         }
      |          1
Error: Invalid character in name at (1)
-----------
Could not compile test file /tmp/tmpm5k5u2ml/bbdir/meson-private/tmpcrh15p74/testfile.f90: 1

I was having this issue with a more complex .f77 file (specifically an Abaqus UMAT), but I have since tested it with a copy of the f77 sanitycheckf.f77 file (which I have titled simple.f77) and am having the same issue.

To Reproduce I have attached my meson.build, generated by f2py, as well as simple.f77 and the files which f2py attempts to compile that with (simple.pyf and simplemodule.c).

I am not running meson natively, instead via f2py, with numpy version 2.0.0. The f2py command I am using is as follows;

export FFLAGS="-ffixed-form"
f2py -c simple.pyf simple.f77 --backend meson

If I run with no FFLAGS set, I get the error: ERROR: No specified compiler can handle simple.f77 Which I believe is due to the compiler expecting free form and getting fixed form Fortran.

If I run with FFLAGS="-ffree-form", I get the longer, above error, with the mystery testfile.f90 being clobbered by errant C code.

Expected behavior Meson should recognize the .f77 file (which it is doing in this most recent release candidate) and then NOT attempt to continue to compile a .f90 file under -ffixed-form or have that .f90 file be clobbered with code that is not valid Fortran (in this case, a strange mixture of C and what I think is plaintext). Instead it should correctly set up the .f77 build directory and, hopefully compile the file.

system parameters

numpy and f2py version: 2.0.0

I have attempted to debug this, but f2py insulates the user from the meson setup via a Python subprocess, and I'm not skilled enough with meson to manually set the same environment up and attempt to reproduce the issue. Any insight into how to debug this would be most welcome!

Thanks for any and all help in advance!

dcbaker commented 5 days ago

I ran into this issue at some point looking at something else, but I don't remember exactly what I was doing. IIRC the issue is that there's an implementation in clike.py that fortran.py inherits, but really needs to override. I might have opened an issue to document it, but I can't remember and don't have time to look ATM

ClarkMyWords commented 3 days ago

@dcbaker Thank you! That really helped!

I found the issue, kind of in four parts. I'm documenting it in this comment. I would be happy to draft a PR, but there's at least one section of this about which I am unsure (see below) and I don't know what the best step(s) forward would be.

From #13319, the sanitycheckf.f file must be: ' PROGRAM MAIN\n PRINT *, "Fortran compilation is working."\n END\n'

The same holds for the 3 snippets of code in the functions find_library, has_multi_arguments and has_multi_link_arguments in compilers/fortran.py. Switching from stop; end program to ' PROGRAM MAIN\n END\n' fixed the issue I was having.

In order for this to work, I had to switch the order of suffixes for fortran in compilers/compilers.py, to 'fortran': ('f', 'for', 'ftn', 'fpp', 'f90', 'f95', 'f03', 'f08'),. This makes f the default file extension, which, much as making all the examples with fixed-form Fortran work.

However, my use-case is very simple, single-platform compilation. As I understand, the errant C code I had in my initial issue was a side effect of the stop; end program failing and not returning correctly. However, this C code is part of the output_is_64bit method in clike.py, which, in turn, calls a number of other methods which also include C code. I do not have enough Fortran experience to know how to best reimplement those things (I figured out the basic 64-bit check, but not any of the cross-compilation ones).

So, in summary, those are the issues at hand. I would be happy to contribute to a PR for some of these things, but I really am not confident in my Fortran skills or knowledge of the meson workflow enough to do it all myself. For my purposes, it turns out my use-case is simple enough that I believe I can patch what I need for my purposes until a release with fixes is made.

Thanks all!

ClarkMyWords commented 3 days ago

Okay, obviously this is not a full solution, and I would not want to pretend that it is, but, for anyone else who stumbles on this issue before it is fixed, I wanted to go ahead and include this gist I made. It modifies the meson installation in place (at least, if it's in a user-writable venv). Almost certainly NOT the best practice, but it'll do the patching I need for my (very simple) use-case, until a more robust solution is officially implemented!