Support additional file extensions

[ivan-pi]

Once fpm will be out in the wild, we should try to support other Fortran file extensions. This is not so important right now, but I thought I will create an issue since I gathered the information.

Here is a table of file extensions with default support by different compilers:

Compiler	Fixed-form	Fixed-form with preprocessor	Free-form	Free form with preprocessor
gfortran	.f, .for, .ftn	.fpp, .F, .FOR, .FPP, .FTN	.f90, .f95, .f03, .f08	.F90, .F95, .F03, .F08
ifort	.f, .for, .ftn, .i	.fpp, .FPP, .F, .FOR, .FTN	.f90, .i90	.F90
nvfortran	.f .for .ftn	.F .FOR .FTN .fpp .FPP	.f90 .f95 .f03	.F90 .F95 .F03
nagfor	.f, .for, .ftn	.ff, .F	.f90, .f95	.ff90, .ff95, .F90, .F95
Cray	.f, .for	.F, .FOR	.f90, .f95, .f03, .f08, .f18, .ftn	.F90, .F95, .F03, .F08, .F18, .FTN
IBM	.f, .f77	.F, .F77	.f90, .f95, .f03, .f08	.F90, .F95, .F03, .F08
g95	.f, .for	.F, .FOR	.f90, .f95, .f03	.F90, .F95, .F03

The only options that are compatible along the tool-chains listed above are: .f, .F, .f90, .F90.

The .ftn extension has a clash between gfortran, ifort, and nagfor which assume fixed-form, and Cray which assumes free-form.

To enhance portability between compilers, we should either allow fpm to (copy and) rename the files silently, or have fpm print a message to the user to consider renaming his files.

Blog posts related to this issue:

• “Source Form Just Wants to be Free” by Doctor Fortran
• Fortran File Extensions by degenerateconic

Edit: Feel free to add more compilers.

[ivan-pi]

On second thought, most compilers provide flags which allow to select fixed- or free-form format, meaning we could try and implement this without the need to rename files (e.g. if fpm doesn't have write access).

But perhaps printing an message to the user would shift the community in the direction of adopting the compatible extensions.

[awvwgk]

Alternatively, we could decide on a minimal set compatible for all compilers and allow explicit extension / overwriting on per project basis in the package manifest (some projects use .f while being completely free format source).

[ivan-pi]

Good point. I recall now the dbcsr uses the .F extension for all their source files which are free-form.

So we should agree on some sane default behavior and allow the package developers to use whatever file extensions and source code format they want via the manifest.

[ivan-pi]

Here is a table of the respective compiler options for controlling input source format and preprocessing:

Compiler	Fixed-form	Free-form	Invoke (C) preprocessor
gfortran	-ffixed-form	-ffree-form	-cpp
ifort	-fixed	-free	-fpp
ifort (Windows)	/fixed	/free	/fpp
nvfortran	-Mfixed	-Mfree	-Mpreprocess
nagfor	-fixed	-free	-fpp
Cray	-f fixed	-f free	-eP, -eZ
IBM	-qfixed	-qfree, -k	-qpreprocess
g95	-ffixed-form	-ffree-form	-cpp
flang	-Mfixed	-Mfreeform	-Mpreprocess
oracle f95	-fixed	-free	-fpp or -xpp=fpp (cpp)

For fixed-form the default line length is 72. For free-form the default line lengths differ between compilers (132, 255, or any).

The preprocessors are not fully compatible between compilers.

[awvwgk]

Added NVHPC/PGI compilers, which additionally support .cuf for CUDA Fortran and .CUF for preprocessed CUDA Fortran.

[ivan-pi]

In the blog post from Steve, there is a link to a post from Intel: Problem: Fortran source files not compiled when building in Visual Studio, which shows how the source and header file extensions are specified in Visual studio.

I haven't studied the manifest or TOML syntax closely, but I imagine this would mean introducing something like this:

[build]
source.free = [".f90", ".f03", ".f08"]
source.fixed = [".f", ".f77"]

fpm would then neglect all files with other extensions. I am not sure if we need to specify any file extensions for include statements, or do we just allow the compiler to search for the files and report any errors?

[awvwgk]

I would remove the dot from the extension, but the general syntax looks good to me. We will probably need additional entries for the preprocessed files as well:

[build.source]
free = ["f90", "f03", "f08"]
free-preprocess = ["F90", "F03", "F08", "fypp"]  # or free-pp
fixed = ["f", "f77"]
fixed-preprocess = ["F", "F77", "fpp"]  # or fixed-pp

[ivan-pi]

I agree that is cleaner.

With respect to the preprocesser, I think we might need to distinguish between external preprocessors (like fypp, or preform) and those built into the compiler (C-like macro processors). Interestingly, ifort has the option to specify an alternative preprocessor using the flag -fpp-name=<name>.

[awvwgk]

The preprocessor and its operation mode ("built-in", "external") could be selected in a separate entry in the [build] table. In case fpm recognizes the Intel compiler, it could try to apply an external preprocessor using the given flag, but I fear we might need to preprocess separately anyway, in case module names are mangled with preprocessor directives.

[urbanjost]

Unless the names are treated differently I am a strong advocate for just ..f|.F|.F90|.f90. If specifying a suffix like .f03 meant Fortran was required to strictly apply the standard as specified for that standard version (which is not the case as far as the standard is concerned) I do not like the other extensions. For a package that is intended to be designed for use by others I think I prefer the user being required to do the preprocessing themselves and only present .f|.f90 files to fpm unless fpm bundles a "standard" preprocessor(s). For fpm use building a local package I'm not quite as strong an advocate for that.

[urbanjost]

Sun used to distribute an fpp command that was written in Fortran if I recall. I use my own. 96% of my Fortran files are actually .ff and .FF files that run through a preprocessor but a lot of experience has shown me you only distribute .f90 and .f files unless there is no alternative.

[ivan-pi]

Unless the names are treated differently I am a strong advocate for just ..f|.F|.F90|.f90. If specifying a suffix like .f03 meant Fortran was required to strictly apply the standard as specified for that standard version (which is not the case as far as the standard is concerned) I do not like the other extensions. For a package that is intended to be designed for use by others I think I prefer the user being required to do the preprocessing themselves and only present .f|.f90 files to fpm unless fpm bundles a "standard" preprocessor(s). For fpm use building a local package I'm not quite as strong an advocate for that.

Personally, I agree with sticking to ".f" for fixed, and ".f90" for free form. Definitely, we should not tie the extensions to specific language standard. However, I am afraid that among developers there is not total consensus. As Steve notes in his blog post: "as far as the standard was concerned, source lines were delivered to the “processor” (compiler) by fairies in the night.", which I believe is still true today. The file extensions are merely a default set by compiler vendors. Misconceptions concerning file extensions and the Fortran language standards pop up routinely on comp.lang.fortran, Stack Exchange, r/Fortran, and now even on our Discourse.

Admittedly, when I started to learn Fortran for CFD, I used the ".f95" extension. At that time I had never heard of Linux before, nor had I used the command line. The lowest entry point I could find was the Silverfrost FTN95 compiler on Windows. The example programs used the ".f95" extension. In fact, I was not even aware that modern editions of Fortran existed, as all the examples I could find on the internet were procedural style CFD codes in F77 or F90. Only recently (three years ago!), when I got access to the Intel Fortran compiler at my university, I found out I need to use the ".f90" extensions to have things work automatically. At the same time you have some respectable Fortran codebases (like cp2k), which bypass the default compiler options, and use ".F" even for free-form.

Since fpm is supposed to automatically detect Fortran source files, I believe in the future we should also search for other file extensions. If any are found, we have two options:

1. Inform the user that unless specified in the manifest, fpm assumes the source file extensions are ".f" for fixed-form, and ".f90" for free-form, and recommend the user to rename his files, or fill out the [build.source] section of the manifest.
2. Detect automatically if the sources are free or fixed, and whether they require preprocessing, and deal with it silently by setting all the necessary compiler flags.

Personally, I am in favor of the first option. As a beginner in Fortran, this would nudge me in the right direction.

Speaking of option 2, it seems like a fun project with neural-fortran, the input to the neural network could be the number of ! and C characters, frequency of spaces in the first six positions, frequency of upper and lower case letters, and the frequency of symbols suchs as #, @, and $ normally used by preprocessors.

[urbanjost]

I like the neural-fortran idea better; but you just reminded me of some elaborate make(1) files I inherited that were all set up for the Intel compiler except these lines that called "gfortran -fsyntax-only ..." with the output module files thrown into a scratch directory and then erased and a bunch of other oddities. It took a moment looking at the subsequent lines to see he was compiling the code with syntax checking with gfortran only and forcing -ffixed-form (or whatever the option is) and if it failed the syntax check he was assuming it was free-form. A dangerous game but apparently it worked the way he wanted. We took it out and renamed the files and changed a few other parts that had .ftn and such built into them. Not recommending anyone use that method! He had been moving a large amount of F77 code to free-format and apparently got tired of renaming the files and making related changes, I think. Several compilers allow for directives in the source files to specify fixed versus free format at least down to the level of a single procedure. When changing old large multi-procedure files that is particularly handy (versus splitting them all apart of having to rewrite them all, and so on). If you are grabbing fixed-format codes and merging them into modules that is initially very useful. If would be nice when free-format was introduced if that had been an attribute you could specify at the beginning of a procedure definition, like

freeformat elemental pure function MYPROC(...

but specifying a code syntax attribute and function attributes together would probably not appeal to everyone.

[ivan-pi]

For a package that is intended to be designed for use by others I think I prefer the user being required to do the preprocessing themselves and only present .f|.f90 files to fpm unless fpm bundles a "standard" preprocessor(s).

I guess this is the right approach for small to medium size libraries, that only aim to provide Fortran procedures. Still we cannot exclude the use of a preprocessor for purposes like conditional compilation depending upon platform, compiler version, or other uses (compile-time selection of precision, using a different back-end library).

I think with fpm we would also like to enable Fortran users to exchange templates and macros.

[FortranFan]

Here is a table of the respective compiler options for controlling input source format and preprocessing:

Compiler Fixed-form Free-form Invoke (C) preprocessor gfortran -ffixed-form -ffree-form -cpp ifort -fixed -free -fpp ifort (Windows) /fixed /free /fpp nvfortran -Mfixed -Mfree -Mpreprocess nagfor -fixed -free -fpp Cray -f fixed -f free -eP, -eZ IBM -qfixed -qfree, -k -qpreprocess g95 -ffixed-form -ffree-form -cpp flang -Mfixed -Mfreeform -Mpreprocess oracle f95 -fixed -free -fpp or -xpp=fpp (cpp) For fixed-form the default line length is 72. For free-form the default line lengths differ between compilers (132, 255, or any).

The preprocessors are not fully compatible between compilers.

@ivan-pi ,

Please keep -extfor compiler option in mind with Intel Fortran compilers. It may be of interest with other processors and if they already do or can provide similar flexibility (currently such a thing may be missing with gfortran, for example) and whether the Community will be keen to leverage it in fpm.