Fortran should consider scrubbing unimplemented F'2003 features

[klausler]

For usage that conforms to the standard as of (say) Fortran 2003 but has not yet been implemented in any of the five or six major Fortran compilers, the standard should consider removing or deprecating the feature. I contend that something that exists only in the standard after 17 years but isn't yet working correctly in any implementation or (one assumes) any production application is not really part of the language, and would not be missed if it were to be removed.

I have several particular examples along these lines, but I would like to discuss the merits of the general principle first. It's too easy otherwise to be distracted by "well, I could think of a use for that" when it comes to specific cases -- and as I describe below, there would be a means for retaining features that deserve to remain.

My selfish motivation, as an implementor, is a desire to avoid the wasted time and expense of supporting features that nobody will use if I do or miss if I don't. But there might also be some benefit to users from overall simplification of the language.

So my specific proposal is this: Fortran 202x would contain a list of "Possible Future Deprecations" that enumerates usage from F'2003 and earlier that is demonstrably not implemented. Each item would contain a minimal test program that conforms to the standard but is not capable of running anywhere. Any item in that list that remains unimplemented as of Fortran 202Y would then become deprecated in that later standard. This would provide a window in which pet unimplemented features could be saved from deletion by any implementor willing to demonstrate them in the meantime, or any user willing and able to force at least one vendor to do so. And a similar list would appear in Fortran 202Y of F'2008 features not yet implemented by any compiler in its time frame.

[certik]

As a general principle this seems fine to me and consistent with another principle that I would like the committee to eventually adopt: only standardize things that have a prior compiler implementation (preferably two).

I would be curious to see the actual cases you found.

[wclodius2]

There are two items in F2003 that were controversial at the time and remain controversial: parameterized derived types (PDTs), and user defined derived type I/O (UDDTIO). Both were added at the urging of single national bodies:The British for PDTs, and Germany for UDDTIO. Great Britain had two members who were focussed on language orthogonality, and thought that if the intrinsic types could specify length and kind, then so should derived types and put in a lot of effort designing this feature. They persuaded much of the rest of the UK committee to their opinion. Germany had one active member who said he would veto the standard unless it had UDDTIO, and then after it was added vetoed the F2003 standard as too complex.

I won't speak to UDDTIO, but I have experimented a little with PDTs for bitsets. As of gfortran 10.2 PDTs for my purposes were broken. ifort was in better shape, but the restrictions on how PDTs could be used made them impractical for bitsets. Maybe Cray's compiler is in better shape.

As to requiring compiler implementations before adoption, good luck with that. A small part of F90 was the adoption of MIL-STD 1753, which made some common compiler extension a requirement for military compilers. Substantial parts of F77, the rest of F90, almost all of F95, and essentially all of F03 lacked prior compiler implementation. The only parts of F95 which may have had compiler implementations were FORALL, which was a failure, and the bug fixes to allocatable, which were obvious and had a TS report. F)8 and F18 did have some partial compiler experience with co-arrays, but most of the rest of the additions were a blank slate.

There are good reasons for that. Steve Lionel has commented elsewhere on how DEC got burned adopting an early version of PARAMETER, and now its successor, INTEL, refuses to adopt non-standardized extensions. The other commercial compilers have the same attitude. gfortran follows the piecemeal interests of its developers. FLANG/F18 has limited funding. LFortran is in a very preliminary state. Currently adopting an extension faces the following hurdles:

1. Developing an extension to a language as complicated as F18 is complicated. The preliminary work required before implementation is daunting. It is difficult to justify that investment in time and money unless you are certain your users will adopt that extension. Even then it may be difficult to justify the investment.
2. The standard bodies do all of that preliminary work for "free", but by the time a feature has been polished enough to be implementable most of the rest of a standard (or TS report) has been finished.
3. Users are reluctant to adopt a feature unless it has been standardized.

[certik]

Thanks @wclodius2 for the historical background.

Regarding prior implementation, the way it would most probably work is that it gets implemented in Flang/LFortran on a branch, but not merged into master. That way there is no maintenance cost for the compiler developers. People and the committee can play with the feature in the branch which would allow to answer the questions "how difficult is to implement the proposal" and "is it actually useful as we are hoping". Then we can talk about standardizing and a possible wider adoption of a given feature.

[cmacmackin]

There are two items in F2003 that were controversial at the time and remain controversial: parameterized derived types (PDTs), and user defined derived type I/O (UDDTIO).

At the risk of taking this thread off-track...

It may have been controversial, but at this point UDDTIO has been implemented in compilers (gfortran has supported it since v7.0). Personally, that always struck me as a useful feature and one that fit in well with other the ability to overload other operators. I was under the impression that there were some compilers to have implemented PDTs, although I agree that as they currently exist they are not practical for anything.

[wclodius2]

Several compilers claim to have implemented PDTs, http://fortranwiki.org/fortran/show/Fortran+2003+status http://fortranwiki.org/fortran/show/Fortran+2003+status including gfortran. I have only tested gfortran and Ifort. in gfortrn PDTs in combination with abstract types are broken. Ifort seemed to be working, at least it gave understandable error messages that indicated that what I wanted to do with PDTs was not allowed by the standard,

[klausler]

There's a useful distinction that can be made between the KIND and LEN genres of derived type parameters. KIND derived type parameters are straightforward to implement and are broadly supported. LEN derived type parameters are widely claimed to be supported, but it's not hard to write test cases using them that conform to the standard but can't be compiled by anybody.

[klausler]

I see that there's at least one downvote on the proposal; are you against deprecating and removing unimplemented features, or is your problem with the suggested process?

[FortranFan]

I personally will be completely opposed to this.

One simply cannot allow the baby to thrown out with the bathwater.

To me, every indication is all the features in the Fortran 2003 standard made it into the language standard following all the usual tugs and pulls of a lengthy process that had strong elements of consensus with voting and give and take and collaboration, however questionable might have been certain aspects of the process. I personally would do everything to respect the output of this long, drawn out effort that took a lot of contemplation and deliberation. To debug (yes, a standard can have bugs too), to refine, to enhance, and to build upon the concepts and features, sure. But to remove something in the standard starting 2003 revision, a no-no for me. This will hold true for me for anything with 2008, 2018, and 202X revisions as well.

Another consideration is besides certain specific aspects of PDTs (gfortran being the exception here with a false start perhaps to its PDT implementation effort and a continued pause thereafter leading to a broken state) and perhaps miscellaneous aspects such as support for additional CHARACTER kinds e.g., toward ISO 10646 character set (which is kinda optional to processors any way), there are quite a few compilers providing rather comprehensive support for Fortran 2003. One can credibly argue the missing aspects are either bugs due to a load of other issues (including perhaps the 'denseness' of the standardese, especially for those compiler writers who grew up outside the british 'commonwealth' locales) and a lack of a standard acceptance testing and QA/validation library or subtleties and intricacies requiring close attention that has not been demanded yet of these implementations by their customers. With PDTs and Intel Fortran, that clearly was the case but I've been feeding them tons of test cases and they have been able to get their compiler fixed each time.

But now with PDTs, as I explain here, it is rather convenient and useful in design of good scientific software. There are some use cases, particularly in the industry domain where I work, that are particularly well-suited for PDTs. Perhaps in 20+ years if the standard with 202y and its processor implementations have sufficiently full-featured generics capabilities, PDTs may become superfluous. In the meantime, considering Intel Fortran is nearly there with this feature in their attempt to serve customers like me (or more like an evangelist at this stage), deprecation of PDTs will be unconscionable.

Additionally, defined IO (aka UDDTIO) is a facility that is mostly a solved problem, it is implemented reasonably well in several compilers including gfortran, it is a nice-to-have feature that has proved convenient, particularly with serialization and deserialization of 'objects' in Fortran including with its NAMELIST aspect, something that helps boosts Fortran's image somewhat in terms of modernity. There cannot be raison d'etre to deprecate this now either.

[klausler]

The proposal would not deprecate any feature that has even a single working implementation of that particular feature. The features that you're defending are working in at least one compiler and are not going to go anywhere. Instead, I'm thinking about features that remain unimplemented after nearly two decades. What should be done with them?

[wclodius2]

The proposal is too vague. If asked what aspects of F03 are unimplemented, my gut feeling would have been PDTs with LEN parameters, but according to FortranFan they have been implemented well enough in ifort to satisfy him. UDDTIO and UCS-4 have several implementations. What unimplemented aspects of F03 are you thinking of?

[klausler]

I was hoping to be able to move from general principles to specific cases, not the other way around, since the problem of unimplemented features of long standing is a general one. But if you need a specific case to consider as an example, take the feature of general non-deferred type parameter expressions in POINTER and ALLOCATABLE components:

  TYPE :: T(N)
    INTEGER, LEN :: N
    TYPE(T(N+1)), POINTER :: P
  END TYPE

I don't know of a compiler that can cope with a non-constant and non-deferred type parameter expression in a POINTER or ALLOCATABLE component declaration. It may be an oversight that they are allowed -- any bounds of POINTER and ALLOCATABLE component arrays are required to be deferred. If their type parameter values were also required to be deferred, they'd obtain their values from assignments and ALLOCATE statements like bounds do. So the general type expression, if it worked, would serve as a default to be used on ALLOCATE.

[klausler]

Subclause 4.3 of Fortran 2018 has a long list of features from earlier standards (F'77 through F'2008) that were deleted or modified in the latest standard, so there's some precedent for removing things that turned out to be problematic. For example, a sequence type was allowed to have type parameters in F'2003 but not today. Were these deletions and modifications controversial? (Some of the F'77 behavior that's changed must have engendered some discussion about backward compatibility, I imagine.) Is there a process by which problematic features in the language can be placed on these lists?

[wclodius2]

Is there a process by which problematic features in the language can be placed on these lists?

One way is an interpretation request. If the request leads to the conclusion that a feature of the language is unimplementable, leads to a contradiction with other parts of the standard, or is otherwise undesirable then it may be removed. That being said an interpretation request against an F03 feature in 2020 feels odd to me. It would have been better done ten or fifteen years ago.

[klausler]

Is there a process by which problematic features in the language can be placed on these lists? One way is an interpretation request. If the request leads to the conclusion that a feature of the language is unimplementable, leads to a contradiction with other parts of the standard, or is otherwise undesirable then it may be removed. That being said an interpretation request against an F03 feature in 2020 feels odd to me. It would have been better done ten or fifteen years ago.

Maybe it would be better to address the root cause of unimplemented &/or unimplementable features getting into the standard in the first place, I guess, as Ondrej mentioned earlier.

[FortranFan]

@klausler wrote Sep 15, 2020 6:27 PM EDT:

Subclause 4.3 of Fortran 2018 has a long list of features from earlier standards (F'77 through F'2008) that were deleted or modified in the latest standard, so there's some precedent for removing things that turned out to be problematic. ..

Exactly. So as @wclodius2 mentioned, the process is in place for changes in a published standard with the committee, mostly via Interp requests. What this appears to lead to, as you all know, is targeted refinements, corrections, specific sub-item removals, etc.

Given this, I don't understand the original proposal, especially with its reference to Fortran 2003 features and the "not implemented" aspect as determined by a processor failing an arbitrary test and the implications thereafter, whether it's wholesale deprecation of the facility.

If the matter is something specific such as non-deferred type parameters of components of POINTER/ALLOCATABLE attribute in a derived type, is this new proposal necessary? Can it not suffice to use the process that led to the current standard having an incompatibility with the original F2003 such as "Fortran 2003 permitted a sequence type to have type parameters; that is not permitted by this document", "Fortran 2003 permitted a statement function to be of parameterized derived type; this document does not permit that.", etc.

How about starting with an Interp request first on this non-deferred type parameter issue?

[johnwcowan]

Maybe it would be better to address the root cause of unimplemented &/or unimplementable features getting into the standard in the first place, I guess, as Ondrej mentioned earlier.

Based on my experience with other standards, the root cause is that the Ayes have more Sitzfleisch than the Nays.