search

j3-fortran / fortran_proposals

Proposals for the Fortran Standard Committee
174 stars 14 forks source link

Formulate long term vision for Fortran in terms of features #59

Open certik opened 4 years ago

certik commented 4 years ago

I would like the community and the committee to formulate a long term vision what features we would like to work towards in Fortran and how they work together, and consequently, which features we do not want.

As an example, let's say that we would like to have some form of templates and exceptions. Then the two features should be formulated in a way so that it allows us to write all the things we want and that they work well together. Then there are a lot of features that should be implemented in a way to play well together, such as: #44, #45 and then there are features that are not needed, such as #58, because one can achieve the same thing with templates and strong concepts / interfaces.

As another example, let's say we do not want generic programming. In that case, we might consider something like #58.

The goal should be to design the language features well, so that we do not end up with lots of orthogonal features that do not play well together. Having such a vision will help us guide which proposals and features we should prioritize and also help us design all the features to work well together.

certik commented 4 years ago

@FortranFan, in https://github.com/j3-fortran/fortran_proposals/issues/2#issuecomment-547173802 you mentioned:

Fortran is not a general-purpose language. Rather, it is a domain specific language for array oriented scientific computing.

That may be the current reality with Fortran, but almost everyone who have worked on its language design and continue to do so will greatly dislike being reduced as such and would very much want Fortran to be seen as a general-purpose language.

Can you expand on this a bit?

My view is that I want Fortran to be strong for array oriented scientific computing. If I want to write compilers, C++ is a better language (and so I chose C++ for LFortran). If I want to write web servers, then Python, Go, or other languages are much better also. But if I want to write numerical code, Fortran is a much better language than C++ (easier to write, read and for compilers to optimize), and I would like it to stay that way.

When you say "general-purpose", what domains do you want Fortran to be used in? And as part of this, what features would you like to see added?

FortranFan commented 4 years ago

@certik asked:

When you say "general-purpose", what domains do you want Fortran to be used in? And as part of this, what features would you like to see added?

I think the Wikipedia page on Fortran got it correct with its description, "Fortran is a general-purpose, compiled imperative programming language that is especially suited to numeric computation and scientific computing." And it summarizes Fortran as "multi-paradigm: structured, imperative (procedural, object-oriented), generic." Fortran is now also a parallel programming language - cheers!

Numeric computations and scientific computing are now highly multidisciplinary and they involve a lot of cross-fertilization and collaboration, as you know. And increasingly, there is the acceptance of "computational science" as a veritable third branch of scientific advancement that draws from and also feeds into progress in the traditional ones of theory and experiments. Given all this, Fortran needs to continually evolve with advances with science and computing and it needs to offer features that cover almost all the paradigms, as listed above.

Processing of massive amounts of data in whatever format, machine or human-readable, toward all manner of activities - whether it be climate modeling, DNA research, machine learning, AI/automation, visualization, etc. - is also just as important to scientific and numeric computing as floating-point computation of matrices. It's often counter-productive, I think, to stress one area, say array arithmetic of reals, too much above some other e.g., bit stream processing,

My vision is for Fortran to rise back as the lingua franca in scientific and technical computing. I think this requires Fortran to be an evolving and advancing language that keeps pace in terms of being "feature complete" toward the programming needs of scientists and technologists.

The way I see things is that Fortran currently has GAPS in a few areas of critical interest to computational scientists and technologists including in research, academia, and especially industry. Thus it is NOT "feature complete". The results of the survey by WG5 cover most of these gaps e.g., generics, exception handling, bit data processing, unsigned integers, etc. And I would add a few more in object-oriented aspects as well as good coding practices. Most of these are essentially "solved problems" in terms of computer engineering; robust and functional implementations are available in too many languages and there is little to no open-ended computer science research/investigation element in almost any of the current gaps e.g., Ada added support for generics during the early 1990s.

So my immediate mission will be to try to help "knock off" as many items on the gaps list as quickly as possible, so Fortran can get back to being competitive with other languages and compute platforms.

Modern Fortran is, unfortunately, like the curate's egg, good only in parts and that too only with a modern i.e., "p.c." interpretation of that phrase. When one tries to "use it in anger" toward any big, new effort or attempt a significant refactoring of an existing FORTRAN legacy code-base one ends up uttering stuff like in this cartoon: WG5 is either unaware or in denial about this basic fact. That has gotta change, otherwise Fortran will soon lose its raison d'etre.

certik commented 4 years ago

@FortranFan thank you. I think that's fine to expand Fortran's reach to be especially suited for "numeric computation and scientific computing". (I think your "general purpose" is the same as my "domain specific". We both want a language for scientific computing, as opposed to, say, for web servers.)

I just want to caution against thinking that generics or exceptions are a solved problem, I don't think they are. For example just half a year ago the committee was considering generics in a form that I would be against. It's extremely easy to get it wrong and end up with a system that has more negatives (such as long compilation time, bad error messages, unreadable code, etc.) than positives. At the same time, I am optimistic, and I think there is a way to do it right, but we have to work hard on that and I think having well designed templates in Fortran would be beneficial to its goal of being the language for scientific computing.

FortranFan commented 4 years ago

@certik wrote:

.. I just want to caution against thinking that generics or exceptions are a solved problem, I don't think they are. For example just half a year ago the committee was considering generics in a form that I would be against. It's extremely easy to get it wrong and end up with a system that has more negatives (such as long compilation time, bad error messages, unreadable code, etc.) than positives. ..

I agree.

When I wrote "solved problem", I only meant the computer science and engineering behind the approach and language design that might be needed to develop any particular feature.

What's extremely, extremely difficult, as can be seen on any issue thread on this site, is to first establish the use cases of sufficient interest, to gain any sense of consensus around them especially when 2 or more people particularly "long in the tooth" are involved, to then agree on requirements and to set priorities. Basically, problem formulation is itself very hard. But once it is done for any of the features including generics, a solution can be developed given the current state of compiler "technology". Now the direction Fortran was going previously with generics was driven by an attempt to limit the language to certain use cases and that was pushed by an influential few. A narrow mind can greatly hijack the conceptualization phase and bias the problem formulation.

gronki commented 4 years ago

What I consider a "Fortran spirit" (or: what makes it NOT C++):

This is all in the context that it is mostly used by people who are paid for the results of their computations rather than the code itself. This is why I think tinkering with user-written primary components (strings, list) is very against the goal here and would like to see more common operations closed in intrinsic functions/subroutines.

FortranFan commented 4 years ago

A comment based on DRY/WET discussions in #1: I hope the vision for Fortran will always consider it as a general-purpose, multi-paradigm language even as it retains a strong focus on scientific and technical programming!

Re: multi-paradigm: to achieve it I think one has to practice it regularly even in small aspects and being more open-minded to facilities that might be in areas outside their own experience base or inclinations (e.g., being considerate to needs in generics even if one doesn't care for it because of being interested only in, say, Fortran 95 style code-bases with coarrays perhaps) and to generally have a wide, open tent attitude to allow its practitioners to follow different approaches that suit their needs, all within some reasonable boundaries of course.

I've lost count of the number of times I suggest some improvement in the realm of object-oriented (OO) programming and some other Fortranner will comment they don't particularly care for OO and they never use it. But now, OO is such a critical aspect of so many large and new code-bases coming up in the scientific and technical computing domains even (particularly in industry) and it's blended so much together with other paradigms including parallel programming that to overlook areas of improvement toward OO in Fortran is a major disservice.

Back to the smallish aspect: I think it is entirely reasonable for a group of practitioners to want to control "namespaces" and particularly their "pollution" (which are at the root of several issues like #1, #87, etc.) in one way, say via

use, namespace :: foo_m
..
call foo_m::bar(..  ! note :: is only an illustrative token
..

and another group to do so by

use foo_m, only : bar ..
..
call bar(..
..

and for Fortran to support both the options which legitimately fall in the category of good coding practices.

The usual arguments against this are language bloat and time-resource constraints for standard development. I personally do not think the first is applicable once a matter at hand has the passed the filter of good coding practice. The latter is something that can be managed better and better by striving continuously for improved collaboration and greater workflow efficiency with online platforms like this.

Fortran seriously needs to evolve faster and further to have any hope of being relevant in the future.

certik commented 4 years ago

Fortran's strengths are in writing prototype codes and pure numerical codes. Much better than C++ there (faster and easier to develop, faster compilation, faster runtime execution, ...). However, for large multiphysics codes, a lot of time is spent doing all the maintenance around the numerical core (reading input files, managing the many multiphysics pieces, turning features on and off, ....) and for that C++ does provide more building blocks that one can use, while in Fortran the hands are tied. And so even though C++ is worse for the numerical part, one you get a very large code, the argument for C++ is very compelling. (It becomes subjective, I personally prefer using Fortran for very large codes also, but people who know both and prefer C++ have a very solid reason for that, that cannot be dismissed as "they don't know what they are talking about").

And so that is the main motivation for adding dictionaries, lists into the language, better string support (#24), some generic programming, improvements in OO, etc., to help maintain large codes and give maintainers of those codes tools to choose from so that their hand is not so tied behind their back. For the performance critical parts of the codes, one should still use raw arrays and loops, just like today.

Fortran strength can also be in parallel programming (coarrays being in the language) and in principle on GPU also. But currently C++ has at least a way to run on GPUs (Kokkos, RAJA), while Fortran does not (yes, there is CUDA Fortran which some of our teams use, but it does not work in every compiler...) --- and that is the primary motivation why there is a strong push away from Fortran to C++.

everythingfunctional commented 4 years ago

I wrote up a blog post about what I think should be done to develop Fortran's "vision". I base my opinion's there heavily on some of the ideas the Rust team seems to be using as found in this video. (the stuff most relevant to my point starting around the 13 minute mark).

I'm curious what you guys would think Fortran's "core values" should be? What about secondary values? What shouldn't we care about?

MarDiehl commented 3 years ago

I hope that Fortran finds a better balance between maintaining backward compatibility and modernization. I know that backward compatibility is important to many users, but I have the impression that it stops the further evolution of the language. The 'code once, run/compile forever' idea that the Fortran community sticks to is IMHO obsolete. Unit testing is industry standard today and enables continuous refactoring and modernization. I find it ridiculous that fixed form code is still used. Instead of forcing compiler developers to spend their time and efforts to ensure that such old code still works, the respective maintainers should to their job and modernize their code.

One striking example is LAPACK: It is the prime example for a very critical, and maintained software that is needed by virtually everyone doing linear algebra. It is written in Fortran, but one does not even get type checks during compilation from modules. Modules are a feature introduced in Fortran 30 years ago! As a result, using this Fortran library from python/numpy is more convenient that using it directly in Fortran. I'm convinced that pressure in the form of 'make modules mandatory' for F2008 would have triggered a re-write of LAPACK in truly modern Fortran and the situation would be much better now.

So my vision of Fortran is a fast, compiled language for scientific and engineering applications that values stability and innovation at the same time. To put numbers on this: A new revision every 2 years, a reference compiler that implements the new features after 2 more years and removal of deprecated features after 10 years.

epagone commented 3 years ago

@MarDiehl I entirely agree. Some time ago, as a stepping stone in this "modernisation" direction, I have proposed a set of defaults for fpm to encourage this transition but reactions were, uhm, mixed.

certik commented 3 years ago

@MarDiehl, I would suggest a slightly bigger scope than what you proposed: not to delete deprecated features, but perhaps they have to be enabled with a compiler option. The ability to compile old codes is very nice and I would like to keep it. Furthermore, many production Fortran codes span much longer than 10 years.

Regarding Lapack, the fact that they do not maintain Fortran modules is probably caused by most of their users being in C, C++ or Python anyway, as well as the fact that the de-facto standard regarding the API has been the F77/C style API, no modules. The Fortran user base has just not been big enough to warrant maintaining modules or to modernize the Fortran API. Also their next generation Lapack will be written in C, for that reason: https://fortran-lang.discourse.group/t/why-abandon-fortran-for-linear-algebra/1191

The solution to the Lapack issue is actually quite simple: we should maintain the modern Fortran interface to it as a community under https://github.com/fortran-lang, and make the package fpm installable. As more and more people use fpm and use such modern Fortran libraries, the demand will rise and eventually everybody will do that.

MarDiehl commented 3 years ago

@certik Would it be correct to summarize your vision as 'continue as before, just add fpm and stdlib'? I doubt that this will give Fortran a future, it at best will slow down its decline.

If Fortran is not the first choice for a library like LAPACK, for which type of projects should it be first choice?

Furthermore, many production Fortran codes span much longer than 10 years.

Yes, even my application is older than 10 years. But if someone is interested in a code, it should be possible to remove deprecated features within 10 years. Removing some features doesn't mean that one has to rewrite the whole code. It only requires to have a readable and maintained code.

When I see F77 code, I simply want to scream and run away. So I don't think any investment in supporting it is a good investment. IMHO it would be much better to spend the time on implementing new features than support old ones. I have the impression that most compiler developers are short on man power and I would prefer if they allocate their scarce resources towards the modern language. I also believe that this would make it possible to advance the standard for quickly. The intersection between features required by the community (https://isotc.iso.org/livelink/livelink?func=ll&objId=19530634&objAction=Open&viewType=1) and the features going into 202X (https://wg5-fortran.org/N2151-N2200/N2184.pdf) is basically empty. Is it wrong to assume that this happened because of compiler vendors being afraid of getting extra work that they cannot handle?

certik commented 3 years ago

Here is a short mission statement that I like:

Enable scientists, engineers, and other domain experts to write programs that naturally express the mathematics and algorithms employed, are portable across HPC systems, remain viable over decades of use, and extract a high percentage of performance from the underlying hardware.

I would argue this is the original mission of Fortran and that should still be the mission today. We can perhaps reformulate the HPC, by which I simply understand "modern computers" of the given era, including (but not limited to) the fastest available at the given time.

MarDiehl commented 3 years ago

Some comments on the aspect remain viable over decades of use and backward compatibility after yesterdays monthly call.

Why backward compatibility?

With backward compatibility I mean the fact that almost any valid F77 program is a valid F2018 program. It is frequently claimed that this type of backward compatibility is one of the features that the community loves about Fortran. But what is the community?

Since there are hardly any new Fortran projects,to me it seems that the community are people working on old projects that are not modernized.That is a pity, but it should be their problem.

What can they do?

  1. In my opinion, there is one reasonable solution: Care about your code and step-by-step re-implement functionality. Use of modern Fortran adds value to the code: readability, maintainability, robustness.
  2. There is another solution that I find less reasonable: Set the standard flag of your compiler to F77/F90 and clearly communicate to the world that you don't care about your project.
  3. The current way how Fortran standards are written provides another option: Compile with F2018 standard flag and ignore the fact that your code has reached a complexity that you cannot handle any more.

So backward compatibility just helps you to ignore the reality, where reality could mean:

The cost of backward compatibility

I think Fortran suffers from feature creep (C++ is also criticized for that). The language specification gets longer with each revision and this clearly increases the complexity. The problem is, IMHO, that the current standard has reached a complexity that cannot be handled anymore. I - not a compiler developer, but a long time (10 years) Fortran user - came to that conclusion based on the following observations:

The only solution if one cannot handle complexity is to decrease it. The current Fortran strategy is not sustainable: Slow down the increase in complexity by avoiding the addition of new features as far as possible.

Breaking changes

Breaking changes just mean that a decision was revised.

So why not simply acknowledge that and expect that in 2021 source code is not written in fixed form?

Managing breaking changes

Breaking changes are annoying. I know this because I am a user of PETSc. They do not care about backward compatibility at all: We will never keep in a bad design decision simply because changing it will require a lot of editing. By the way, reading the rest of https://www.mcs.anl.gov/petsc/documentation/faq.html#work-efficiently is also very enlightening.

This means that every time there is a new PETSc version I need to check if my code still runs. Ironically, I need to do the same when a new Fortran compiler version is released: Not because they contain intentional changes, simply because the Fortran compilers that I use tend to get new bugs at approximately the same rate that old bugs get removed.

The good point about the strategy of PETSc is the following: The changes I need to make my application are usually small, predictable, documented: Release early, release often And programmers can not only cope with these changes, they apparently like it. I think the impact of PETSc on contemporary science/engineering applications is bigger than than Fortan's impact. The PETSc user manual has 300 citations per year, more than the whole Fortran Forum journal.

Function to remove from the standard:

As outline above, I believe that cleaning up the standard to get room for improvements would be a sensible step. The following functionality came into my mind when I thought about the less valuable parts of the language:

A comment to F202X

I hope that no one from the standard committee takes this personal, but to me the addition of ACOSD, ASIND, ATAND, ATAN2D, COSD, SIND, TAND, ACOSPI, ASINPI, ATANPI, ATAN2PI, COSPI, SINPI, and TANPI is ridiculous. Even an unskilled programmer can write these functions in 1 hour. Why even bother? A multiplication with a constant has no performance implications in modern, memory bound applications. Providing a function deg2rad or even defining a constant PI (so far the best solution I found is real, parameter :: PI = acos(-1.0)) would be much more general and internally a compiler could optimize statements like sin(deg2rad(()) if there is really the need for that. This is really feature creep at its best. To me it summarizes the evolution of the standard pretty well: Add features that almost no one really needs as long as they are easy to implement.

My vision, finally

A breaking change in the Fortran strategy: Look forward, not backward. Accept that breaking changes are needed.

The risk of trying is much smaller than the risk not trying it. Fortran has little to loose but a lot to gain. Continuing as before and hope scientists/engineers to use Fortran for their projects reminds me of the saying: The definition of insanity is doing the same thing over and over again and expecting a different result.

gronki commented 3 years ago

Hello and I hope that everything is going well in these challenging times.

While I agree with your approach, there seems to be a rather big attachment to legacy fortran, especially from the industry side (if it ain't broke don't fix it). I would also love to see Fortran as a modern and faster competitor to Numpy/Matlab and easier for scientists to handle than C/C++. While I like it that my Fortran programs will likely be valid in 20 years, I feel like the decisions which features are okay to drop are rather arbitrary and probably come from large customers of compiler developers (such as government agencies). And I understand their point, after all while "cleaning up" the language would be wonderful for us enthusiasts, it also needs to keep doing the job it was intended to, or otherwise it would just lose its relevance. As many codes are being moved from Fortran to C/C++, hopefully less legacy will be holding the decision makers back from dropping rather objectively bad features of the language (common blocks etc).

Dominik

śr., 16 cze 2021 o 17:03 Martin Diehl @.***> napisał(a):

Some comments on the aspect remain viable over decades of use and backward compatibility after yesterdays monthly call. Why backward compatibility?

With backward compatibility I mean the fact that almost any valid F77 program is a valid F2018 program. It is frequently claimed that this type of backward compatibility is one of the features that the community loves about Fortran. But what is the community?

Since there are hardly any new Fortran projects,to me it seems that the community are people working on old projects that are not modernized.That is a pity, but it should be their problem.

What can they do?

  1. In my opinion, there is one reasonable solution: Care about your code and step-by-step re-implement functionality. Use of modern Fortran adds value to the code: readability, maintainability, robustness.
  2. There is another solution that I find less reasonable: Set the standard flag of your compiler to F77/F90 and clearly communicate to the world that you don't care about your project.
  3. The current way how Fortran standards are written provides another option: Compile with F2018 standard flag and ignore the fact that your code has reached a complexity that you cannot handle any more.

So backward compatibility just helps you to ignore the reality, where reality could mean:

  • Me and my team stopped learning and we only want to do things in the same way as we have always done it.
  • The code of my project is an unmaintainable mess.

The cost of backward compatibility

I think Fortran suffers from feature creep (C++ is also criticized for that). The language specification gets longer with each revision and this clearly increases the complexity. The problem is, IMHO, that the current standard has reached a complexity that cannot be handled anymore. I - not a compiler developer, but a long time (10 years) Fortran user - came to that conclusion based on the following observations:

  • It takes years until a standard is supported by at least some compilers (I use gfortran and ifort).
  • Fortran compilers are buggy: I have reported about 10 serious compiler bugs (rejection of valid code, wrong results, segmentation faults in my application, internal compiler errors) in the compilers I use (Intel, GNU) or tried to use (PGI). I never found a bug in Python which I use equally extensive.
  • 2/3 attempts to get a Fortran compiler into LLVM failed. Fortunately, the current efforts seems to be on a good way.

The only solution if one cannot handle complexity is to decrease it. The current Fortran strategy is not sustainable: Slow down the increase in complexity by avoiding the addition of new features as far as possible. Breaking changes

Breaking changes just mean that a decision was revised.

  • Was fixed form a good choice in the 1960? Probably yes!
  • Would any sensible person today design a programming language with fixed form? Certainly not!

So why not simply acknowledge that and expect source code from 2021 is not written in fixed form? Managing breaking changes

Breaking changes are annoying. I know this because I am a user of PETSc. They do not care about backward compatibility at all: We will never keep in a bad design decision simply because changing it will require a lot of editing. By the way, reading the rest of https://www.mcs.anl.gov/petsc/documentation/faq.html#work-efficiently is also very enlightening.

This means that every time there is a new PETSc version I need to check if my code still runs. Ironically, I need to do the same when a new Fortran compiler version is released: Not because they contain intentional changes, simply because the Fortran compilers that I use tend to get new bugs at approximately the same rate that old bugs get removed.

The good point about the strategy of PETSc is the following: The changes I need to make my application are usually small, predictable, documented: Release early, release often https://en.wikipedia.org/wiki/Release_early,_release_often And programmers can not only cope with these changes, they apparently like it. I think the impact of PETSc on contemporary science/engineering applications is bigger than the impact of Fortran. It's user manual has 300 citations per year. Function to remove from the standard:

As outline above, I believe that cleaning up the standard to get room for improvements would be a sensible step. The following functionality came into my mind when I thought about the less valuable parts of the language:

  • fixed form: Belongs into the museum, together with punch cards.
  • code not included in modules/submodules: The compiler of a strongly and statically typed language should be able to tell the programmer if a function was called with the wrong arguments. Immediately. Always.
  • name-list IO: Not flexible enough for many realistic use cases.
  • common blocks: Global variables are the best option for ruining code.
  • data statement: The only thing it seems to add is partial initialization of arrays.
  • equivalence: Now there are pointers.
  • co-arrays: I like the feature, but I'm not aware of any serious application using it. Not much bang for the buck.

A comment to F202X

I hope that no one from the standard committee takes this personal, but to me the addition of ACOSD, ASIND, ATAND, ATAN2D, COSD, SIND, TAND, ACOSPI, ASINPI, ATANPI, ATAN2PI, COSPI, SINPI, and TANPI is ridiculous. Even an unskilled programmer can write these functions in 1 hour. Why even bother? A multiplication with a constant has no performance implications in modern, memory bound applications. Providing a function deg2rad or even defining a constant PI (so far the best solution I found is real, parameter :: PI = acos(-1.0)) would be much more general and internally a compiler could optimize statements like sin(deg2rad(()) if there is really the need for that. This is really feature creep at its best. To me it summarizes the evolution of the standard pretty well: Add features that almost no one really needs as long as they are easy to implement. My vision, finally

A breaking change in the Fortran strategy: Look forward, not backward. Accept that breaking changes are needed.

The risk of trying is much smaller than the risk not trying it. Fortran has little to loose but a lot to gain. Continuing as before and hope scientists/engineers to use Fortran for their projects reminds me of the saying: The definition of insanity is doing the same thing over and over again and expecting a different result.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/j3-fortran/fortran_proposals/issues/59#issuecomment-862457561, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC4NA3PY4MLG7HHFXVE7JNLTTC4M7ANCNFSM4JFUVZ6A .

certik commented 3 years ago

Thanks @MarDiehl for writing this up. There are a lot of threads, I'll try to answer them. Overall, what you described is incredibly frustrating, and that is why I decided to get involved and see if I can help fix this.

I hope that no one from the standard committee takes this personal, but to me the addition of ACOSD, ASIND, ATAND, ATAN2D, COSD, SIND, TAND, ACOSPI, ASINPI, ATANPI, ATAN2PI, COSPI, SINPI, and TANPI is ridiculous. Even an unskilled programmer can write these functions in 1 hour. Why even bother?

If it was me, I would not put these in either! The argument that I heard was that sin(pi) might not be exactly 0, and that if you want the degree functions to return exactly zero, it requires some special implementation. To which I would say, in the rare cases when a simple multiplication by pi/180 is not enough, we can have more accurate versions in stdlib. This should not go into the standard.

Long term solution: involve the community in these decisions what should and should not get standardized. Functions like these should go into stdlib first, get some usage. If there is large and documented interest in the community, we can think about adding them to the standard later.

Function to remove from the standard:

I think these can be implemented in a compiler to give warnings when you use it. I plan to do this in LFortran. I agree with your list except co-arrays. Co-arrays are not used currently much. But they are not bad as a feature and once compilers support them reliably on all platforms, I think this allows to write more readable parallel programs. I think the jury is still out.

Managing breaking changes

I think the way to manage breaking changes is along the lines of some of the proposals in #83.

However, with good default compiler warnings (or even errors) to effectively disable things like implicit typing, there might be just a few things that would need breaking. If there is a way to figure out how not to make any breaking changes, that is always preferable. But if we want to introduce something that would not be compatible, doing it via the langauge or edition keyword would ensure that old code continues working.

Once LFortran works, I am happy to brainstorm with you and others what (possibly breaking) changes we would like to see and prototype them in the compiler. Then we can have a productive discussion and experience and evaluate the pros and cons.

Fixed form

I still like the idea of using .f for free form and that can be done via new compilers as well as fpm (https://github.com/fortran-lang/fpm/issues/363). Yes, we should not use fixed form for new codes, I think everybody agrees.

Quality of Fortran compilers

As a user I wish that Fortran compilers were better. So I decided to start LFortran and see if I can do it better from the perspective of both organizing the development by the community as well as arriving at a technically high quality product. This can't be done overnight, this is a long term project, an investment. As I mentioned on the call, two years ago I was asking people to give us 5 years to deliver. So 3 years from now, summer 2024. To clarify: we are shooting to compile first projects this summer, so I am hoping we can deliver much sooner. But a conservative estimate for a new compiler to deliver in 5 years is reasonable, in fact people usually say 10 years.

@certik Would it be correct to summarize your vision as 'continue as before, just add fpm and stdlib'? I doubt that this will give Fortran a future, it at best will slow down its decline.

That is not my vision. I agree that just adding stdlib and fpm is not enough, that would only slow down (at best) the decline, exactly as you said. We need a high quality compiler or compilers, we need a good community, we need to start seeing lots of new projects being started in Fortran, and Fortran needs to be able to compete on both technical and social level with the alternatives. (We also need the Standard Committee to work more with the community, to get a community buy-in and agreement with the direction where things are heading --- we made big progress on this too, as evidenced by this very discussion on the "incubator repository", but I do not consider the problem fixed yet.)

Also: compilers should drive the development. Let's brainstorm and implement prototypes or extensions for new features and directions. Only later let's standardize.

If you (or anyone else!) want to chat more about this, I am happy to do a video call anytime.

klausler commented 3 years ago

@MarDiehl wrote:

Accept that breaking changes are needed.

I simply don't understand this conclusion at all. Older features that you don't like aren't slowing progress. If you don't want to use the older features, then don't use the older features. If you want to delete them from the standards, that just makes their documentation harder to find. If you want compilers to delete the older features, you're in for a disappointment; we all have users (& paying customers, for commercial products) that would be affected, and there's no advantage to be gained for us for removing working features.

Backward compatibility is not what's dooming Fortran. Backward compatibility is not at the expense of new features. The reason you don't have templates or parametric modules or threading and memory models or multicore/GPU support in the standard language is not because the standard still has fixed form!

certik commented 3 years ago

If you want compilers to delete the older features, you're in for a disappointment; we all have users (& paying customers, for commercial products) that would be affected, and there's no advantage to be gained for us for removing working features.

Not all compilers, for LFortran I want to have an optional mode where the compiler warns (or perhaps even gives errors) for features that you don't want to use. This would be super helpful to ensure your (new) project does not accidentally use some older features. This was one of the selling points of Python, that there was just "one obvious way" to do things. One can achieve something similar if the compiler "encourages" (via warnings and errors) just "one obvious way". (We can discuss if we can agree on such a subset as a community, or whether this might need to be configurable per project.)

MarDiehl commented 3 years ago

@MarDiehl wrote:

Accept that breaking changes are needed.

I simply don't understand this conclusion at all. Older features that you don't like aren't slowing progress.

The logic is the following: 1) The standard is large and complicated. 2) This makes implementation of the standard difficult. 3) New features make the standard even larger and more complicated. 4) It would make sense to sacrifice some old features in favor of new ones.

@klausler: What is than slowing down the progress in your opinion?

If you don't want to use the older features, then don't use the older features.

Yes, I certainly don't use them. But I'm affected by the poor quality of the compiler and the slowly evolving language.

If you want to delete them from the standards, that just makes their documentation harder to find.

It would even be good if people don't find concepts like fixed form if they google for Fortran because it will scare them away. But those who insist on using old style can simply get a copy of the old standard.

If you want compilers to delete the older features, you're in for a disappointment; we all have users (& paying customers, for commercial products) that would be affected, and there's no advantage to be gained for us for removing working features.

I don't suggest to remove features without notification. But if there is a clearly communicated roadmap and sufficient time, people can adopt. I know that this requires work, but continuous modernization is the only way to prevent a code from getting unmaintainable legacy.

As I wrote, I am not a compiler developer, but as a Fortran developer I'm not satisfied with the quality of the compilers that I have tried (Intel, GNU, PGI) and I also see that even F202X is missing features that would make my life easier.

Setting priorities needs to be done in every software project. One cannot implement every possible feature because every feature comes at the price of complexity. Backward compatibility was always top priority of Fortran. But the result is obvious: Fortran is popular among old projects that benefit from backward compatibility, but new projects rarely choose Fortran. So I agree with @klausler, backward compatibility is on short term the right strategy. But it is pretty obvious that it is not a future-proof long term strategy to simply milk the cash cows of today and don't care about tomorrows customers.

gronki commented 3 years ago

I think everyone agrees that removing features is good and it actually happens in the standard. There have been many deleted features. I guess the disagreement is about how fast this process happens. Am I correct?

Dominik

śr., 16 cze 2021, 20:17 użytkownik Martin Diehl @.***> napisał:

@MarDiehl https://github.com/MarDiehl wrote:

Accept that breaking changes are needed.

I simply don't understand this conclusion at all. Older features that you don't like aren't slowing progress.

The logic is the following:

  1. The standard is large and complicated.
  2. This makes implementation of the standard difficult.
  3. New features make the standard even larger and more complicated.
  4. It would make sense to sacrifice some old features in favor of new ones.

@klausler https://github.com/klausler: What is than slowing down the progress in your opinion?

If you don't want to use the older features, then don't use the older features.

Yes, I certainly don't use them. But I'm affected by the poor quality of the compiler and the slowly evolving language.

If you want to delete them from the standards, that just makes their documentation harder to find.

It would even be good if people don't find concepts like fixed form if they google for Fortran because it will scare them away. But those who insist on using old style can simply get a copy of the old standard.

If you want compilers to delete the older features, you're in for a disappointment; we all have users (& paying customers, for commercial products) that would be affected, and there's no advantage to be gained for us for removing working features.

I don't suggest to remove features without notification. But if there is a clearly communicated roadmap and sufficient time, people can adopt. I know that this requires work, but continuous modernization is the only way to prevent a code from getting unmaintainable legacy.

As I wrote, I am not a compiler developer, but as a Fortran developer I'm not satisfied with the quality of the compilers that I have tried (Intel, GNU, PGI) and I also see that even F202X is missing features that would make my life easier.

Setting priorities needs to be done in every software project. One cannot implement every possible feature because every feature comes at the price of complexity. Backward compatibility was always top priority of Fortran. But the result is obvious: Fortran is popular among old projects that benefit from backward compatibility, but new projects rarely choose Fortran. So I agree with @klausler https://github.com/klausler, backward compatibility is on short term the right strategy. But it is pretty obvious that it is not a future-proof long term strategy to simply milk the cash cows of today and don't care about tomorrows customers.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/j3-fortran/fortran_proposals/issues/59#issuecomment-862604183, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC4NA3NITIKEKM6ZUDTKJVLTTDTDBANCNFSM4JFUVZ6A .

klausler commented 3 years ago

You seem to think that a compiler has some fixed maximum number of features that it can support, and once that limit is reached, that each new feature added requires an existing feature to be deleted. That is not the case.

MarDiehl commented 3 years ago

@certik: Many thanks for your detailed reply. As mentioned yesterday, I really find your work on building a community (stdlib, https://fortran-lang.org) for Fortran amazing. I totally agree that involving the community is something that pushes the language forward. Apparently, Fortran users and Fortran compiler developers have different interests and I'm willingly to express my interests as a Fortran user. Regarding co-arrays: This was is certainly an interesting feature and I was hesitating to put it on the list. But from an outside-view, it seems to be a rather complicated feature with (at least currently) a few users. Maybe @klausler can enlighten us about NVIDIAs plans for co-array support.

MarDiehl commented 3 years ago

You seem to think that a compiler has some fixed maximum number of features that it can support, and once that limit is reached, that each new feature added requires an existing feature to be deleted. That is not the case.

Yes, that is what I believe. But I forgot to mention that this maximum is not set by the language itself but by the capabilities of the compiler vendor. Some, like NVIDIA, struggle to include the F08 standard while others, like Intel, found a way to increase the number of features until F18 fits in.

certik commented 3 years ago

Thank you @MarDiehl.

Apparently, Fortran users and Fortran compiler developers have different interests and I'm willingly to express my interests as a Fortran user.

I think this perception comes from the fact that historically the communication between compiler vendors, users and the standards committee members was not always optimal. I think we have greatly improved the situation.

Second, I have been trying to ensure the goals of LFortran are aligned with the users (myself included!). More generally, I think we should have a community maintained compiler(s) as part of fortran-lang (precisely to ensure that it does what Fortran users want and that the community has a "buy-in"), and I'll be happy to if LFortran would eventually become one.

FortranFan commented 3 years ago

@MarDiehl wrote Jun. 16, 2021, 2:21 PM EDT:

I also see that even F202X is missing features that would make my life easier.

I apologize for not being able to understand your description of issues, I'm running into difficulties with your write-up above that advocates features you find backward or not useful removed e.g., coarrays. But then you express interest in items you find convenient ("make my life easier") introduced in the upcoming and future revisions. I don't know how to reconcile what comes across in your statements to readers who are remote from you and how to drive toward consensus across different needs. Perhaps you can elaborate further on your points?

FortranFan commented 3 years ago

@septcolor wrote jun. 16, 2021 2:19 PM EDT:

Is it not possible to "feature freeze" the current Fortran (or possibly F202X), while continuing the development of the "successor" with breaking changes ..

@septcolor , please note this is happening out there in the real world all the time and has picked up rapid pace in the last 5-6 years! Just that the "successors" are modern C++, C#, Java, Julia, Python, etc.

MarDiehl commented 3 years ago

@MarDiehl wrote Jun. 16, 2021, 2:21 PM EDT:

I also see that even F202X is missing features that would make my life easier.

I apologize for not being able to understand your description of issues, I'm running into difficulties with your write-up above that advocates features you find backward or not useful removed e.g., coarrays. But then you express interest in items you find convenient ("make my life easier") introduced in the upcoming and future revisions. I don't know how to reconcile what comes across in your statements to readers who are remote from you and how to drive toward consensus across different needs. Perhaps you can elaborate further on your points?

sorry, it is true that my thoughts are not easy to understand.

In short, I propose to remove language features for which modern alternatives exist. I don't go into detail about new features that I like to add because the https://github.com/j3-fortran/fortran_proposals/issues contains already a long list. Personally, I find for example https://github.com/j3-fortran/fortran_proposals/issues/1, https://github.com/j3-fortran/fortran_proposals/issues/172, https://github.com/j3-fortran/fortran_proposals/issues/70, and https://github.com/j3-fortran/fortran_proposals/issues/16 relevant.

MarDiehl commented 3 years ago

Second, I have been trying to ensure the goals of LFortran are aligned with the users (myself included!). More generally, I think we should have a community maintained compiler(s) as part of fortran-lang (precisely to ensure that it does what Fortran users want and that the community has a "buy-in"), and I'll be happy to if LFortran would eventually become one.

Yes, having a community that actively participates in pushing the standard seems to be very important. Python and Rust do that very successfully. And the list of issues on the J3 repository is already quite long.

FortranFan commented 3 years ago

See this comment at #108.

My view is as follows:

At some point, Fortran has to be open to certain targeted deletions from the standard. The adverse impact I see from requiring implicit none in every program unit and interface body; from implicit save. and the continued need to have new features integrate with obsolescent ones are pernicious. It's rather unfortunate there is no acceptance of this reality, it's total denial.

Newer languages that are getting embraced and employed far more readily in technical computing, particularly in industry, are able to achieve exemplary feature introductions in about 4 years that can take decades with Fortran. Sure, there are multiple root causes to this, however certain thorns in the attitude cannot be ignored.

This is my primary reason to hope for and nudge the Fortran Community to become more aware and vigilant and brave and absolutely demanding with at least some targeted deletions from the modern Fortran standard, in opposition to certain WG5 members and certain vendors.

Otherwise, progress will really be held back and the old features will simply remain a millstone around the neck of poor, persevering, practitioners striving to use modern Fortran even. And who might as well then just wrap their existing creations of Fortran codebase in libraries invokable from other programming languages and migrate away from Fortran. Vast swaths of domains elsewhere, particularly in industry, have done so and doing more of this.

This is again a vision thingy for the language, it's entirely lacking in some key spaces among the language bearers.

dacarnazzola commented 1 year ago

I know this issue hasn't seen any activity lately, but I believe that is a mistake. I find myself mostly in agreement with @FortranFan and @MarDiehl as far as stated opinions, and also with @everythingfunctional that Fortran needs core values. The original purpose of FORTRAN was to provide a high level language that was capable of yielding better performance than hand written assembly. If it is deemed no longer possible for Fortran to provide best-in-class performance, then I ask what purpose does the language serve? Many industries have large (but shrinking) legacy codebases, yet zero new projects beyond the scope of 100 lines are being started in Fortran. This includes for codes deemed "high performance." This quote from @MarDiehl sums up the situation brilliantly:

If Fortran is not the first choice for a library like LAPACK, for which type of projects should it be first choice?

I like Fortran. I want the language to succeed. I also want to write code that executes quickly with acceptable accuracy. SIMD intrinsic functions are here. Massively parallel accelerators (including GPUs) are here. Fortran has no in-language access to either of these. I must agree that co-arrays are less than desirable. Gfortran requiring external libraries to execute more than one image is a joke of a "language intrinsic parallel feature." At least ifort supports them (I think?) and OneAPI is free now. Speaking of, this reminds me of DO CONCURRENT, which despite it's name, is does not execute anything concurrently, except via nvfortran, only on nvidia GPUs. Cool?

I believe the core values of Fortran should be as such: 1) Performance - if Fortran cannot maintain class leading performance, there is no reason to accept the numerous limitations of the language (terrible string handling, no web interoperability, no graphic display capability, no unsigned types, limited bit manipulation, extremely limited GPU/accelerator offload)

I really don't have any others. I was writing something about backwards compatibility, but no. At this point, the "maintain backwards compatibility at all costs" attitude is rapidly approaching "all costs" and will lead to death and disuse of the language. No new projects are started, year after year legacy code is converted to C++ (or other language, but most "large" projects I see in industry are all C++ at the moment, if they deem a compiled language necessary). Eventually the remaining legacy project maintainers will retire and that will be that, goodbye Fortran.

The language must continue to advance and modernize. Personally, I do not use any OO (I think? I like interfaces and would like them expanded to allow more generic, compile time programming), but it is a programming model that is expected of a modern language. By now there have also been many examples of successful OO models, and we know which features are actually used/useful, and which are nothing but pain and suffering. Ideally, Fortran should support OO programming to the extent that it is useful while maintaining the primary value - performance.

certik commented 1 year ago

@dacarnazzola I agree with your overall sentiment.

In my personal opinion, the only way to fix this is by developing the compiler(s) and tooling to achieve all the things you mentioned (and more!). No amount of talk will fix it. I personally believe LFortran is one such way (although not the only way) that can fix all these things once delivered, here is our current status: https://lfortran.org/blog/2023/05/lfortran-breakthrough-now-building-legacy-and-modern-minpack/, and I invite everybody to join us to get there faster. I agree, performance (and all hardware, including GPUs) is the key, no compromise. Right now my personal focus is on compiling most codes, but as you can see in the blog post, our performance on Minpack seems to be within 2x from gfortran (as close as just 25% slower on some platforms), and once we can compile, say, 10 medium size 3rd party codes, we'll focus on optimizations, as well as running on all hardware. If anybody here is interested to help out, please get in touch with me. Disclaimer: This is just my opinion. There are many other Fortran compilers: https://fortran-lang.org/en/compilers/ that people can also contribute to.