Proposal for Workflow

[certik]

Ideal Plan

Here is a proposal that I would like to get feedback on and discuss. This is my idea how this GitHub repository can work.

How Opensource Works

Successful open source projects, such as SymPy that I started 13 years ago, have:

• one or two owners/leaders
• 10-20 core developers with push access, those people can merge GitHub Pull Requests (PRs)
• hundreds of people who contribute PRs
• thousands of people who open issues and discuss them

How the Fortran Committee Could Work

The Fortran Standards document is compiled from LaTeX source code, and proposals that eventually get approved must be of very high quality and must succeed multiple rounds of approvals. Fundamentally this is no different to opensource development.

Here is how it could work:

• The J3 committee chair and the WG5 committee chair are the two owners/leaders

• The J3 and WG5 comittee voting members are the core developers, who approve proposals in the same way they currently do (the proposal must be submited at the J3 website, then advocated at the committee, and eventually approved).

• Hundreds of people use this GitHub repository to submit PRs, where each PR is a particular proposal. We all collaborate on the PRs, and if it gets merged, it means that the proposal is of very high quality and is worth for the committee to spend their time to consider it. We use issues to track how the proposal gets processed by the committee and either approved or rejected. We send further PRs to improve upon it based on the committee feedback.

• Thousands of people open issues to propose new features and discuss them.

• Anybody can create a PR to try to create a proposal. Only members of the committee have push access (are allowed to merge PRs) and if they merge a PR, they are responsible to advocate for it at the committee and collect the commitee's feedback.

• The committee itself does not need to change at all how it works, members of the committee who do not have time or interest to be involved at GitHub do not need to. For this plan to work, just several members of the committee need to be actively involved (we already have 3 or 4 members who are involved, and that is enough to get us started). As this GitHub repository becomes more popular, I can imagine eventually all members of the committee to at least read some of the responses and activity and what issues are popular.

• The whole community works on PRs and essentially works nonstop (with hundreds of people in all time zones one gets a new PR every couple days, as in SymPy). That way, when the committee meets, it has high quality proposals to consider, and it does not need to spend precious committee time to develop high quality proposals, because that work is outsourced to the wide community.

• The core developers (committee members) are the ones who ultimately decide what goes in, and what does not. Just like now. The only thing that changes is that the workflow becomes more efficient.

---

Some lessons from open source development that will apply here:

• If we are successful, we will get thousands of issues from people all over the world. Some good, some not as high quality. People will discuss issues. Initially, many will be frustrated at how slow the committee works. The best reaction that we can do is to encourage people to create PRs and try to create high quality proposals for the issue(s) that they care about. And invite them to collaborate on the PRs.

• Experience shows that this scales really well. We build a team. People start feeling positive about the future, because their voice is heard, and their work (in PRs) is used and their (small or large) contributions help shape the future of Fortran.

[certik]

The other issue that I think this workflow can fix is that the committee carefully discusses each proposal in a small group and then in the whole committee in person. But many of the objections and questions and back and forth are more efficient to discuss in a GitHub issues and PRs, so that we can calmly think about each question and response and then collaborate to make the proposal better. So that when the committee meets in person, the simple stuff is already done, all the pros / cons are written up, and we can spend our time weighting the alternatives, as opposed to wasting time discussing things that could have been done beforehand at GitHub (such as that the proposal needs more examples).

I've seen how the committee works and the most common objections, and I can now ask a lot of them myself and ensure each proposal meets a minimal level of quality here at GitHub.

A related problem is that the committee only has a few members who are true experts on almost everything in the Fortran language. Most, including myself, only know some parts of Fortran really well, and not so some other parts. So by getting feedback from lots of people about a proposal at GitHub, asking lots of questions and discussing it here, we can iteratively improve the proposal to the level that an expert committee member would write right away. I have seen that happen just last week --- we spent an hour discussing pros and cons of various approaches, eventually agreeing on an approach that neither of us liked at first, but after doing the work, it seemed like the best solution --- and in the meantime an expert committee member sent us an email with exactly the same solution, that he intuitively figured out on his own.

[tclune]

I think it best that this GitHub project serve as a mechanism for refining concepts into papers submitted to the committee. Any suggestion (as above) for a process that requires active participation by committee members to manage PR's is problematic for reasons I will not discuss.

I think it is also very important to emphasize that the new feature list for F202x is closed. Issues opened here should either identify a specific item from the F202x in their subject line or clearly specify that the request is for a new feature in F202y.

[certik]

The above proposal does not require active participation by committee members beyond those already participating. And that's why I really like it.

On Sun, Oct 20, 2019, at 2:45 PM, Tom Clune wrote:

I think it best that this GitHub project serve as a mechanism for refining concepts into papers submitted to the committee. Any suggestion (as above) for a process that requires active participation by committee members to manage PR's is problematic for reasons I will not discuss.

I think it is also very important to emphasize that the new feature list for F202x is closed. Issues opened here should either identify a specific item from the F202x in their subject line or clearly specify that the request is for a new feature in F202y.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/j3-fortran/fortran_proposals/issues/26?email_source=notifications&email_token=AAAFAWGTQFTQGKSL4XO4R7TQPTGQFA5CNFSM4JCR34LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBYUYTQ#issuecomment-544296014, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAFAWGA32HNV4WJNWKJWZDQPTGQFANCNFSM4JCR34LA.

[certik]

The other comment is that it's not enough to just write a proposal. One has to write a proposal for every meeting, keep track of it, react and discuss comments to it, etc. So creating proposals is a big part of it, but I want this repository to be a lot more than that. I would like it to become the outward facing interface to the committee.

Let me reiterate that this doesn't require current committee members to change how they work. All it takes is for a few members to participate here and to keep the issues in sync with what the committee is doing -- and we already have that. We have 3-4 members already participating here.

On Sun, Oct 20, 2019, at 8:34 PM, Ondřej Čertík wrote:

The above proposal does not require active participation by committee members beyond those already participating. And that's why I really like it.

On Sun, Oct 20, 2019, at 2:45 PM, Tom Clune wrote:

I think it best that this GitHub project serve as a mechanism for refining concepts into papers submitted to the committee. Any suggestion (as above) for a process that requires active participation by committee members to manage PR's is problematic for reasons I will not discuss.

I think it is also very important to emphasize that the new feature list for F202x is closed. Issues opened here should either identify a specific item from the F202x in their subject line or clearly specify that the request is for a new feature in F202y.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/j3-fortran/fortran_proposals/issues/26?email_source=notifications&email_token=AAAFAWGTQFTQGKSL4XO4R7TQPTGQFA5CNFSM4JCR34LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBYUYTQ#issuecomment-544296014, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAFAWGA32HNV4WJNWKJWZDQPTGQFANCNFSM4JCR34LA.

[milancurcic]

I overall like this proposal for workflow a lot. My impression of this repository at its conception (even before reading this issue) was that it would allow the community to propose and discuss changes to the language in an organized, centralized, and streamlined way. GitHub provides tools to do exactly that.

Maintainers of the repository being committee members allows them to communicate these proposals to the rest of the committee and have them considered for adoption. As Ondrej put it, it's an outward facing interface to the committee, and that's how I understood it from the get go.

Nothing about the workflow seems to suggest about how the committee should operate -- that's already in place. Other committee members may or may not choose to participate. However I'd be surprised if most chose not to participate despite this repo churning out a few high-quality proposals or more. Community needs not fight the committee. It just needs to create enough great stuff that it'd be hard to ignore it.

I suggest to carefully put together a separate document in this repo (perhaps GUIDELINES.md) that would describe in detail how can Joe Fortran Programmer who stumbles upon this repo contribute a high quality proposal that could land onto the committee table at a meeting. Some is already in README.md (great work), and a nice proposal layed out by Ondrej in this issue.

[certik]

@milancurcic thanks a lot for posting your opinion. I agree with you. I was thinking of putting this into the README itself, but we can certainly have a separate file such as GUIDELINES.md for this also and just reference it from the README. I'll try to create a PR for this and we can refine the wording at the PR.

[FortranFan]

@certik I've been thinking about what you asked in the comments section of #61 and it's unclear to me as to the content and value of the two documents you seek.

Before committing to any further work myself to develop documents toward the workflow, what I would find useful first is a honest discussion with WG5 to help answer the following basic questions:

1) For whom Fortran?

2) Does WG5's answer to question (1) truly and directly include the needs and interests of many non-institutional practitioners like @rweed, @Beliavsky, or Jean, etc.? If not, to what extent does WG5 think it is a problem?

3) If the answer to 2) is yes, other than the 2018 survey what does WG5 do to listen to these practitioners on a continual basis?

4) What is up with "national bodies" in WG5 in this day and age?

5) If at all there must be national bodies - like US (with J3), UK, and Japan, to what extent are non-institutional practitioners residing in these nations represented in these 3 bodies?

6) What about other national bodies or lack thereof? What about a Fortranner in Brazil or France or Germany or India or Nigeria and how do these Fortranners' inputs get attention from WG5, particularly if they do not have any institutional support toward any ISO IEC advocacy or charter or means to get such support? This goes back to question 1.

7) Many of the practitioners in 2) are unhappy with the slow pace of Fortran language advancement and their feedback in WG5 survey appears to be mostly ignored in Fortran 202X work-list per J3, as seen in comp.lang.fortran threads, example here and here. What does WG5 plan to do about this?

8) Now the two national bodies of UK and Japan have proposed further slowing down the pace and scope of Fortran standard revision due to insufficient processor support - this runs counter to majority views in WG5 survey as well as UK survey and comp.lang..fortran threads and now GitHub. What does WG5 plan to address such differences in opinion? See question 9 below re: this.

9) If WG5 accepts at face value the proposals by UK and Japan, what are the criteria in terms of processor support toward current standard 2018? How many processors must offer Fortran 2018 support to consider it good coverage? Is it 2 or 5 or 10? Which ones?

10) If there are practitioners of Fortran who wish for faster standardization of Fortran language features now and/or have needs for inclusion of more items than that intended by J3 in the work-list for Fortran 202X, what does WG5 itself propose that can be pursued to address such a need?

Bottom-line: I am strongly inclined to pose the above basic but important questions and user needs with WG5 and give WG5 some space and time to respond, rather than develop any proposals or documents toward any plans or workflows at this stage.

[certik]

From a practical perspective, I see only two separate issues:

• How to make WG5 and J3 more efficient, and I think I know how: consider every submitted proposal, work between sessions, shorter the Fortran standard iteration, etc.

• Get compiler vendors up to speed with the standard. There I don't have many suggestions beyond the efforts already underway, such as MLIR that, if successful, would allow compiler vendors to reuse most of the machinery, and each vendor just has to maintain its own frontend.

The second point is very important, I agree that if compiler vendors can't keep up, then that's a problem.

[FortranFan]

From a practical perspective, I see only two separate issues:

• How to make WG5 and J3 more efficient ..
• Get compiler vendors up to speed with the standard ..

Consider the first issue, "How to make WG5 and J3 more efficient". To be blunt and frank, gaining of a genuine acceptance among the top influencers on WG5 and J3 of such a need i.e., to be "more efficient" is in and of itself a HUUUGE challenge. And it will only be by a broad consideration of Diversity (of thought in terms of needs of many coders) and concern toward Inclusivity of a wider group of Fortranners to ensure their voices are consistently and sufficiently represented that WG5 and J3 can come to support and embrace the approach to do more faster. It is with this in mind the 10 or more questions like the ones upthread become important. Without a discussion along such lines and contemplation toward the underlying, a focus on efficiency will entirely lack any passion nor gain any urgency - things will just remain the way they are and circular arguments will defeat any change.

Note there is a certain mass of Fortranners in the "national" agencies and in other institutional areas and also certain sectors elsewhere for whom status quo is either "just fine" or who even question or oppose revisions beyond FORTRAN 77 (but use extensions) or Fortran 90 (but want some HPC features , or especially Fortran 95 (but want improvements to ALLOCATABLEs). The needs of those in this "camp" are adequately met by the current situation. So then one can argue why bother changing anything, things are good as they are.

Consider ISO/IEC JTC1/SC22/WG23 - Programming Language Vulnerabilities and what they write about Fortran in their document: in their last section 8 Implications for standardization, their last sentence is, "Identifying, deprecating, and replacing features whose use is problematic where there is a safer and clearer alternative in the modern revisions of the language or in current practice in other languages."

It is only when one is willing to look at Fortran language more broadly and by paying close attention to "other" coders (besides the institutional ones) and "other" languages more intently one can have the motivation and impetus to be "more efficient". This is sorely missing at present with Fortran even if there is some superficial recognition of it.

[certik]

I think it's ok to have a very high bar for new features, especially big features like OO programming, templates or exceptions. I have a very high bar myself. But what is not ok is not to consider popular proposals, and alternatively consider proposals that do not have wide support. I think we need the committee to seriously consider any high quality proposal that we will submit. With just that we should be able to get what we want, and the committee does not need to change much. Then we'll go from there.