Write section "Proposed Additions and Extensions"

[GoogleCodeExporter]

The language appendix "Proposed Additions and Extensions" is outdated and needs 
to be re-written to reflect our new process with the optional and required 
library specs.

Original issue reported on code.google.com by danbonachea on 17 Jun 2012 at 8:19

[GoogleCodeExporter]

Jim seems to be the most interested party for specifying the process of spec 
change, so I nominate him to draft an updated version of Appendix A for spec 
1.3. This is a "critical" priority, because the current wording makes no sense 
under the new document split arrangement.

Notable points to be updated:
* Appendix A no longer contains libraries, nor is the core library in the 
language document subject to extension.
* Extension libraries first appear in the optional library document, subject to 
the initial set of criteria.
* After satisfying the secondary set of criteria they can be promoted from the 
optional to required document.
* Extension libraries define not only a feature macro but also a dedicated 
header file.
* Need a new section documenting our procedure for localized changes to any of 
the documents (from the Wiki).

Original comment by danbonachea on 17 Aug 2012 at 12:08

• Added labels: Priority-Critical
• Removed labels: Priority-Medium

[GoogleCodeExporter]

This appendix is probably also the best place to cross-reference the required 
and optional library documents, whose existence is not currently mentioned 
anywhere in the main language spec document.

Original comment by danbonachea on 17 Aug 2012 at 12:14

[GoogleCodeExporter]

Set default Consensus to "Low".

Original comment by gary.funck on 19 Aug 2012 at 11:26

• Added labels: Consensus-Low

[GoogleCodeExporter]

Change Status to New: Requires review.

Original comment by gary.funck on 19 Aug 2012 at 11:37

• Changed state: New

[GoogleCodeExporter]

I guess this is what I get for speaking up.  :)  Sure, I can lead the effort to 
update Appendix A.

Original comment by james.di...@gmail.com on 20 Aug 2012 at 8:34

[GoogleCodeExporter]

Bumping this as a reminder - we discussed this in the call and agreed it needs 
to be rewritten.
Jim are you working on replacement text?

Original comment by danbonachea on 26 Sep 2012 at 8:16

[GoogleCodeExporter]

Thanks for the reminder.  I'll bump this up my to-do list.

Original comment by james.di...@gmail.com on 28 Sep 2012 at 6:52

[GoogleCodeExporter]

Jim has indicated he won't have time available to handle UPC matters for the 
next month, so barring any other volunteers I'll take over this issue.

Original comment by danbonachea on 5 Oct 2012 at 8:55

• Changed state: Accepted

[GoogleCodeExporter]

Miscommunication, I can still handle this ticket.

Original comment by james.di...@gmail.com on 5 Oct 2012 at 6:08

[GoogleCodeExporter]

Here is the proposed change to this section.  With this change, we would 
include language changes in Appendix A and library changes in the optional 
libraries section:

{{{
Index: upc-lang-extensions.tex
===================================================================
--- upc-lang-extensions.tex (revision 148)
+++ upc-lang-extensions.tex (working copy)
@@ -3,14 +3,17 @@
 \index{proposed extensions}

 \np This section contains proposed additions and extensions to the UPC
-    specification.  Such proposals are included when stable enough for
+    language specification.  An optional library specifications section
+    is included, which contains proposed additions and extension to the
+    UPC library specification.  Such proposals are included when stable enough 
for
     developers to implement and for users to study and experiment with
     them.  However, their presence does not suggest long term support.
     When fully stable and tested, they will be moved to the main body of
     the specification.

 \np This section also describes the process used to add new items to the
-    specification, which starts with inclusion in this section.  Requirements
+    specification, which starts with inclusion in this section or the optional
+    libraries section.  Requirements
     for inclusion are:\footnote{These requirements ensure that most of
     the semantic issues that arise during initial implementation have
     been addressed and prevents the accumulation of interfaces that no
@@ -36,7 +39,7 @@
     specification are:

 \begin{enumerate}
-\item Six months residence in this section.
+\item Six months residence in this section or the optional libraries section.

 \item The existence of either one (or more) publicly available "reference" implementation 
 written in standard UPC OR at least two independent implementations (possibly specific 
}}}

Original comment by james.di...@gmail.com on 5 Oct 2012 at 7:09

• Changed state: Started

[GoogleCodeExporter]

This proposed change is missing several of the points raised in comments 1 and 
2.

Original comment by danbonachea on 5 Oct 2012 at 7:29

[GoogleCodeExporter]

From comments #1-#2:

* Appendix A no longer contains libraries

Should be covered with the change in comment 10.

* Nor is the core library in the language document subject to extension.

Can you explain what you mean by this?

* Extension libraries first appear in the optional library document, subject to 
the initial set of criteria.

This is covered.

* After satisfying the secondary set of criteria they can be promoted from the 
optional to required document.

This is covered.

* Extension libraries define not only a feature macro but also a dedicated 
header file.

This needs to be added.

* Need a new section documenting our procedure for localized changes to any of 
the documents (from the Wiki).

Can you explain what you mean by this, or provide a link?  From reading 
Appendix A, it seems like changes to the main body of the spec need to be 
included in Appendix A itself before they can be moved into the main body of 
the text.  This is horrifying, but it seems to be what is stated by the 
existing text.

* This appendix is probably also the best place to cross-reference the required 
and optional library documents, whose existence is not currently mentioned 
anywhere in the main language spec document.

This is pretty much the worst place to present the separation of libraries from 
the main document and the split into optional and required chapters.  This 
should be done at the beginning, rather than at the end of the document.

Original comment by james.di...@gmail.com on 5 Oct 2012 at 7:39

[GoogleCodeExporter]

Here is an updated change, which includes the statement about header files:

--snip--

Index: lang/upc-lang-extensions.tex
===================================================================
--- lang/upc-lang-extensions.tex    (revision 148)
+++ lang/upc-lang-extensions.tex    (working copy)
@@ -3,14 +3,17 @@
 \index{proposed extensions}

 \np This section contains proposed additions and extensions to the UPC
-    specification.  Such proposals are included when stable enough for
+    language specification.  An optional library specifications section
+    is included, which contains proposed additions and extension to the
+    UPC library specification.  Such proposals are included when stable enough 
for
     developers to implement and for users to study and experiment with
     them.  However, their presence does not suggest long term support.
     When fully stable and tested, they will be moved to the main body of
     the specification.

 \np This section also describes the process used to add new items to the
-    specification, which starts with inclusion in this section.  Requirements
+    specification, which starts with inclusion in this section or the optional
+    libraries section.  Requirements
     for inclusion are:\footnote{These requirements ensure that most of
     the semantic issues that arise during initial implementation have
     been addressed and prevents the accumulation of interfaces that no
@@ -36,7 +39,7 @@
     specification are:

 \begin{enumerate}
-\item Six months residence in this section.
+\item Six months residence in this section or the optional libraries section.

 \item The existence of either one (or more) publicly available "reference" implementation 
 written in standard UPC OR at least two independent implementations (possibly specific 
@@ -56,3 +59,7 @@
    to be the interface version of the extension if it is supported, otherwise
    undefined.

+\index{header files}
+\np For each library extension, a separate header file whose name begins with
+    {\tt upc\_} must be specified.  This header file must be provided by an
+    implementation if the extension is supported.

Original comment by james.di...@gmail.com on 5 Oct 2012 at 7:48

[GoogleCodeExporter]

"This section contains proposed additions and extensions to the UPC 
specification. ... An optional library specifications section is included ... 
starts with inclusion in this section"

Originally Appendix A was intended to actually contain subsections of 
proposed/optional libraries. In 1.2 it contained the UPC-IO library.
This is no longer the purpose of Appendix A - it is now purely a process 
documentation, not a container for actual specs. Actual specs are contained in 
the optional and required library documents.

" When fully stable and tested, they will be moved to the main body of the 
specification."

Our current lifecycle for libraries is proposed to optional to required, but it 
stops there. The "core" library in the language document is effectively 
"closed" to major new library additions, so that text is outdated.

"* Need a new section documenting our procedure for localized changes to any of 
the documents (from the Wiki).  
Can you explain what you mean by this, or provide a link? "

My point here is we should augment appendix A with the process we've worked out 
on the wiki for changes to the non-library sections of the language document.

"From reading Appendix A, it seems like changes to the main body of the spec 
need to be included in Appendix A itself before they can be moved into the main 
body of the text.  This is horrifying, but it seems to be what is stated by the 
existing text."

Right - this is part of what needs to be updated :) Appendix A was originally 
intended only to define process for defining new library functionality. The 
process for language changes was not considered when writing it.

Original comment by danbonachea on 5 Oct 2012 at 9:27

[GoogleCodeExporter]

"* This appendix is probably also the best place to cross-reference the 
required and optional library documents, whose existence is not currently 
mentioned anywhere in the main language spec document.

This is pretty much the worst place to present the separation of libraries from 
the main document and the split into optional and required chapters.  This 
should be done at the beginning, rather than at the end of the document."

Actually I think it's not a bad place. We can certainly define the reference 
symbols to the other library documents at the beginning (probably in section 
2), but it's kind of putting the cart before the horse to talk about the 
optional libraries before section 7 which defines the core libraries. C99 uses 
Appendixes to summarize the libraries and reference several other specification 
documents which are relevant to the C spec (notably the IEC floating point 
standard). I guess I was hoping we could include whatever cross-references we 
want to add as part of the change proposal for this issue.

Original comment by danbonachea on 5 Oct 2012 at 9:33

[GoogleCodeExporter]

Dan, others--

Thanks for the feedback, and sorry about the high latency.  I've drafted a new 
proposed additions and extension section, and attached the patch.

I think that we have been using the term "libraries" for this section, because 
it reflects how we are currently using it.  However, this section is really 
meant for any "additions and extensions" -- one could also imagine 
language-level extensions (e.g. looping constructs, etc) that should follow 
this same process.  With that in mind, I think I've put together a draft that 
should be workable for this broader usage model.

I doesn't seem like this section was providing rules for making updates to 
existing parts of the spec (language, required/optional extensions).  If we 
want to define that in the spec, then I think we will be adding new rules.  
What's the feeling on this?  Is it something that should be in the spec, and to 
what extent?

Cheers,
 ~Jim.

Original comment by james.di...@gmail.com on 17 Oct 2012 at 3:35

Attachments:

• add_ext_10-17-2012.patch

[GoogleCodeExporter]

> + The proposed extensions
> +    specifications section contains proposed additions and extensions to the
> +    UPC specification.  Such proposals are included when stable enough for
> +    developers to implement and for users to study and experiment with them.
> +    However, their presence does not suggest long term support.  When fully
> +    stable and tested, proposed additions and extensions will be moved to the
> +    required or optional section of the specification.
...
> -    Otherwise, the requirements for inclusion of an extension in the main 
body of the
> -    specification are:
> +    Otherwise, the requirements for inclusion of an extension in the 
required or optional
> +    extension section of the specification are:
> 
> -\item Six months residence in this section.
> +\item Six months residence in the proposed extensions section.

Perhaps I misunderstood our overall plan for the 1.3 release, but I thought the 
idea was to release three official documents (language, required libs and 
optional libs), not a fourth "proposed" document as the new text seems to 
imply. My understanding was the library proposal documents we've been passing 
around the committee are solely internal working copies to iron out details and 
presentation, not something intended for general release (I apologize if my 
comment 14 was misleading - re-reading it I can see how it was ambiguous). I 
believe the committee agreed early on that the three major proposals slated for 
1.3 (non-blocking, castability and AMO) would appear directly in the optional 
library document for 1.3 ratification. 

Appendix A previously specified a 6 month delay between libraries "in this 
section" (what has become the optional library) and "the main body of the 
specification" (which has been divided into the core library embedded in the 
language document and the required library document). I believe we want that 
same effect despite the cosmetic reorganization - ie 6 months in the ratified 
optional document for evaluation before consideration for movement to the 
required library. I don't think we want an additional 6 month delay preceding 
optional - libraries in that section are already optional (allowing 
implementations to completely ignore them if they choose) and they remain there 
until there is broad consensus to accept them as a requirement for conformance.

"one could also imagine language-level extensions (e.g. looping constructs, 
etc) that should follow this same process."

The overall goal of 1.3 was to provide some long-overdue library extensions and 
minor language clarifications, but not to introduce major semantic changes to 
the language itself - so I'm not sure we need to solidify a process for those 
kinds of changes just yet. I had originally suggested copying our wikified spec 
clarification procedures into the appendix, but upon further reflection that 
doesn't really seem a worthwhile inclusion; we are presenting the reader with a 
ratified language specification, and the process used to tweak and ratify that 
document is irrelevant to them, as it is already done. I'm sure the ISO C 
committee has mountains of process documentation, but none of it appears in the 
actual specification product. What they DO include is sections like "Future 
language directions" and "Future library directions" that summarize 
deprecations and other changes likely to appear in a future spec.

I see the purpose of the new Appendix A is to:
1. Introduce the two library documents, and explain what each implies for 
implementation conformance
2. Explain that "optional" library features may eventually become "required" 
(ie future directions)
3. Sketch the broad process for libraries to appear in the optional section and 
eventually move to required.

However the more I think about it the more #3 seems out of place in a formal 
specification. The primary readers of our ratified document are end users and 
implementers, both of whom care about syntax and semantics, not specification 
process. It's important that our community have concrete, documented processes 
in place for evolving specifications, I'm just not sure the description of that 
process belongs in the spec itself. What do others think?

Original comment by danbonachea on 18 Oct 2012 at 6:49

[GoogleCodeExporter]

Dan,

Sorry, the misunderstanding was on my end.  I looked in upc-specification/lib, 
saw opt, req, and proposed, and thought we had moved to three extensions 
sections.  See attached patch.

 ~Jim.

Original comment by james.di...@gmail.com on 18 Oct 2012 at 2:57

Attachments:

• add_ext_10-18-2012-2.patch

[GoogleCodeExporter]

Thanks for the corrections, minor nitpick:

+ Otherwise, the requirements for inclusion of an extension in the required or 
optional
+ extension section of the specification are:
...
+\item Six months residence in the optional extensions section.

Delete phrase "or optional".

I also think we need to include explicit bibliographical references, to clarify 
that "optional extensions specifications section" is referring to the "UPC 
Optional Library Specifications" document (and similarly for required). I just 
submitted issue 100 to define these references in the introduction, and 
Appendix A should use them.

Original comment by danbonachea on 18 Oct 2012 at 9:16

[GoogleCodeExporter]

Whoops, thanks for catching that typo.  I also changed a couple instances of 
"must" to "shall" to follow the existing style.

I agree on adding refs.  Should we just wait for a solution in issue 100?

 ~Jim.

Attached: updated patch and pdf.

Original comment by james.di...@gmail.com on 18 Oct 2012 at 9:46

Attachments:

• add_ext_10-18-2012-3.patch
• upc-lang-spec-draft-issue54.pdf

[GoogleCodeExporter]

Thanks for the updates - the latest version looks good to me.

"I agree on adding refs.  Should we just wait for a solution in issue 100?"

Unless someone objects to the spelling of the reference tags, we might as well 
insert them now. The patch for this issue and issue 100 can both go into the 
working draft "together".

Original comment by danbonachea on 18 Oct 2012 at 10:02

[GoogleCodeExporter]

This issue was discussed in the 10/19 telecon, including a long discussion 
about whether process belongs in the spec at all.

A few post-call thoughts:

1. The version of Appendix A in the current working draft (without Jim's 
changes), is currently nonsense due to the document split - it refers to 
sections that don't exist and fails to describe the new library docs. As a 
matter of expediency I would like to see this fixed in time for the SC12 draft, 
so we don't distribute a document containing a section that we know makes no 
sense.

2. Jim's changes are a unilateral improvement to #1. With his patch, the 
Appendix once again "makes sense", and provides essentially the same 
information and content as spec 1.2 Appendix A. Ie it's no WORSE than what's 
currently there, and definitely a significant improvement.

3. The questions of whether or not the spec should include process 
documentation at all and what process updates are required may end up affecting 
the same text, but is an orthogonal issue and may require protracted discussion 
to resolve. 

Based on this I think we should proceed immediately with the 4 week comment 
period and merge a patch for this issue (with the minor presentation update to 
the references), and then open a new separate issue to discuss matters of 
process documentation.

Original comment by danbonachea on 19 Oct 2012 at 9:29

[GoogleCodeExporter]

Hi All,

I updated the spec with the [UPC-EXT-REQ] and [UPC-EXT-OPT] pointers.  I think 
we should use "EXT" rather than "LIB" since extensions need not be limited to 
linkable libraries (e.g. a new parallel loop could be defined, which is a 
language extension).  Updated patch and PDF are attached.  If all looks good, 
let's send this our for review.

Cheers,
 ~Jim.

Original comment by james.di...@gmail.com on 22 Oct 2012 at 5:49

Attachments:

• upc-lang-spec-draft-issue54.pdf
• issue_54_10-22-12.patch

[GoogleCodeExporter]

"I think we should use "EXT" rather than "LIB" since extensions need not be 
limited to linkable libraries (e.g. a new parallel loop could be defined, which 
is a language extension)."

I strongly disagree with this. Currently our 2 auxiliary documents are titled 
"UPC Optional/Required Library Specifications", and are very strictly library 
extensions only. Each section defines a header file, a feature macro, and 
associated library content. Changing them to contain anything else (language 
changes) would be a major format change to these documents, and a significant 
departure from anything discussed or agreed upon by the committee. We currently 
have no plans to introduce "optional" language features in 1.3, and no 
discussion about if or how we'd do that in future revisions. Calling them 
anything but library documents for 1.3 just introduces confusion.

Also, I think the Appendix section should use the word "specification" when 
referring to these documents, not "section" or "section of the specification" - 
as they are independent documents and not sections within the current document.

Original comment by danbonachea on 22 Oct 2012 at 6:35

[GoogleCodeExporter]

As I read it, the intent of the 1.2 spec is that these sections be used for 
"additions and extensions".  Restricting this to libraries seems like an 
unnecessary constraint that will limit its utility in the future.  I think it's 
important to maintain the flexibility we have in the 1.2 spec, and to preserve 
the process that was laid out for additions and extensions, which the 1.2 spec 
does not restrict to libraries.

In terms of the header text, we can state that it must be provided if it is 
needed.

I can rename "section" -> "specification".  Before I do, I should get this into 
SVN.  Is there an existing branch, or should I make one?

 ~Jim.

Original comment by james.di...@gmail.com on 22 Oct 2012 at 8:30

[GoogleCodeExporter]

"As I read it, the intent of the 1.2 spec is that these sections be used for 
"additions and extensions".  Restricting this to libraries seems like an 
unnecessary constraint that will limit its utility in the future.  I think it's 
important to maintain the flexibility we have in the 1.2 spec, and to preserve 
the process that was laid out for additions and extensions, which the 1.2 spec 
does not restrict to libraries."

The Appendix can still refer to the process for "extensions", but when you 
refer to the actual optional and required *documents*, it should be "library", 
because that's current title and organization of those documents. Calling them 
something else is inconsistent and invites confusion. If 1.4 or 2.0 adds 
optional language extensions, we can add a new document with a new ref or 
update the appendix text references accordingly.

> I can rename "section" -> "specification".  Before I do, I should get this 
into 
> SVN.  Is there an existing branch, or should I make one?

No textual changes may be committed to the trunk until after the 4 week comment 
period. You're welcome to create a private branch for this issue, or keep 
updating the patch file, whichever you find more convenient.

Original comment by danbonachea on 22 Oct 2012 at 8:41

[GoogleCodeExporter]

At some point we renamed "additions and extensions" (the original title of 
Appendix A) to "libraries".  My assertion is that this change was not made 
deliberately and should be undone.  Am I incorrect?

Original comment by james.di...@gmail.com on 22 Oct 2012 at 8:59

[GoogleCodeExporter]

Hi All,

Dan and I have assembled a draft of the ticket 54 change.  This ticket has been 
moved to pending approval, please review the proposed change and send any 
feedback.  If we have consensus in time for the SC draft, we would like to 
include this change.

 ~Jim.

Original comment by james.di...@gmail.com on 26 Oct 2012 at 1:44

• Changed state: PendingApproval
• Added labels: Consensus-High
• Removed labels: Consensus-Low

Attachments:

• upc-lang-spec-draft-issue54-10-25-12.pdf
• issue_54_10-25-12.patch

[GoogleCodeExporter]

This PendingApproval change appeared in the SC12 Draft 3 release.
It was officially ratified at the 11/29 telecon.

Original comment by danbonachea on 29 Nov 2012 at 8:03

• Changed state: Ratified