search

gap-system / gap

Main development repository for GAP - Groups, Algorithms, Programming, a System for Computational Discrete Algebra
https://www.gap-system.org
GNU General Public License v2.0
812 stars 161 forks source link

Simplify the creation of release notes #4257

Closed ThomasBreuer closed 1 year ago

ThomasBreuer commented 3 years ago

One step in the preparation of a release is to put the release notes together. The natural source for these notes is the list of merged pull requests that belong to the release. More precisely, we can ignore those pull requests which are labelled "release notes: not needed". For the other ones, up to now the label "release notes: to be added" was intended to be assigned, until the entry in CHANGES.md was actually made, and the label was replaced by "release notes: added".

Today we (Daniel, Paula, and I) have dealt with some pull requests relevant for GAP 4.11.1 or GAP 4.12.0, and tried whether the titles of the pull requests can be used as their entries for the release notes. We have actually also replaced some pull request titles by ones that are suitable for release notes.

This seems to work in many cases, thus it makes sense to introduce a new label "release notes: use title" (thanks for this, Max) with the meaning that pull requests with this label can be treated automatically in the sense that their release notes entries are given by their titles.

Of course this does not work in all cases, and the remaining pull request (labelled with "release notes: to be added") still have to be checked by hand. Possible reasons are:

Perhaps it makes sense to introduce one more label "release notes: highlight" which marks a pull request as relevant for the section on "New features and major changes" in the release notes. I think that this section must be composed by hand, already because a real highlight will probably consist of several pull requests, but automatically extracting a first approximation of its contents would be helpful.

Some more ideas, which we should discuss and extend:

ThomasBreuer commented 3 years ago

How could a semi-automatic creation of release notes look like?

We suppose

The sections of the new chapter in CHANGES.md correspond to certain categories of changes (bug fixes, new features, etc.), which are given by (combinations of) labels. (We do not want to increase the number of labels too much, because then it gets difficult to deal with the list, so perhaps other flags than labels could we thought of.)

Ideally, each intended section corresponds to the result of a query to the GitHub database, which specifies

For example, we have four section on bug fixes. The worst case is "kind: bug: wrong result", then there are "kind: bug: crash" and "kind: bug: unexpected error", and for anything else there is "kind: bug". (I think it suffices to list each pull request involving a bugfix only once, under the first applicable category.)

Thus we can create the "scaffold" of the new chapter automatically, as a list of categories and corresponding query URLs.

If it is possible to request the result of such a query (as a JSON text?), it can be processed further automatically. (Perhaps the real question is whether we are allowed to use this feature. I remember that MathSciNet people were not happy when we tried to use their database for automatically updating the GAP bibliography.) For example, the release note entries for pull requests with the new label "release notes: use title" can then be created automatically. And if it is possible to perform an automatic addition of the label "release notes: added" to a given selection of pull requests then also this step can be done automatically.

(By the way: Would it perhaps make sense to replace the label "release notes: to be added" by "release notes: needed"? Then it would be clear that just adding the label "release notes: added" is sufficient to finalize the release notes process for a given pull request. With the current setup, additionally the label "release notes: to be added" has to be removed.)

For pull requests without "release notes: use title" label, the worst situation is then that one has to open (by hand) the pull request via the link, extract the text, set the label "release notes: added", and enter the text into CHANGES.md.

fingolfin commented 3 years ago

I've added a new label release notes: highlight, really like it

olexandr-konovalov commented 3 years ago

I am now having a prototype of extracting info using PyGithub.

So, for example

pr label:"release notes: added" is:closed merged:2019-09-10..2033-12-31 base:master is:merged

is about this in PyGithub

prs = {}

# Sort to have newest PRs first, useful for testing with shorter queries
all_pulls = repo.get_pulls(state='closed', sort='created', direction='desc', base='master')

for pr in all_pulls:
    if pr.merged:
        if pr.closed_at > datetime(2019, 9, 10):
            labs = [lab.name for lab in list(pr.get_labels())]
            if 'release notes: added' in labs:
                prs[pr.number] = [ pr.title, 
                                   pr.closed_at.isoformat(), 
                                   [lab for lab in labs if lab.startswith('kind') ] ]

(with some additional filtering, e.g only labels of "kind" type) and it now gives me a dictionary which I can further process:

{4245: ['Free (associative or Lie) algebras: admit non-fields as left acting domains, fixed the zero-dimensional case.',
  '2021-02-09T16:50:13',
  ['kind: bug', 'kind: bug: unexpected error']],
 4239: ['Added missing comparison w.r.t. equality of something and an object with memory (`IsObjWithMemory`).',
  '2021-02-08T19:04:15',
  ['kind: bug', 'kind: bug: unexpected error']],
 4232: ['Iterators for lists: fixed `IsDoneIterator`, improved the performance',
  '2021-01-27T22:43:02',
  []],
 4219: ['Performance additions to generic 2-cohomology and Automorphism group/Isomorphism test',
  '2021-02-09T16:48:18',
  ['kind: bug', 'kind: enhancement', 'kind: performance']],
 4215: ['`libgap` API: Expose a minimal interface to the garbage collector, via `GAP_MarkBag`, `GAP_CollectBags`',
  '2021-01-27T22:54:15',
  []],
 4207: ['Added the group constructors `PGammaL`, `PSigmaL`, and their methods for permutation groups.',
  '2021-01-13T09:40:14',
  ['kind: enhancement']],
 4206: ['Fixed `DirectoriesPackageLibrary`, `DirectoriesPackagePrograms` when they are called during package loading.',
  '2021-01-14T00:10:52',
  ['kind: bug', 'kind: bug: wrong result']],
 4201: ['Support an optional `transformFunction` in `Test()`, similar to `compareFunction`.',
  '2021-02-15T08:53:44',
  ['kind: enhancement']],
 4186: ['Improved the performance of `CoeffientsQadic` for long results.',
  '2020-12-02T21:39:54',
  ['kind: enhancement']],
 4178: ['Fixed bugs in `RestrictedPerm` with second argument a range.',
  '2020-11-18T12:25:48',
  ['kind: bug: wrong result']],
 4168: ['Improved the documentation concerning GASMAN, added `CollectGarbage`.',
  '2020-11-26T12:06:28',
  ['kind: discussion', 'kind: enhancement']]}

etc.

fingolfin commented 3 years ago

@alex-konovalov in your script, I think repo.get_pulls supports more complex queries: it should be possible to already there specify the other filters (must be merged into master after a certain date). That will make the network roundtrip time shorter

ThomasBreuer commented 3 years ago

This morning, we (Alexander, Chris, Daniel, Max, Russ, Wilf) had a discussion during the GAP Days about how to proceed, concerning release notes.

The vision is that any GAP developer can run this script on the local computer in order to produce a markdown file containing a snapshot of the release notes for the next release, according to the github repository. Then it should be easy to fix wrong or unsatisfactory entries by jumping to the pull requests in question, and fixing labels, titles, etc., such that producing the final CHANGES.md at release time essentially needs no hand-editing.

olexandr-konovalov commented 3 years ago

@fingolfin but https://pygithub.readthedocs.io/en/latest/github_objects/Repository.html says

Parameters: | state – stringsort – stringdirection – stringbase – stringhead – string

only

danielrademacher commented 3 years ago

In addition to Thomas Breuers last comment, in particular item 1 and 3, one could suggest making a priority list of labels. For example the following:

  1. release notes: highlight
  2. topic: julia
  3. topic: HPC-GAP
  4. kind: bug: wrong result
  5. kind: bug: crash
  6. kind: bug: unexpected error
  7. kind: bug:
  8. kind: new feature
  9. kind: enhancement

A PR would then only be listed under the point in the release notes with the lowest number. For example, a PR with the labels "release notes: Highlight" and "kind: new feature" would only be listed under Highlight, as this has a lower number in the priority list. This approach would have various advantages:

The PRs would have to be filtered beforehand for the label "release notes: use title", since only these can be automatically output in the release notes. Maybe one could also output a second file with the remaining PRs that have to be checked manually.

This idea could be realized with the following code:

f = open("releasenotes.md", "a")
prioritylist = ["release notes: highlight", "topic: julia", "topic: HPC-GAP", "kind: bug: wrong result", "kind: bug: crash", "kind: bug: unexpected error", "kind: bug", "kind: new feature", "kind: enhancement"]

f.write("Release Notes \n\n\n")

for priorityobject in prioritylist:
    f.write("Category " + priorityobject + "\n")
    removelist = []
    for item in list:
        if priorityobject in item[2]:
            f.write("- [#")
            temp = item[1]
            length = len(temp)
            issuenumber = temp[length-4:length]
            f.write(issuenumber)
            f.write("](")
            f.write(temp)
            f.write(") ")
            f.write(item[0])
            f.write("\n")
            removelist.append(item)
    for item in removelist:
        list.remove(item)
    f.write("\n\n\n")

f.close()

f = open("remainingPR.md", "a")
for item in list:
        f.write("- [#")
        temp = item[1]
        length = len(temp)
        issuenumber = temp[length-4:length]
        f.write(issuenumber)
        f.write("](")
        f.write(temp)
        f.write(") ")
        f.write(item[0])
        f.write("\n")
f.close()

Here the entries of list have the form:

[['Fix warning about checking unsigned < 0 by making signed', 'https://github.com/gap-system/gap/pull/4273', []], ['buildsys: fix Makefile build order for GMP, zlib', 'https://github.com/gap-system/gap/pull/4267', ['gapdays2021-spring', 'regression', 'topic: build system']],...]
fingolfin commented 3 years ago

I like the suggestion by @danielrademacher ; and I'd like to point out that order we use for "priority" need not match the order we use to display the entries.

fingolfin commented 3 years ago

One thing that's not yet quite clear to me is how we'll deal with PRs that don't have the release note: use title label? Based on @ThomasBreuer list of several possibilities for such PRs:

  1. lot of changes: that may mean we need multiple entries for that PR in the log; I guess we could handle this by storing and later extracting relevant texts from the issue body (e.g. by looking for a "Text for release notes" sections as suggested in our PR template)
  2. changes of different kinds: similar to 1, but also the different entries may require different labels... Of course ideally such a thing would not be done, but certain people prefer to have a single PR for a bunch of changes, and while in general I'd like to discourage that, we may want to accept it for useful contributions. If we want to automate this, we could adapt 1. by also specifying a syntax to add label overrides? E.g. the "Text for release notes" section might contain contain: 
    - fix crash in function`Foobar` (label "bug: crash")
- add new function `CakeMaker` (label "enhancement")
  3. If what the pull request does differs from what it means, then again we can use the code for checking the "Text for release notes" section
  4. references to other PRs or issues: we certainly can parse #1234 strings and convert them into suitable hyperlinks
  5. sometimes in the past we group together multiple PRs into a single release notes entry... This could be done by inserting a special command into the "Text for release notes" section, say link-with-PR-1234 -- or perhaps we could just make sure to specify identical release note texts for such PRs, and then our script automatically group PRs with identical description?
fingolfin commented 3 years ago

@fingolfin but https://pygithub.readthedocs.io/en/latest/github_objects/Repository.html says

Parameters: | state – stringsort – stringdirection – stringbase – stringhead – string

only

That's a pity. It seems this filtering feature is only available in version 4 of the GitHub API, which is based on GraphQL, but not in v3 which is using REST. Ah well, as long as it is not too slow, we don't really care :-)

ThomasBreuer commented 3 years ago

I propose the following list of categories for release notes, in this order:

(Each category except the first one must also have at least one of the labels release notes: added, release notes: needed or release notes: use title.)

category labels (+: present, -: not present) meaning
1 New features and major changes + release notes: highlight PRs introducing changes that should be highlighted at the top of the release notes
2 Fixed bugs that could lead to incorrect results + kind: bug: wrong result PRs fixing bugs that result in mathematically or otherwise wrong results
3 Fixed bugs that could lead to crashes or unexpected error messages (+ kind: bug: crash OR + kind: bug: unexpected error) - kind: bug: wrong result PRs fixing bugs that do not lead to wrong results but may cause GAP to crash or to run into an unexpected error
4 Other fixed bugs + kind: bug: - kind: bug: crash - kind: bug: unexpected error - kind: bug: wrong result PRs fixing bugs not in any of the above categories
5 Removed or obsolete functionality not applicable -- introduce a label PRs changing lib/obsolete.g* (move functions there or remove something from there)
6 Improved and extended functionality + kind: enhancement OR + kind: new feature OR + kind: performance PRs implementing enhancements
7 Improvements in the experimental way to allow 3rd party code to link GAP as a library (libgap) + topic: libgap PRs that are related to libgap
8 Improvements in for the Julia integration + topic: julia PRs that are related to the Julia integration
9 Changed documentation + topic: documentation PRs improving the documentation
10 Packages + topic: packages PRs related to package handling, or specific to a package (for packages w/o issue tracker)
11 Other changes - kind: bug: - kind: bug: crash - kind: bug: unexpected error - kind: bug: wrong result - kind: enhancement - kind: new feature - kind: performance - topic: libgap - topic: julia -topic: documentation - topic: packages PRs relevant for the release notes but without a label that fits --think about finding one or introducing a new one

Besides/below that, the following categories should appear in CHANGES.md which are not related to pull requests (and hence also not to labels).

category meaning
Packages no longer redistributed with GAP the list of those packages (name, version, authors, abstract, with link to the package homepage) which have been distributed with the previous GAP version but not with the current one
New packages redistributed with GAP the list of those packages (name, version, authors, abstract, with link to the package homepage) which are distributed with the current GAP version but not with the previous one
Updated packages redistributed with GAP the list of those packages (name, version, authors, abstract, with link to the package homepage) which are distributed with both the previous and the current GAP version, but with different package versions

Concerning the ordering, I think that form a viewpoint of somebody who reads the release notes of a new GAP version will first be interested in highlights, then in bugs which may affect one's computations (wrong results), then in removed functionality (which may break old private code), then in general improvements. Splitting the pull requests according to topics (such as libgap or Julia) is somewhat orthogonal to that viewpoint, and here it is not obvious whether one wants to see the bug fixes for the given topic a second time, or only in the bug fixes category.

Concerning the release notes: use title label, my understanding is that the approximation to CHANGES.md produced by the script should already extract the title for those items with label release notes: use title, and for the moment leave the text empty if this label is not present. In a next step, we can then try to define a convention how release notes text can be extracted from the pull request body.

(For the release, empty categories can be removed from the file CHANGES.md.)

ThomasBreuer commented 3 years ago

Concerning @danielrademacher's suggestion, a hierachical point of view simplifies the decisions a lot. However, I am not sure whether all our categories fit to that. As I wrote above, the "bugs" categories can be regarded as hierarchical, but I think this does not hold for the "topics" categories.

I think that eventually a dynamical way to look at release notes would be desirable, but this is a diferent issue. (It would not be difficult to provide BrowseReleaseNotes inside a GAP session, with the possibility to categorize by any label. The data for that can be created by the script mentioned above, at least for forthcoming releases.)

ThomasBreuer commented 3 years ago

I just had a discussion with @danielrademacher. Now our opinion is: We can take the viewpoint that the release notes file should be short, and thus each entry should appear just once. (Making the list n times as long by showing each entry under its n most important categories is annoying for anyone who just wants to get an overview.)

This means that we should define a hierarchy as @danielrademacher proposes, and then let each entry appear in CHANGES.md only in the most reasonable place, according to this hierarchy.

For the purpose of categorizing the list by other aspects, we can propose calling BrowseReleaseNotes in a GAP session. I can provide such a function as soon as we agree how to proceed. I think that the input data should be a JSON text that contains the URL of the repository, metadata about the version in question (used for the header), and the list of entries, each given by number, release notes text, and set of labels. The input can be given by a concrete file (for a released version of GAP) or can be computed at runtime using the script (as a snapshot of the forthcoming release). This function can then be used also for showing release notes of packages, provided they are based on the same data as the GAP release notes.

olexandr-konovalov commented 3 years ago

I've pushed a prototype of the PyGithub part of the job at #4277 for a preview.

olexandr-konovalov commented 3 years ago

@danielrademacher With #4277, would you like now to incorporate your code from above there?

olexandr-konovalov commented 3 years ago

So, the next version, after adding @danielrademacher's code and some further polishing produces now markdown which I have copied and pasted below without any changes, as well as flags PRs that have to be added to release notes and categorised:

release notes: to be added

Uncategorized PR

Release Notes

New features and major changes

Fixed bugs that could lead to incorrect results

Fixed bugs that could lead to crashes

Fixed bugs that could lead to break loops

Other fixed bugs

Improved and extended functionality

New features

Performance improvements

Improvements in the experimental way to allow 3rd party code to link GAP as a library

Improvements in the experimental support for using the Julia garbage collector

Changed documentation

Packages

olexandr-konovalov commented 3 years ago

P.S. I picked up categories names from past release notes - now that I've reread @ThomasBreuer's tables above, I will update some names of the categories in PR #4277 tomorrow.

ThomasBreuer commented 3 years ago

@alex-konovalov The long list of release notes which is shown above contains also entries not intended for GAP 4.11.1. For example, #3740 is in master but has not been backported to GAP 4.11. I would suggest to take the hand-edited list which is now in CHANGES.md.

olexandr-konovalov commented 3 years ago

@ThomasBreuer yes, I think I need to add a filter for the backported-to-4.11 label, then see close it will be to the hand-edited one.

ThomasBreuer commented 3 years ago

Here is a prototype of the proposed function BrowseReleaseNotes. (It needs a utility function BrowseData.UrlOpen, cf. issue #4272.)

# The following works only when the 'Browse' package is available.
LoadPackage( "Browse" );

# utility function: try to open a browser URL from a GAP session;
# ideas copied from 'lib/helpview.gi'
BrowseData.UrlOpen:= function( t, url )
  if ARCH_IS_MAC_OS_X() then
    Exec( Concatenation( "osascript <<ENDSCRIPT\n",
                         "tell application \"Firefox\"\n",
                         "activate\n",
                         "open location \"", url, "\"\n",
                         "end tell\n",
                         "ENDSCRIPT\n" ) );
  elif not ( ARCH_IS_WINDOWS() 
             # or ARCH_IS_WSL() 
           ) then # UNIX
    Exec( Concatenation( "firefox \"", url,"\" >/dev/null 2>&1 &" ) );
  else
    BrowseData.AlertWithReplay( t, [ "sorry, I do not know how to open",
                                     url ] );
  fi;
end;

# the function for visualizing the release notes data
BrowseReleaseNotes:= function( data )
    local version, repository, list, sel_action, m, n, winwidth, numwidth,
          labelwidth, textwidth, t;

    version:= data.version;
    repository:= data.repository;
    list:= data.data;

    # Define the 'click' on a link.
    # (Actually, one can click anywhere in the row.)
    sel_action:= rec(
      helplines:= [ "open the URL of the current row" ],
      action:= function( t )
        local i;
        if t.dynamic.selectedEntry <> [ 0, 0 ] then
          i:= t.dynamic.indexRow[ t.dynamic.selectedEntry[1] ] / 2;
          BrowseData.UrlOpen( t,
            Concatenation( repository, "/pull/", list[i][2] ) );
        fi;
      end );

    m:= Length( list );
    n:= Maximum( List( list, Length ) );

    # Define the widths of the columns.
    # The number and the text shall use the screen width,
    # such that one can see the whole text column
    # when the table is categorized by the labels column.
    winwidth:= NCurses.getmaxyx( 0 )[2];
    numwidth:= Maximum( 6, Maximum( List( list, x -> Length( x[2] ) ) ) + 1 );
    labelwidth:= Maximum( List( list, x -> Maximum( List( x[3], Length) ) ) );
    textwidth:= winwidth - numwidth - 10;

    # Create the Browse table.
    t:= rec(
      work:= rec(
        align:= "t",
        header:= t -> BrowseData.HeaderWithRowCounter( t,
                        Concatenation( "Release Notes for ", version ),
                        m ),
        main:= List( list,
          l -> [ rec( rows:= l[3], align:= "tl" ),
                 rec( rows:= [ [ NCurses.ColorAttr( "blue", -1 ), true,
                                 NCurses.attrs.BOLD, true,
                                 Concatenation( "#", l[2] ) ] ],
                      align:= "tr" ),
                 rec( rows:= SplitString(
                               BrowseData.ReallyFormatParagraph( l[1],
                                 textwidth, "left" ), "\n" ),
                      align:= "tl" ) ] ),
        m:= m,
        n:= n,
        labelsCol:= [ [ rec( rows:= [ "labels" ], align:= "l" ),
                        rec( rows:= [ "number" ], align:= "l" ),
                        rec( rows:= [ "text" ], align:= "l" ) ] ],
        sepLabelsCol:= [ "-", "-" ],
        sepRow:= "-",
        sepCol:= Concatenation( [ "| " ], List( [ 1 .. n-1 ], x -> " | " ),
                                [ " |" ] ),
        widthCol:= [ , labelwidth,, numwidth,, textwidth ],
        SpecialGrid:= BrowseData.SpecialGridLineDraw,
        Click:= rec(
          select_entry:= sel_action,
          select_row:= sel_action,
        ),
      ),
    );

    BrowseData.SetSortParameters( t, "column", 1,
      [ "hide on categorizing", "yes",
        "add counter on categorizing", "yes",
        "split rows on categorizing", "yes" ] );

    NCurses.BrowseGeneric( t );
end;

# some data from the GAP 4.11.1 release notes
notes:= rec(
repository:= "https://github.com/gap-system/gap",
version:= "GAP 4.11.1",
data:= [
["Free (associative or Lie) algebras: admit non-fields as left acting domains, fixed the zero-dimensional case.",
 "4245",
 ["kind: bug", "kind: bug: unexpected error", "release notes: added", "topic: library"]],
["Added missing comparison w.r.t. equality of something and an object with memory (`IsObjWithMemory`).", "4239",
 ["kind: bug", "kind: bug: unexpected error", "release notes: added", "topic: library"]],
["Iterators for lists: fixed `IsDoneIterator`, improved the performance", "4232",
 ["release notes: added", "topic: error handling", "topic: library"]],
["Performance additions to generic 2-cohomology and Automorphism group/Isomorphism test", "4219",
 ["kind: bug", "kind: enhancement", "kind: performance", "release notes: added"]],
["`libgap` API: Expose a minimal interface to the garbage collector, via `GAP_MarkBag`, `GAP_CollectBags`", "4215",
 ["release notes: added", "topic: kernel", "topic: libgap"]],
["Added the group constructors `PGammaL`, `PSigmaL`, and their methods for permutation groups.", "4207",
 ["kind: enhancement", "release notes: added", "topic: library"]],
["Fixed `DirectoriesPackageLibrary`, `DirectoriesPackagePrograms` when they are called during package loading.", "4206",
 ["kind: bug", "kind: bug: wrong result", "release notes: added", "topic: packages"]],
["Support an optional `transformFunction` in `Test()`, similar to `compareFunction`.", "4201",
 ["kind: enhancement", "release notes: added"]],
["Improved the performance of `CoeffientsQadic` for long results.", "4186",
 ["kind: enhancement", "release notes: added", "topic: performance"]],
] );;

Call the function with

BrowseReleaseNotes( notes );

then navigate in the table by keystrokes: scscX -- categorize by the labels, expand all categories sed -- select an entry, navigate down to the first table row <Return> -- try to open the URL of this row in a browser q -- leave the selection mode ! -- clear the categorization sc -- select the labels column fkind: bug -- enter a string for filtering <Return> -- filter the table Q -- leave the table, return to the GAP prompt

ThomasBreuer commented 3 years ago

@danielrademacher @fingolfin A draft of the documentation on the creation of release notes (file dev/release_notes.readme.md) can be found in the branch of pull request #4256.

olexandr-konovalov commented 3 years ago

@ThomasBreuer @fingolfin with additional condition to check for the backport-to-4.11-DONE label it produces what's below (I did not remove the categories with no PRs):

Release Notes

New features and major changes

Fixed bugs that could lead to incorrect results

Fixed bugs that could lead to crashes

Fixed bugs that could lead to break loops

Other fixed bugs

Improved and extended functionality

New features

Performance improvements

Improvements in the experimental way to allow 3rd party code to link GAP as a library

Improvements in the experimental support for using the Julia garbage collector

Changed documentation

Packages

olexandr-konovalov commented 3 years ago

P.S. this is just a demo of opportunities, I think if we have already the hand-crafted list ready for 4.11.1, we should go with it.

fingolfin commented 3 years ago

Regarding #4053, it is listed under "fixed crashes", but IMHO should be in the "Julia" section -- because this is a crash that does not affect the general public.

Yet I also think that Julia section should be about where it is right now.

This is precisely why I mentioned earlier the possibility that the order in which sections appear need not strictly match the order in which we assign issues. It complicates things a bit, but in the end, all that is needed is to also assign an integer to each section which controls "the other order" (so you start out with one order to bin the PRs into different sections; hten you use SortBy (or the python equivalent) to determine the order in which to print things).

ThomasBreuer commented 3 years ago

@fingolfin Your argument about ordering works in the example of #4053, which fixes a crash related to the Julia integration, but one could also think about a pull request with the same labels that fixes a (general) crash and also had to adjust something in the Julia integration. (Perhaps the situation "crash"+"hpcgap" is less hypothetical in this respect.)

More generally, take two items with the same labels A and B, where label A is regarded as more important for the first and label B is regarded as more important for the second. This cannot be modeled by priorities of labels.

I think the situation is similar to the ordering of GAP's methods by rank. It works because true concurrence situations are rare, and because of (dubious) hand editing of individual ranks.

fingolfin commented 3 years ago

Well any projection from a multidimensional space to a one dimensional one will suffer from these issues. That includes the current approach discussed here and implemented in PR #4277 ...

Anyway, for that Julia PR, I'll simply remove the crash label now.

danielrademacher commented 3 years ago

In order not to have to classify these pull requests manually, one could add an additional label for each label, e.g. release notes: julia release notes: fixed crashes

If a pull request should now have the labels "julia" and "fixed crashes", as well as should be in the release notes under "julia", then this PR should be given the labels "release notes: julia" and "fixed crashes". This would allow the script to recognize from the label "release notes: julia" that it belongs there after all.

fingolfin commented 3 years ago

For now, I don't think the release notes will be done 100% automatically anyway, so if I have to occasional move an item manually, that will also be fine. It's still a vast improvement over what we had to do so far.

ThomasBreuer commented 3 years ago

Concerning the "demo of opportunities" output shown above: The entries in each section are sorted w.r.t. decreasing pull request number.

I had chosen the reversed ordering for the "hand-crafted" version of CHANGES.md, having people in mind who actually read through the release notes. The section "Packages" in the above list shows an example why I think that sorting by increasing pull request number is more logical (or at least less confusing): First #3683 describes the addition of a file, then #4016 describes improvements in this file.

fingolfin commented 1 year ago

At this point we've used a version of these scripts for at least three releases. I am not aware of specific unresolved issues here, so I am closing this. Even if there are remaining wishes or issues listed here, I think it'd be better to open new issues for them.