search

fNIRS / snirf

SNIRF Format Specification
http://fnirs.org/resources/software/snirf/
Other
59 stars 34 forks source link

Document release procedure #74

Closed rob-luke closed 3 years ago

rob-luke commented 3 years ago

As discussed in #70

@dboas @fangq @sstucker does this seem reasonable to you?

The main point I would like to push is 7.1.
Currently when I visit the SNIRF specification I have no way to know if it is a released version or work in progress. So I may develop my software thinking it will be compatible with SNIRF, but I am targeting some shifting sand. This isn't hypothetical, this is a confusion I have had.
If we are going to use the word draft in the format string (not my preference, I'd rather just tag as 1.1 but @fangq did not approve this suggestion in #51) then we still add the word development to anything that isn't released to make it clear to users.

For the release naming I suggest using https://semver.org. So for example the release naming would go...

rob-luke commented 3 years ago

So to be completely clear for the next release (as it somewhat differs due to the draft wording situation) I will:

And from then on we follow the procedure outlined in the document/above.

dboas commented 3 years ago

@fangq , are you okay with this? It does make it a little more clear for me.

fangq commented 3 years ago

Established versioning schemes can be found in many Linux distribution packaging guidelines, for example, this is what is used for Fedora:

https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/

by only going over various "upstream version" styles, you can see 1) it is very diverse, and 2) often times it needs more than just than 2 elements (major.minor) to represent various software release stages.

so, I am afraid that major.minor-fixedtag will not be enough. we need something like 1.1-draft1,2,.., 1.1pre1,2,.., 1.1-rc1,2,.. etc to accommodate multiple release stages and feedback cycles.

my recommendation is major.minor-draft1,2,3 for working drafts (with significant updates between numbers), and major.minor-rc1,2 for near-complete release candidates. This does not mean we will use a lot of release cycles, we can only do 1-2 drafts plus a single RC if the updates are not complex. If any of you prefer, we can also replace draft to pre, although I think draft maybe better suited for specs.

one other minor thing related to release procedures - in the past, in order to make a so called "RFC" (request for comment) release, what I did was to create a branch, say Draft 3, then I make 3 commits just in this branch - in the first commit, I remove all content in the spec md file, in the 2nd one, I restore the full content, in the 3rd one, I update the README.md with the RFC commit hash, so that people can use the commit to make line-by-line comments, see example here

https://github.com/fNIRS/snirf/commits/Draft_3

I wonder if you think this is still something we would like to continue.

let me know what you think.

rob-luke commented 3 years ago

The current versioning approach is not working for SNIRF, so something needs to change or be clarified. People are writing code around a shifting specification. We just need a simple scheme that is documented.

This isn't a complex beast like a linux distribution, its a specification that people need to target. Even if there are issues with the current version, vendors and users are writing code around it. My main consideration is that an approach is documented, that I can understand it, an we can implement it. There are squilions of versioning schemes, we just need a simple one we can use.

As such, I suggest using the standard https://semver.org approach, but feel free to open an alternative PR if you have a better solution. This approach is already used by the neuroimaging community, so it is more appropriate than modelling our approach on a linux distribution.

I simply don't understand many aspects of your comments (specific replies below). To help me understand, perhaps rather than talking in hypotheticals, let's talk in examples. What is a situation where my current proposal would not suffice? Then let's fix the problems.

I think we need to be pragmatic here and just pick an approach. I would be pleased to close this PR and accept an alternative proposal.

Established versioning schemes can be found in many Linux distribution packaging guidelines,

I don't see what the relation between this project and a Linux distribution is. Instead we should follow standard guidelines used by other neuroimaging softwares. For example MNE uses the widely accepted semantic versioning scheme used https://semver.org I think this system is easier to understand than the linux approach, and already used within our neuroimaging community.

so, I am afraid that major.minor-fixedtag will not be enough. we need something like 1.1-draft1,2,

I don't understand what the purpose of multiple tags for the drafts or release candidates is? When a version is in development it can be commented on, we make improvements, then you release a version and repeat the cycle. What is the purpose of these drafts for a small community project like this?

But if you don't think that major.minor is sufficient then could we use the semver approach and use major.minor.patch? This is also easier to parse in software than using different wording.

my recommendation is major.minor-draft1,2,3 for working drafts

I think this is my confusion. Anything that isn't a release is a draft right? People should comment on the spec at any time and offer improvements. Unlike a linux distribution, I don't see what the purpose of these drafts is.

We have a shifting development version (as we do now), and then at specific instances when we think it's stable we tag a release. Im sorry, I think I fundamentally don't understand your approach. If you still wish to use your suggestion can you provide a more detailed step by step explanation of how it would work.

one other minor thing related to release procedures - in the past, in order to make a so called "RFC" (request for comment) release, what I did was to create a branch, say Draft 3, then I make 3 commits just in this branch

I don't understand what the benefit here is? I don't think I understand what you are suggesting or what the use case is. This just seems like more work for whoever is organising the release (currently me) with no apparent benefit. Also I think using branches is confusing for contributors. Why cant people comment on the current development version (they already are)? We can always use the SNIRF mailing list to ask the community for feedback at specific times.

fangq commented 3 years ago

it boils down to the following question:

towards each release (1.0, 1.1, 1.2, ...), how many small feedback cycles (i.e. pre-releases) do you prefer to make?

if just once, your approach 7.1 should be fine, if we want to have multiple cycles, then you need to append a pre-release number.

fangq commented 3 years ago

one other minor thing related to release procedures - in the past, in order to make a so called "RFC" (request for comment) release, what I did was to create a branch, say Draft 3, then I make 3 commits just in this branch

I don't understand what the benefit here is? I don't think I understand what you are suggesting or what the use case is. This just seems like more work for whoever is organising the release (currently me) with no apparent benefit. Also I think using branches is confusing for contributors. Why cant people comment on the current development version (they already are)? We can always use the SNIRF mailing list to ask the community for feedback at specific times.

this is not for organizing releases, instead, it takes advantage of the github's per-line comment feature (plus button) to let people to provide feedback and discussion down to each line, like this page

https://github.com/fNIRS/snirf/commit/88baa2a2ed3347e868ec184b9daa4b357ddbbfd1

rob-luke commented 3 years ago

if just once, your approach 7.1 should be fine, if we want to have multiple cycles, then you need to append a pre-release number.

Given we are moving towards version 1 and the specification has been battle tested via implementation in several software packages (Homer, MNE, Fieldtrip, Brainstorm, etc), and via multiple vendors (NIRx and Kernel), I think the modifications from here on will be of incremental nature. And a single cycle would be sufficient.

But I take your point, so lets also use the patch number of semver. Here are the details for those reading along...

Given a version number MAJOR.MINOR.PATCH, increment the:

MAJOR version when you make incompatible API changes,
MINOR version when you add functionality in a backwards compatible manner, and
PATCH version when you make backwards compatible bug fixes.

I have modified the opening comment to this thread to provide specific examples for this approach. But I repeat here for convenience...

Further, we will note on the front page and in the contribution documentation that we welcome feedback from the community at all times via issues in the GitHub issue tracker (see #76).

fangq commented 3 years ago

@rob-luke, first, I agree with you that we need a tag to indicate between-release unstable transient state, which is your -development suffix (alternative to that is to use the rcs-keyword/.git_filters approach like used in mcx, but it is a bit complicated). The -dev tag does not indicate a release per se, but a unstable non-release state.

regarding how many pre-releases (or feedback cycles) do we need per each release, I will let you and @dboas decide. we can start with the simple one like in this PR, and add additional 3rd digit when we need pre-releases.

@dboas @sstucker, let us know which way you lean towards.

dboas commented 3 years ago

I don't have strong opinions about this. But @rob-luke 's suggestion is easier for me in that it looks more familiar to me. @sstucker , do you have a preference here?

sstucker commented 3 years ago

This semver model @rob-luke outlines above looks good to me. I understand the appeal of branches but as this is a simple project with few files I think having a draft or in-development file in this repository is appropriate.

rob-luke commented 3 years ago

Great. It seems no one has an issue with the actual PR (the procedure), so I will merge this.

Once we go through a release we will likely pick up on something I missed here and we can iteratively update the procedure.

Regarding the versioning scheme. It seems 3 of 4 ([@rob-luke, @dboas, @sstucker] vs [@fangq]) seem happy with the semver approach, so unless @fangq calls a hard veto Im going to push forward with that plan.