Feedback / discussion regarding release-cycle.

[Dronek]

I would like to address the fact / the "issue" that the frequency of iNav (pre-)releases has been obviously decreasing since the beginning of 2017 (at least) while popularity and support have grown at the same time. To be clear up front, this is not meant to criticize or discuss the development pace of iNav whatsoever.

My request is rather to increase the accessibility of experimental features to "regular" pilots / users. Many of us were "brave enough" to fly alpha releases of iNav on our aircraft in order to test new features. As far as I recall, this had a positive effect of improved contribution from non-developers in terms of functionality-oriented feedback on RCG. Over the last several months discussions on RCG have seemingly reduced towards getting supported hardware to work and configuration.

Would it be feasible to make the future releases more granular / frequent, including "alpha-releases"?

[digitalentity]

@Dronek good point. It's all comes to balance between having enough time, testing everything and desire to have as many new bells and whistles as possible in new releases.

How about dropping the release-candidate cycle and changing the versioning: Three-digit versions will be "unstable releases" Two digit versions will be "stable releases".

This way we'll be able to do more frequent releases for those who are brave while still keeping somewhat adequate stable release cycle.

Thoughts?

[stronnag]

I agree witj @Dronek. We should perhaps try to have no more than a month between minor releases. Now we seem to have settled into a culture of 'just one more big feature' and the cycle stretches. We seem to get a lot of 'can someone please build me ...' requests on RGC. So how about a feature freeze and 1.7.3 in the new 'semi-stable' scheme by the end of the week? Do we have any show-stoppers in the works?

[digitalentity]

We don't have any show-stoppers for 1.7.2 apart from few minor features. We can have a semi-stable 1.7.3 by end of the week for sure.

[oleost]

How about release every 4 week.

So it would go like this:

A stable release is done. After 3 weeks release an release candidate. Freeze new feature pullrequest, only bugfix. After 4 weeks if no outstanding critical bug, release a new stable version. If still outstanding bugs release an release candiate. It will be stuck in this stage until bug are fixed and a new stable version is out.

( So this would mean it can be extended more than 4 weeks, and its noth calender based )

This would leave 3 week where one can implement new features, and 1 week to fix any issues. ( Or more if necessary. )

Versions numbers doesnt matter, as they will probably live they own life depending on what have been done the last 4 weeks.

[krzysztofmatula]

At the time of building it's hard to state that this will be the stable version. For me, major/minor number increase could indicate new features, while the micro number could be added for bugfix releases. I also vote for short release cycle, 1 month at most. In case problems are detected bugfix release(s) should follow.

[digitalentity]

@krzysztofmatula what about release-candidates? Releasing RC's makes release-cycle considerably longer.

[krzysztofmatula]

For me, additional RC phase could be skipped. Each "new feature" release (major/minor bump) could be the "RC" (regardless if we name them so or not), followed by bugfix release (could be considered "stable", but if more problems will occur, we may build another "stable" with micro number bumped). I'm not a fan of releasing a version named x.x-RC, and then not releasing the "stable" shortly (e.g. because there are no problems, or because there are some severe ones that should go to the release). It makes the impression that something is not done to the end (at least for me).

[martinbudden]

Personally I think 1 month is too short. A big new feature will take more than one month to develop and so will have to be done in a way that spans more than one release. This will discourage development of "big" features.

What really is required is to make the latest builds available to "test pilots", so they can fly them and test them. I think the way to do this is to set up a "nightly" build server that builds development every night and makes it available. Betaflight does this with the Jenkins build server, iNav should do something similar.

[krzysztofmatula]

Nightly builds are welcome and this will offload developers from the "please build this for me to test" requests. However I don't get why developing a feature that takes longer than release cycle could discourage one from work...

[martinbudden]

However I don't get why developing a feature that takes longer than release cycle could discourage one from work...

Because then you have to spend more time doing code merges/rebases and that is always a pain...

[digitalentity]

Nightly builds are good - maybe it's possible to use same server as Betaflight to build INAV's development branch?

Regarding big features - in most cases they could be done incrementally, without affecting end users much. But I agree with @martinbudden - there are features that are big in terms of required coding or testing and also atomic (couldn't be split) - we had a few of those and they were a pain.

[krzysztofmatula]

Merges/rebases could be pain, I agree. But... should this block other features from being delivered? I see this opposite - having developed a feature, publishing PR and waiting months for this thing to be released into public could be discouraging... Just my humble opinion.

[giacomo892]

It could be nice releasing a build when something really needs to be tested on many machines, without a fixed schedule, specifying what changed and what to test.

Also inav-configurator needs to be released to match the current build. (no-go with nightly)

That can really help the development with logs and bug reports.

Letting users to run straight nightly builds without letting them know what's really changed can led to some dangerous situations or a led to a general lack of development awareness.

When I build a firmware from development I usually take a look at every commit to understand where the project went and then I'm ready to flight.

[digitalentity]

@giacomo892 good point. Configurator has to match the build number.

The way I see it: For minor releases stick to 1mo release cycle, postpone features that didn't make it to the code yet. Declare feature freeze 1 week before release and only accept bugfixes. These releases could be done straight from development branch. We'll mark these releases as "pre-release" on github.

Minor releases would essentially be a release-candidates for a bigger release.

What do you think?

[Dronek]

I think the way to do this is to set up a "nightly" build server that builds development every night and makes it available. Betaflight does this with the Jenkins build server, iNav should do something similar.

I were about to suggest a similar thing to Jenkins. However, it could actually be more harmful to provide nightlies for systems with navigation / autonomous capabilities compared to BF. If so, nightlies should be rather considered for bench testing only. As for release numbering, e.g. 1.7.x releases could be considered as "testing release" and 1.x "stable". Testing releases could include experimental features which might not necessarily make it into stable releases, therefore being different to a release candidate. I support the idea of getting rid of release candidates. Instead, x.x.9 could indicate that no new features will be added, being the last testing release in order to find bugs before next stable release.

[krzysztofmatula]

I'd point out that simplicity and avoiding doubts should be the principle. E.g. bigger number should always indicate newer version. If we can provide stable releases - great. They should be clearly marked then as such. As well as bugfix releases to the stable releases in case the latter require any.

[DzikuVx]

OK, here are my thoughts on the topic:

1. Users do not want fast release cycle. They want new targets and features, but not new releases ;)
2. We are a bunch of dudes that do other stuff too. INAV is after hours. I do it because I like it but only when I like it. And I suppose other devs are in the same position. So planning how often something will be released is more less useless, since
   1. There is no roadmap
   2. Resources (devs) come and go and there is no way to plan for that
   3. Devs works on what they want to work on
3. 1 month release cycle will never work
4. 2 month release cycle more less works, but to be honest I would prefer 3 months release cycle
5. We should not release in "official" channels unstable releases. Miniquad racer will survive almost everything. Airplane or other >1kg multirotor will require rebuild in case of software bug and crash
6. RC phase for stable release should take at least 4 weeks

[Dronek]

I agree that defining and sticking to a fixed release cycle might be rather unrealistic for projects like this. I experience roadmap changes and postponed deadlines on a regular basis in a professional context. Not to mention how much overhead project management creates. However, please keep "test pilots" in mind who are also regular dudes and who like to contribute experience from the field in order to help shaping iNav's functionality... in their spare time as well. Regarding safety concerns: every user has the choice to enable the "show unstable versions" switch in the firmware flasher or to keep this switch off to stay more or less safe. As of now, this switch is quite useless. Prividing testing releases in a more frequent (not necessarily fixed) manner could provide a broader feedback on a comparable basis. There is quite a gap between flying a stable release and bench testing development builds.

[digitalentity]

My thougs on this:

I agree with @DzikuVx that 1mo releases are unrealistic. However, I think we should make an "unstable" release approx. each month regardless of new features added/removed.

Stable release should be made every 2-3 months. RC phase (assuming feature-freeze) for 4 weeks is probably an overkill - 2 weeks should be enough. Our release schedule will look like this:

1.7.3 (unstable) --- (1mo) ---> 1.7.4 (unstable, RC) ---> (1w, some bugs fixed) --->1.7.5 --- (2 days, critical bug found and fixed) ---> 1.7.6 --- (1w, no new bugs) ---> 1.8 (stable) ---> (1mo) ---> 1.8.1 ...

[martinbudden]

If we make an "unstable" release then it should be clearly named as such. Ie we should not use the number to denote stability. Ie it should be 1.7.4-RC1, or 1.7.4-alpha1 or some such name.

[digitalentity]

On GitHub it's possible to mark release as "Pre release" and in Configurator Flasher it won't be shown unless you tick the "Show unstable releases" switch and unstable releases will be shown as "(release candidate)" in the list:

I think it would be enough - no need to explicitly add anything to version number.

[krzysztofmatula]

1.7.6 --- (1w, no new bugs) ---> 1.8 (stable) ---> (1mo) ---> 1.8.1 ...

The first impression to users could be that 1.8.1 is a bugfix release to 1.8 and not something completely new... However I don't have any constructive proposal how to make it more clear...

[martinbudden]

I think it would be enough - no need to explicitly add anything to version number.

That assumes people are not using local files to flash with.

What is the reason not to add something to the version number?

[krzysztofmatula]

Maybe: 1.8.dev1 -> 1.8.dev2 -> 1.8 (stable) -> 1.8.1 (potential bugfix) -> 1.9.dev1

I'm not sure about this "dev"... But anything like "alpha", "beta" or "rc", all have the meaning that the feature set is fully implemented, but not completely stable yet. This is not the case of development unstable snapshots.

Maybe "1.8.pre1" like "preview" or "pre-release"...? I vote for including the information that build is not stable in the build name.

[martinbudden]

But anything like "alpha", "beta" or "rc", all have the meaning that the feature set is fully implemented

That is not correct. Alpha specifically means that the feature set may not be fully implemented. Once feature set is complete the software becomes beta.

[digitalentity]

That assumes people are not using local files to flash with.

Local file only has 3-digit version number anyhow.

[digitalentity]

Let's keep the 3-digit versions and use the following conventions:

1. Stable releases are two digit: 1.8
2. Before intended stable release we add at least 2 week feature freze and only accept bugfixes
3. If closely following a stable release we find a bug - we do a bugfix release 1.8.1
   • This bugfix is done via a PR to master branch and is backported to development
4. Every month (approx.) we do a beta releases which have 3-digit versions with added "beta" label: 1.8.2 (beta)

[digitalentity]

Using this convention our next release will be unstable 1.7.3 (beta) and after that - stable 1.8