RST3.6 release

[egthomas]

So we're now only a few weeks away from our tentatively scheduled release date for v3.6 of the RST. There are currently several outstanding pull requests for bug fixes of varying complexity which range from ~1-3 weeks old. I would like to start making pull requests for some of the actual improvements to major routines like make_grid and map_potential, however it doesn't look like we have the activity level to get those larger pull requests approved in the near future. I fully appreciate that everyone has their own tasks and priorities, but I think now might be a good time to take another head count of who is willing/able to contribute to the testing process in preparation for the new release.

Thoughts? Comments? ...Bueller?

[ksterne]

I can help out some here. Though for testing it is MUCH, MUCH easier if a test is given so that all I need to do is pull code from request branch. Execute example code given in the pull request. See that I get test result.

Many pull request have not had this to date. Even if its a bugfix, give code that demonstrates the issue on the current develop branch and that the new code fixes this bug.

[ksterne]

Also, you think activity is low on this...you should see the activity on davitpy...or ROS!

[egthomas]

One could point out that development activity is much more active here than with davitpy at the moment (last commit / pull request over a month ago? next closest pull request was 3 months ago?), but I do appreciate you taking the time to work through those pull requests last week.

Thinking ahead a little bit, is there any reason not to delete the existing master branch and create a new one based on the existing develop branch when we're ready for v3.6? master is nearly 2 years old, a relic of VTRST3.5, probably not used by anyone, and has several merge conflicts.

[pasha-ponomarenko]

I cannot recall discussing the tentative release date for RST3.6. I guess you'd like to have it up and running before the workshop? While looking ahead is always welcome, anything but rushing, I'd say. I would also not bother drawing any parallels with davitpy, which seems to be a different beast altogether, both peoplewise and goalwise.

[ksterne]

We talked about the creation of a release branch back with #8. I haven't heard any objections to creating a release branch on April 26th which we put effort into testing this branch only. At this point all new pull requests should be frozen. Then, I've proposed May 31st we do the release branch to master pull so that yes, there is a new master branch and a release before the workshop.

@egthomas, only reason I'd see on not deleting the existing master branch is to keep things kosher with git. Yes there are merge conflicts currently which we can hopefully clean up between April 26th and May 31st. We'll also need to make master the default branch whenever we have the new release out.

I'm hoping as well that the people going to the Workshop that are working on this make sure to pass along to everyone, or make some kind of big announcement that this is the current RST repo and no longer the VTRST3.5 repo. I've pointed Rob Fear to this repo, but there are probably others that don't know yet. I'll change the README.md file in the VTRST3.5 repo when we create the release branch here.

[pasha-ponomarenko]

Yep, now I see it. That time I was speaking Japanese... ;-)

[sshepherd]

I think it is unlikely that make_grid, etc. will be fully tested before the workshop. These are the types of things we need to make available and let the users test. They will let us know if there are problems.

On Mon, Apr 10, 2017 at 10:12 AM pasha-ponomarenko notifications@github.com wrote:

Yep, now I see it. That time I was speaking Japanese... ;-)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/SuperDARN/rst/issues/44#issuecomment-292961260, or mute the thread https://github.com/notifications/unsubscribe-auth/AC2DP7npXF5ax1MzfIIczIyaupQ9GN8_ks5rujixgaJpZM4M0V_c .

[egthomas]

Thinking about this some more, should we label the new release as RST 4.0 (rather than 3.6) to indicate a clean break from the previous development system to a more collaborative community approach via github? We have also made pretty drastic changes to the contents and directory structures since Rob's last release (3.3). In fact, his development comments indicate that 3.3 was intended to be the final release of the "3.x series". Just an idea.

[pasha-ponomarenko]

After the amount of effort invested by now I think this will be more than appropriate. ;-) When are you coming to Saskatoon and for how long?

[sshepherd]

agreed

On Wed, Apr 12, 2017 at 10:12 AM, Evan Thomas notifications@github.com wrote:

Thinking about this some more, should we label the new release as RST 4.0 (rather than 3.6) to indicate a clean break from the previous development system to a more collaborative community approach via github? We have also made pretty drastic changes to the contents and directory structures since Rob's last release (3.3). In fact, his development comments indicate that 3.3 was intended to be the final release of the "3.x series". Just an idea.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SuperDARN/rst/issues/44#issuecomment-293589582, or mute the thread https://github.com/notifications/unsubscribe-auth/AC2DP7o04ObAOqKJMWyPUOyeU4ZqiHjIks5rvNu_gaJpZM4M0V_c .

[egthomas]

@pasha-ponomarenko - I'll be in Saskatoon in about 2 weeks from April 26-28 (arrive late Tues afternoon on the 25th). Hopefully then we can talk about how we want to present this at the workshop and take a look at some of the fitacf3 results.

[pasha-ponomarenko]

A bit short but will do.

[ksterne]

Sounds good on the 4.0 numbering. Usually the major number change represents a major change in the code. I think this is even justified since the large section of ROS has been removed from this repo which is a major change.

[egthomas]

@ksterne , when you say that "all new pull requests are frozen" upon creation of a release branch, do you mean that no new pull requests should be made? Or that any outstanding pull requests are ignored until testing is complete? That will have an impact on how many more pull requests I try to squeeze in before leaving for Canada next week.

I'm still not entirely convinced that the current 'master' branch contains any meaningful history or information that we need to retain but I'll let the git veterans make that decision.

[ksterne]

Hey @egthomas, I think the spirit is more of the later that any new pull request will be ignored until the new release is merged into master. With that idea, it won't make sense to make a pull request aside from trying not to forget about it. We may want to stick with the no new pull request idea in case there are significant changes during the release branch testing period.

[sshepherd]

Hi guys,

A couple of thoughts about the new RST "release". I don't think it is very useful to lock things down for a full month. What additional testing will be done? Most people will be busy getting ready for the workshop. It would be better to get it into the hands of as many users as possible. The changes to this point have been mostly organizational with some bug fixes and improved documentation. Evan and I have added substantial features to the RST that we have not submitted pull requests for: Gridding, Map Potential and some plotting capabilities. I won't speak for Evan, but I have not implemented all the changes that need to be added, but have enough features in the map potential (AACGM-v2, MLT-v2, IGRF-v2 and using the CS10 statistical model) to make it useful for us and others. I will be making changes when I have the time or when the need arises.

So, I am suggesting that we have a more fluid "release" and get the software to as many users as we can. There will be bugs, but we will fix them when they are identified.

Simon

On Fri, Apr 21, 2017 at 10:42 AM, Kevin Sterne notifications@github.com wrote:

Hey @egthomas https://github.com/egthomas, I think the spirit is more of the later that any new pull request will be ignored until the new release is merged into master. With that idea, it won't make sense to make a pull request aside from trying not to forget about it. We may want to stick with the no new pull request idea in case there are significant changes during the release branch testing period.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SuperDARN/rst/issues/44#issuecomment-296210142, or mute the thread https://github.com/notifications/unsubscribe-auth/AC2DP3mNH5-UchXWjQxQaJFmJrCgc3o1ks5ryMBygaJpZM4M0V_c .

[ksterne]

@sshepherd, the idea of the release branch period is to test this branch out to make sure there aren't obvious bugs. The time that this branch is around until the new master is made is meant to give people lots of time to test it that they may not have otherwise done. The idea is for us to find the bugs before it goes out to the master branch and what users/non-devs should be using.

And we can still get it in the "hands" of as many people as you want before the workshop as you can direct them to either the release-4.0 branch once its created or the develop branch.

If there is so much work that is going on, I'm not sure why we haven't seen pull requests coming out of that work. So far it seems as though effort has been made more on removing the ROS code, cleaning up the codebase and reorganizing it. Which isn't to say that that's not appreciated and didn't need to be done. But if you wanted to get changes to gridding and map potential in the code, why didn't those take priority?

Would delaying the release branch creation help out?

[sshepherd]

I think the procedure you are describing is too formal. It might be fine for a software company but we are bunch of scientists making improvements to the code. There has been zero development to RST for several years. We are attempting to reinvigorate the development and maintenance of the RST. So far there has been limited participation.

There are several reasons for delaying pull requests on the gridding and map potential features. The first is that the RST needed to be cleaned up before we could begin., hence the 'urgency' with getting these changes approved. The new AACGM-v2, MLT-v2 and IGRF-v2 were part of this. They are, however, just libraries. Using them requires significant effort in changing the software and that is what we are working on. Evan and I are making changes to the gridding and map potential software and have been using this software. So, yes, this has been our priority all along.

What additional testing are you planning to do that hasn't already been done? My point is that the users will be testing it by using it. I think someone should decide what exactly is going to be in the new release of the RST (fitacf 3.0, gridding, map potential, etc.) and then we should work toward that.

On Fri, Apr 21, 2017 at 12:36 PM, Kevin Sterne notifications@github.com wrote:

@sshepherd https://github.com/sshepherd, the idea of the release branch period is to test this branch out to make sure there aren't obvious bugs. The time that this branch is around until the new master is made is meant to give people lots of time to test it that they may not have otherwise done. The idea is for us to find the bugs before it goes out to the master branch and what users/non-devs should be using.

And we can still get it in the "hands" of as many people as you want before the workshop as you can direct them to either the release-4.0 branch once its created or the develop branch.

If there is so much work that is going on, I'm not sure why we haven't seen pull requests coming out of that work. So far it seems as though effort has been made more on removing the ROS code, cleaning up the codebase and reorganizing it. Which isn't to say that that's not appreciated and didn't need to be done. But if you wanted to get changes to gridding and map potential in the code, why didn't those take priority?

Would delaying the release branch creation help out?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SuperDARN/rst/issues/44#issuecomment-296240523, or mute the thread https://github.com/notifications/unsubscribe-auth/AC2DP6LH58F_XekUfm5S8XD7xVBq_zyxks5ryNsGgaJpZM4M0V_c .

[pasha-ponomarenko]

Here are my five Australo-Canadian cents (a nickel?). Kevin, the first (and most important) task is to refactor the RST so that it is easy to work with, which includes both installiation and modification. This can well be RST4.0. After that you may start making more fundamental changes. I guess this explains why these more serious modifications have been put on hold.

With respect to "frozen" or "fluid" release, I guess it is good to have some time for the package to remain unchanged so that different people test the same code.

[ksterne]

I thought it was discussed before that the next release is a clear change to the development and code away from the RST/ROS mix of the old VTRST3.5 repo. I'd rather forge on here with a new release and agree with what @pasha-ponomarenko is saying.

I also agree that the package remaining unchanged allows it to stabilize before we put it out. I'm not tied to any particular date here, though I think there should be several weeks between release branch creation and the new release. During this time, we'll need to clear up merge conflicts with the current master so that it's ready to be merged. The idea here was to create a new master branch in time for the workshop so that it could be made known there's a new RST and where people can go to get it.

My apologies here with not understanding needing to clean up and reorganize code is necessary with the new gridding and mapping you've been working on. From the outside it seems as though these could be unrelated.

So, I'm OK with another week if you think there can be more done then. It may be easier for me to get to testing in the first couple days of May than next week.

[ksterne]

I've pushed the release-4.0 branch up now. I'll make the pull request shortly. So the goal here is to only use this branch and any small minor bugs should be fixed in this branch. I'll flip VT's main processing code over to this in the next couple of days.

Should we keep this issue open to discuss things with the release branch or is it good to close now?

[egthomas]

I think this issue is safe to close as discussion has shifted to the pull request ( #54 ).