timing of next release 0.9-19

[dankelley]

I was checking to see if the new R was available for osx and looked at the oce check page

https://cran.cnr.berkeley.edu/index.html

where I see errors on every platform. I had thought that we would get errors only on the devel R version. I don’t like seeing errors and think we should make a new release. Right now, the main thing (in my arena) seems to be units for sbe, and I think I have that working quite well. (I didn’t do the whole table in the manual, but things are okay for the 3 test files we have, so I really feel it’s ready for users to test.) A proposed plan is below. Seem okay to you? (I know this leaves aside a lot of rsk work, but we don’t have users asking for those changes and I’d prefer to get rid of all these errors that apply to all users.)

• now until mid-May: code slush--bug fixes and overdue improvements (e.g. re accessors #938)
• now until mid-May: both of us do some thinking about whether the existing flag syntax seems sensible. This is the main new feature and we don't want to have to break the UI we set in this release.
• late May: code freeze--fix bugs and doc improvements (with lots of winbuild checks, and everyday usage)
• end May: release to CRAN

This is actually an updated scheme, bumped by 2 weeks because of lots of new SBE issues. Since SBE is important to many users, it's important to have these issues cleared up and a lot of testing, so unless things are looking good by perhaps May 15, I will add 2 weeks to the end date, and keep doing that.

[richardsc]

One thing to consider is that I'll be doing a workshop on May 28th, so it would be nice to have an updated CRAN version before then.

It's not the end of the world if that's not possible -- I could always direct people to install from pre-built tarballs if necessary.

[dankelley]

Right. Maybe we'll have to see on the timing. The CRAN process can take up to a week and this SBE work is really fairly tricky ... and I feel quite bad about the present CRAN having these found.SOMETHING errors, which I put down to a release before sufficient user-testing time.

PS. I have decided I don't worry much about the CRAN build-test failures. Those were occasioned by another package having been updated, after all.

[dankelley]

I think SBE problems associated with the new scheme of deriving the Oce name from SBE short-name were an unexpected release-blocker for a while. But now, (a) we decode almost 1/2 of the official SBE names (#962) and (b) an unrecognized name now yields a warning that should be very helpful and (c) we have the columns argument working properly (#944). So, from my point of view, SBE is in good shape.

I propose that we continue to dump cnv files in the dropbox site to test. Just type make afterwards and look for warnings (which appear in the unix terminal ... no need to look in the .out file) and post a comment (at #962) so I can add the short-name. This will give us a read.ctd.sbe that's good enough for a CRAN release, I think.

However, I do have an SBE question, #965.

[dankelley]

I had thought for a while that a cran submission was not in the cards before the workshop but now, looking at the issues, I think it should be considered.

I am quite happy with mapping again. I think that SBE has come a long way, too, with a good warning message and reasonable behaviour for unknown SBE names. (Doing all listed SBE names will be ~30h of clear-minded coding, which we won’t get by the end of the month.)

My perspective is partly based on the issues, and partly based on the complexity of solving them. For example, #956 (use all rsk metadata in making into ctd) is simple to state but the code is a bit too tangled to make it a quick fix. Besides, there are several outstanding rsk issues, so I think people using rsk are doing to be working from github "releases" anyway.

Maybe our criterion for a release should be that normal, cran-based, users get new features that will help them in their work. I definitely think we are at that stage. I've been busy with other things and have not had what I consider to be sufficient time for extensive real-world use of oce. But, through this last period of development I've been careful to add testthat tests to the build sequence, and that gives me some confidence in the code.

What are your thoughts, Clark? If we do decide to go for a release, I think we'd both need to get serious about running some tests. My main tests are building OAR, building my course notes, etc. I don't quite feel that rebuilding sleiwex is a good test, because oce has changed in the intervening years. Certainly oce has been fine in my everyday work, so I'm not worried about basic things.

[richardsc]

I think it's worth trying for, given the things that have been fixed/added since the last release and with the workshop coming up. I'll probably be testing a fair bit while as I put together materials for the workshop, so I'm game to go into a feature-freeze and just look for fixes.

[dankelley]

Bear in mind that the cran process can take 3 to 10 days, even if we are lucky. So we don't have tons of time. On the other hand, we don't really have to worry a whole lot if we don't get it built up before the workshop because

• linux people can build from source
• windows is not too hard to build AND I can always prepare binaries on dropbox (using win-builder)
• osx can be handled by prepared binaries on dropbox

[richardsc]

Good point on the built binaries -- that is a good way to get people setup with only a little preparation.

I'm still open to trying even if we don't manage to get all the way through before the workshop.

[dankelley]

I think pressure is good, but I also am not totally keen on submitting something to cran that turns out to be deficient in some way that we stems from rushing. The whole 'read more sbe' topic has been really very useful for users, I think, and it has also encouraged me to do similar for two other ctd/section data types. This is all really fantastic -- and it should have been done 2 years ago -- but it involves a lot of code-base changes so I'm a bit scared of breaking things. I think we have had only one release that was actually broken, and that was the last one, with the found.pressure (etc) bug. That came about not because of a new feature that was poorly coded (for these seldom happen) but rather because of adjustments made to old code. That's really my worry. A lot of things lately have been changing old code, and all I've done is (1) try to be careful and (2) test with my own data. Granted, we now have a fairly large .cnv test suite, but that's not at all the case on section data, argo data, etc ... we have only a few test cases.

With a workshop on the 28th, and allowing 8 days to filter through cran (assuming zero errors or arguments with the cran bosses), we have just 1 week to settle things. I favour trying to do that, for sure, but if by next Wednesday I would only want to make the final push if everything "felt right".

PS. there is that manufacturer issue that I am about to comment on. I don't know if that's an issue since we don't have much info. You handled the previous occurrence of that one and I hope you might have insights on the new one. If that part of the code is really broken, I'd like to fix it.

[dankelley]

1. I've done quite a lot of work on some of the recent rsk issues and am gaining confidence that we can imagine a cran submission in the upcoming week. But I need Clark's clear-headed approval, after he has had time to look at the rsk changes. I don't use rsk data enough to have any of my own work, with which to check the changes.
2. From my perspective, nothing except my worry with rsk is blocking a cran submission. But, again, I'd like to get Clark's view on that. I feel that the previous submission had been done with insufficient testing, given that we caused problems for several users with the found.SOMETHING bugs.

[dankelley]

Offline discussion had the conclusion: we hold off on a CRAN release until after the CMOS workshop. That way, we can get the benefit of feedback from new users.

[dankelley]

@richardsc let's get back to talking about release dates.

1. Did any bugs come up during the workshop?
2. Do you figure we should reset the timer and have a week of usage, for a release next Saturday, perhaps?

I vote +1 on Q2, unless we get some release-blocker bugs posted. I guess an argument could be made to work through the 100+ remaining SBE names (#962) but I think we have a really good fallback position and I'd rather add things when people need them, plus maybe a few new names every weekend (that's a lot of weekends though).

[richardsc]

Sorry for being silent -- it was a busy week and then a long drive back home.

Do you figure we should reset the timer and have a week of usage

I think this is a good idea. To my knowledge there were no bugs that came up because of the workshop. I don't think we should try making coding all those SBE names a release-blocker. If more come up, we can add them when they do.

Perhaps we can have a phone chat this week sometime to go over plans.

[dankelley]

Summary of phone meeting:

1. Do not attempt to do all SBE names. Just leave #962 to its own timetable.
2. DK and CR will continue to try to find .cnv files for ~/Dropbox/oce-working-notes/CNV/torture_test. Note that the trick used in the code in that directory is to make warnings into errors, so a 'make' is all it takes to test a lot of files.
3. DK will work a bit on #981 to see whether changes can be made without too much ripple effect. If changes are thought to have a ripple effect, though, then #981 will be left to after CRAN release. (Some rough work in the kelley branch shows some promise ... I just need to see how to insert expressions within expressions.)

[dankelley]

As I see it, we have one issue that seems serious enough to block release ( #989 ) but I sort of think it's been addressed. I propose a release on next Friday (St Jean Baptiste Day), unless we get some big new bugs.

[richardsc]

I agree with a release on SJBD!

[dankelley]

FYI, the dropbox now has osx and windows updated to develop commit b9714a81d2124e75930eec732b95923ca6d6ad6f

[dankelley]

The dropbox now has osx and windows updated to develop commit 8bb2324433107a7eaa94d89e04f49234f5f1a905

[dankelley]

The dropbox now has osx and windows updated to develop commit ae68d2753485b150d292f27d4cc4681608d470a1

[dankelley]

develop has been merged into master, yielding master commit 8768f5514528fa75eeb582b9ca602105003e78bc

[dankelley]

Status check, submission day minus 1:

• DK: "go"

[richardsc]

CR: "go" (though I'm about to open a discussion issue that shouldn't necessarily delay the release)

[dankelley]

I took some time on this extra discussion issue ( #995 and #997 ) and think things are okay on that front, now. So, I think we are back on the 'monitor a few days, then submit' track.

[dankelley]

commit e2e09f4dfe976a3fbc503fdc640ada746a94fce8 Author: Dan Kelley kelley.dan@gmail.com Date: Fri Jun 24 15:52:27 2016 -0300

fix hyperlinks to terralook website

[dankelley]

I merged develop into master (commit 1864c95a701e1200127c383c6181874398c611bb) and updated the OSX and MSwindows binaries on dropbox. I'll keep submitting to winbuilder every day, because that catches broken URLs, which we certainly don't want on the day of submission.

[dankelley]

I merged develop into master and rebuilt the vignette (commit 3cb3aa7c57f54f1262fd90a3432f1f03926bea54), then updated the OSX and MSwindows binaries on dropbox. With no cran-blocking bugs listed, the plan is to sit and wait until Friday, Jul 1, between 8AM and 9AM, at which time I plan to submit to CRAN.

[dankelley]

I propose pushing the submission date. We've had many issues in play over the past several days, and I worry about code ripples. Friday is only a "magic" day to some of us ... I'd prefer to let oce settle a bit and submit in a week's time, perhaps. Seem ok, @richardsc ?

[richardsc]

So, countdown for Friday July 8? I agree.

[dankelley]

I've merged the latest develop into master, and put the built-up OSX and windows master in the dropbox.

[dankelley]

Submitted to CRAN earlier today. So far, fedora linux and osx have been built by the system. Time to close this issue. (PS it was a bad name ... it should have had the version number in it...)

[richardsc]

Woo hoo!

[AnneMTreasure]

Congratulations! Thank you for a fantastic package!