Release GMT 6.0.0rc1 - Githubissues

seisman commented 5 years ago

Version: 6.0.0rc1

Before release:

[x] run src/gmt_make_*.sh to update some .c and .h files
[x] run coverity scan and fix defects (@joa-quim)
[x] check if all tests pass on macOS, Linux and Windows
[x] update changelog
[x] update INSTALL.md
[x] build documentations and fix warnings
[x] check/change cmake/ConfigDefault.cmake
- [x] GMT_VERSION_YEAR is current year
- [x] GMT_PACKAGE_VERSION_* is correctly set
- [x] GMT_LIB_SOVERSION is correctly set
- [x] set GMT_PUBLIC_RELEASE to TRUE
[x] freeze codes and commit all changes to GitHub

Release:

[x] create source packages (tar.gz and tar.xz) (@PaulWessel)
[x] create macOS Bundle (@PaulWessel)
[x] create Windows binary packages (win32 and win64) (@joa-quim)
[x] make a tag and push it to github
```
git tag 6.0.0rc1
git push --tags
```
[x] go to GitHub Release and make a release. Remember to attach the source packages, Windows binary packages and macOS Bundle.
[x] upload source packages, Windows binary packages and macOS Bundle to the GMT FTP (@PaulWessel)
[x] update README and VERSION files on the GMT FTP
[x] announcements

After release:

[x] update GMT_PACKAGE_VERSION_* in cmake/ConfigDefault.cmake
[x] set GMT_PUBLIC_RELEASE to FALSE

3rd-party updates:

[x] update conda packages via conda-forge/gmt-feedstock (@leouieda, @seisman)
[x] update homebrew formula (@seisman)
[x] update fink package (@remkos)
[x] update macports ports
[x] Party :tada: (don't tick before all other checkboxes are ticked!)

PaulWessel commented 5 years ago

Actually, Joaquim does the Coverity but we should try to automate that as well

joa-quim commented 5 years ago

Two things more: I don't think we should call it a beta version. Modern nomenclature calls them RC (Release Candidates). A RC is promoted to the final thing if nothing wrong shows up. If it does, we fix and release RC2, etc ...

The other thing is that we must create a 6.0 branch.

seisman commented 5 years ago

I don't think we should call it a beta version. Modern nomenclature calls them RC (Release Candidates). A RC is promoted to the final thing if nothing wrong shows up. If it does, we fix and release RC2, etc ...

Agree.

The other thing is that we must create a 6.0 branch.

Do we really need a 6.0 branch now? I thought it's only needed after the 6.0.0 official version is out.

joa-quim commented 5 years ago

Sure, branching after the release should be just fine.

PaulWessel commented 5 years ago

I just fixed a bug in gmtselect but we need to stop and just release since it will take some time to just do the building and releasing, especially where the products will be placed, etc. I think we have done some of the items on the list (Coverity several times but that is never ending), @dongdong updated the include files I think Tests pass (minus 2 on purpose) for me. How do we move forward? Not sure if @leouieda is up and about yet.

joa-quim commented 5 years ago

I launched another Converity hours ago but it apparently stalled. Will also make a PR commenting one of the tests in proj4.sh because there is some bug in GDAL3 that is making it fail.

joa-quim commented 5 years ago

We should do this today or at most tomorrow. Wednesday I'll still be arround but the rest of the week I'll be in London without a computer.

PaulWessel commented 5 years ago

Yes, I will be gone Wednesday too. Tuesday is the very latest.

PaulWessel commented 5 years ago

We really have no ChangeLog anymore. Not easy to auto-craft one from the git logs either. Thoughts? We do not have a good list of new features either, even if we ignore listing actual bug fixes.

PaulWessel commented 5 years ago

Also, the tag: SHould it not just be 6.0.0RC1 or something?

seisman commented 5 years ago

Also, the tag: SHould it not just be 6.0.0RC1 or something?

The tag should be 6.0.0rc1 (most software use lowercase rc).

seisman commented 5 years ago

We really have no ChangeLog anymore. Not easy to auto-craft one from the git logs either. Thoughts? We do not have a good list of new features either, even if we ignore listing actual bug fixes.

Changelog is important to motivate users to upgrade to the newest version. Since we have limited time now, we can skip the changelog temporarily. I think we still need a changelog in the feature.

PaulWessel commented 5 years ago

Yes, I agree we need one for 6.0.0 proper. Can we start checking of boxes on the top? We did Coverity but that is more of an ongoing thing - there are always more cases. tests for me run as expected (2 displayed failures on purpose - several hidden), documentatino is in reasonably good shape. I'd like to get through those steps so I can build the tarballs and hand over to @joa-quim.

seisman commented 5 years ago

Yes, feel free to check off the list.

seisman commented 5 years ago

Will #978 be merged?

joa-quim commented 5 years ago

Tag are cheap but I think it should be done only on 6.0.0 (no RCs). And I also think Coverity can well go off the list.

PaulWessel commented 5 years ago

Just merged #978, forgot. Yes, let's' keep Coverity off the list.

joa-quim commented 5 years ago

And I think the first, gmtmake*.sh, can go too. For example I never run them (always do it manually) and whatever they may change is not going to be found right away (may even break something), since it afects only the external interfaces.

PaulWessel commented 5 years ago

I think the docs are built nightly and there are no errors (?). I just built locally and no problems but lots of warnings. I think we are not going to fix all those before the rc1 so I am checking that box too.

PaulWessel commented 5 years ago

Hi @joa-quim and @seisman. I've built the release tarball and macOS bundle. Please help check if they are OK and contain what we expect. You can test them via these links

ftp://ftp.soest.hawaii.edu/pwessel/rc1/gmt-6.0.0rc1-src.tar.gz ftp://ftp.soest.hawaii.edu/pwessel/rc1/GMT-6.0.0rc1-Darwin.dmg

If OK ,then @joa-quim can build the Win installers, which we should at least install and test to make sure all odd DLLs are in place.

PaulWessel commented 5 years ago

Sorry, the dmg does not have coastlines etc. Hold on...

seisman commented 5 years ago

@PaulWessel Can you also post the md5sums that we can verify?

PaulWessel commented 5 years ago

MD5 (gmt-6.0.0rc1-src.tar.gz) = 90eafb2728242ad1276d9d2f01e2722a MD5 (GMT-6.0.0rc1-Darwin.dmg) = 1ba010dacae68a14f6893db2ca2c94d2

I uploaded both after fixing missing GSHHG/DCW.

PaulWessel commented 5 years ago

Strange, installed from bundle into dummy account. Dont get splash screen when double-clicking icon, path is set (/Applications/GMT...) but gmt returns no response. I like to think it is just my account that is weird.

seisman commented 5 years ago

@PaulWessel I also don't have a splash screen.

$ gmt -Vd                                                                                                                                                                                                                                                        
gmt [DEBUG]: Obtained the ppid from parent: 53543
gmt [DEBUG]: Enter: gmtinit_new_GMT_ctrl
gmt [DEBUG]: GMT->session.SHAREDIR = /Users/pwessel/GMTdev/gmt-dev/rbuild/gmt6/share
gmt [DEBUG]: GMT->session.HOMEDIR = /Users/seisman
gmt [DEBUG]: GMT->session.USERDIR = /Users/seisman/.gmt [created]
gmt [DEBUG]: GMT->session.CACHEDIR = /Users/seisman/.gmt/cache [created]
gmt [DEBUG]: GMT: 0. Will try to find subdir=postscriptlight stem = PSL_custom_fonts suffix=.txt
gmt [DEBUG]: GMT: 1. gmt_getsharepath trying current dir
gmt [DEBUG]: GMT: 2. gmt_getsharepath trying USERDIR /Users/seisman/.gmt
gmt [DEBUG]: GMT: 3. gmt_getsharepath trying USERDIR subdir /Users/seisman/.gmt/postscriptlight
gmt [DEBUG]: GMT: 4. gmt_getsharepath trying SHAREDIR subdir /Users/pwessel/GMTdev/gmt-dev/rbuild/gmt6/share/postscriptlight
gmt [DEBUG]: GMT: 5. gmt_getsharepath failed
gmt [DEBUG]: Map distance calculation will be Cartesian
gmt [DEBUG]: Exit:  gmtinit_new_GMT_ctrl
gmt [DEBUG]: Enter: New_PSL_Ctrl
gmt [DEBUG]: Exit:  New_PSL_Ctrl
gmt [DEBUG]: Enter: gmt_manage_workflow
gmt [DEBUG]: Exit : gmt_manage_workflow
gmt [DEBUG]: Enter: PSL_beginsession

PaulWessel commented 5 years ago

Oops, it has pwessel paths in there. I need to do a fresh build with a different ConfigUser.cmake from mine. Do we have a minimal one posted for building releases, e.g., all settings except local paths to GSHHG/DCS to copy from?

seisman commented 5 years ago

This is what I use:

set (CMAKE_INSTALL_PREFIX "/opt/GMT-6.0.0rc1")

set (GSHHG_ROOT "/Users/seisman/Desktop/workspace/gmt-6.0.0rc1/share/coast")
set (DCW_ROOT "/Users/seisman/Desktop/workspace/gmt-6.0.0rc1/share/dcw")
set (COPY_GSHHG TRUE)
set (COPY_DCW TRUE)

set (GMT_INSTALL_MODULE_LINKS FALSE)
set (GMT_USE_THREADS TRUE)
set (GMT_ENABLE_OPENMP TRUE)

# recommended even for release build
set (CMAKE_C_FLAGS "-Wall -Wdeclaration-after-statement ${CMAKE_C_FLAGS}")
# extra warnings
set (CMAKE_C_FLAGS "-Wextra ${CMAKE_C_FLAGS}")

joa-quim commented 5 years ago

I rely in my master installation

    set (GSHHG_ROOT ${GMT_INSTALL_DIR}/master/share/coast)
    set (GSHHG_PATH ${GMT_INSTALL_DIR}/master/share/coast)

and the build works fine. gmt printed the plash message

seisman commented 5 years ago

I tried to compile the source tarball on macOS and Linux. Both work well.

PaulWessel commented 5 years ago

Good, still rebuilding the Bundle.

PaulWessel commented 5 years ago

This is what I use:
set (CMAKE_INSTALL_PREFIX "/opt/GMT-6.0.0rc1")

set (GSHHG_ROOT "/Users/seisman/Desktop/workspace/gmt-6.0.0rc1/share/coast")
set (DCW_ROOT "/Users/seisman/Desktop/workspace/gmt-6.0.0rc1/share/dcw")
set (COPY_GSHHG TRUE)
set (COPY_DCW TRUE)

set (GMT_INSTALL_MODULE_LINKS FALSE)
set (GMT_USE_THREADS TRUE)
set (GMT_ENABLE_OPENMP TRUE)

# recommended even for release build
set (CMAKE_C_FLAGS "-Wall -Wdeclaration-after-statement ${CMAKE_C_FLAGS}")
# extra warnings
set (CMAKE_C_FLAGS "-Wextra ${CMAKE_C_FLAGS}")
Can we use environment variables for the two paths? If so then developers can have $GMT_DCW_SOURCE and $GMT_GSHHG_SOURCE set and we could add the CMakeCOnfig snippet above to GitHub, after replacing stuff like set (DCW_ROOT "$GMT_DCW_SOURCE"), etc.

seisman commented 5 years ago

Yes, FindGSHHG.cmake and FindDCW.cmake already accept environmental variables $GSHHG_ROOT, $DCW_ROOT.

You can

export GSHHG_ROOT=/path/to/gshhg
export DCW_ROOT=/path/to/dcw

Then the minimal cmake snippet is:

set (CMAKE_INSTALL_PREFIX "/opt/GMT-6.0.0rc1")
set (COPY_GSHHG TRUE)
set (COPY_DCW TRUE)

PaulWessel commented 5 years ago

New bundle uploaded. MD5 (GMT-6.0.0rc1-Darwin.dmg) = 6d0fb517fee6183611a0685b261b788c

Works fine under mine account if I install it. Log out and in via dummy account and /Applications/GMT-6.0.0-.rc1 says it is 0 bytes and double-clicking yields "damaged file" error. I wonder if the bundle installs with restricted permissions?

seisman commented 5 years ago

It still don't work for me.

gmt defaults -Vd
gmt [DEBUG]: Obtained the ppid from parent: 77184
gmt [DEBUG]: Enter: gmtinit_new_GMT_ctrl
gmt [DEBUG]: GMT->session.SHAREDIR = /opt/GMT-6.0.0rc1/share
gmt [DEBUG]: GMT->session.HOMEDIR = /Users/seisman
gmt [DEBUG]: GMT->session.USERDIR = /Users/seisman/.gmt [created]
gmt [DEBUG]: GMT->session.CACHEDIR = /Users/seisman/.gmt/cache [created]
gmt [DEBUG]: GMT: 0. Will try to find subdir=postscriptlight stem = PSL_custom_fonts suffix=.txt
gmt [DEBUG]: GMT: 1. gmt_getsharepath trying current dir
gmt [DEBUG]: GMT: 2. gmt_getsharepath trying USERDIR /Users/seisman/.gmt
gmt [DEBUG]: GMT: 3. gmt_getsharepath trying USERDIR subdir /Users/seisman/.gmt/postscriptlight
gmt [DEBUG]: GMT: 4. gmt_getsharepath trying SHAREDIR subdir /opt/GMT-6.0.0rc1/share/postscriptlight
gmt [DEBUG]: GMT: 5. gmt_getsharepath failed
gmt [DEBUG]: Map distance calculation will be Cartesian
gmt [DEBUG]: Exit:  gmtinit_new_GMT_ctrl
gmt [DEBUG]: Enter: New_PSL_Ctrl
gmt [DEBUG]: Exit:  New_PSL_Ctrl
gmt [DEBUG]: Enter: gmt_manage_workflow
gmt [DEBUG]: Exit : gmt_manage_workflow
gmt [DEBUG]: Enter: PSL_beginsession

PaulWessel commented 5 years ago

Well, we left off a few key steps per Florian's writeup:

set (CMAKE_BUILD_TYPE relwithdebinfo) set (CMAKE_INSTALL_PREFIX "gmt-${GMT_PACKAGE_VERSION}") set (GMT_RELEASE_PREFIX ${GMT_BINARY_DIR}/gmt-${GMT_PACKAGE_VERSION})

Working on a new on now.

leouieda commented 5 years ago

Working on the conda package now.

Tag are cheap but I think it should be done only on 6.0.0 (no RCs).

I disagree. The rc is a release, though not a stable release. It's important to know exactly which commit was rc1 vs rc2 etc. As you mentioned, tags are cheap.

leouieda commented 5 years ago

On that note, since we have no tag I don't know which commit is rc1. I'm assuming it's the current top of master (f75038ab617363e358a1555b8d54b61ff6e37660) is that correct?

seisman commented 5 years ago

On that note, since we have no tag I don't know which commit is rc1. I'm assuming it's the current top of master (f75038a) is that correct?

Yes. But maybe you should use the source tarball rather than a commit hash.

PaulWessel commented 5 years ago

Yes, not made any commit since the one that set GMT_PUBLIC_RELEASE to true this morning.

leouieda commented 5 years ago

Yes. But maybe you should use the source tarball rather than a commit hash.

True. I'll get it started on the commit to check if things work and will update to the tarball once that is uploaded.

joa-quim commented 5 years ago

And why is important to know which commit is the RC (the tarball)?

leouieda commented 5 years ago

Fair point. My thinking if that it helps with record keeping. Same reason why we want to know exactly which commit is 6.0.0 or 5.4.6.

PaulWessel commented 5 years ago

Looks like cmake --build . --target gmt_release doesn't build supplements.so. cpack failed for taht reason. I did a ninja -j20 install and then it built the supplements.

PaulWessel commented 5 years ago

Still strange, new bundle: MD5 (GMT-6.0.0rc1-Darwin.dmg) = c1368be52c8255594d092ce6eb0252d6 It installs for me with only access for me, no access for other users. Not sure where this is set - is it perhaps my system defaults and not something with the build. I manually change the permissions. Now the dumb users sees the application at 206 Mb (and not zero) but running gives same result as @seisman gets.

seisman commented 5 years ago

@PaulWessel I tried to build a bundle myself, and have the same problem.

I think the problem is GMT_SHAREDIR or GMT->session.SHAREDIR.

See codes below. If we don't define any GMT_SHAREDIR environmental variables, then GMT->session.SHAREDIR is set to GMT_SHARE_DIR, which is @CMAKE_INSTALL_PREFIX@/@GMT_DATADIR@ (i.e. /opt/GMT-6.0.0rc1/share in our case). https://github.com/GenericMappingTools/gmt/blob/f75038ab617363e358a1555b8d54b61ff6e37660/src/gmt_init.c#L2689-L2690

So GMT->session.SHAREDIR uses a hard-coded value. Can cmake or cpack change such hard-coded value?

seisman commented 5 years ago

In the 5.4 branch, I see some codes, which were removed in GMT 6. See here: https://github.com/GenericMappingTools/gmt/blob/ac6ac2fe5698dd63bd8bd6701ea85788e01ebc5b/src/gmt_init.c#L2444-L2457.

I tried to add these codes back (with some changes), and the bundle works.

The output is:

gmt -Vd
gmt [DEBUG]: Obtained the ppid from parent: 48226
gmt [DEBUG]: Enter: gmtinit_new_GMT_ctrl
gmt [DEBUG]: GMT->session.SHAREDIR = /Applications/GMT-6.0.0rc1.app/Contents/Resources/share
gmt [DEBUG]: GMT->session.HOMEDIR = /Users/seisman
gmt [DEBUG]: GMT->session.USERDIR = /Users/seisman/.gmt [created]
gmt [DEBUG]: GMT->session.CACHEDIR = /Users/seisman/.gmt/cache [created]
gmt [DEBUG]: GMT: 0. Will try to find subdir=postscriptlight stem = PSL_custom_fonts suffix=.txt
gmt [DEBUG]: GMT: 1. gmt_getsharepath trying current dir
gmt [DEBUG]: GMT: 2. gmt_getsharepath trying USERDIR /Users/seisman/.gmt
gmt [DEBUG]: GMT: 3. gmt_getsharepath trying USERDIR subdir /Users/seisman/.gmt/postscriptlight
gmt [DEBUG]: GMT: 4. gmt_getsharepath trying SHAREDIR subdir /Applications/GMT-6.0.0rc1.app/Contents/Resources/share/postscriptlight

PaulWessel commented 5 years ago

OK, good, was just going to post that my dumb users can install 5.4.5 successfully from the bundle so something clearly changed. Please make a PR, but presumably not the SVN_REVISION stuff?

seisman commented 5 years ago

It would be better if you can make a PR yourself.

I just copy the codes above from 5.4 branch, and remove Lines 2444-2445 (because gmt_verify_sharedir_version was removed) and GMT_PACKAGE_VERSION_WITH_SVN_REVISION.

PaulWessel commented 5 years ago

OK, so do I just skip the check on GMT_SHARE_DIR since no longer have a gmt_verify_sharedir_version and we do not set that parameter?. I.e., it would become

    if ((this_c = getenv ("GMT6_SHAREDIR")) != NULL)    /* GMT6_SHAREDIR was set */
        GMT->session.SHAREDIR = strdup (this_c);
    else if ((this_c = getenv ("GMT5_SHAREDIR")) != NULL)   /* GMT5_SHAREDIR was set */
        GMT->session.SHAREDIR = strdup (this_c);
    else if ((this_c = getenv ("GMT_SHAREDIR")) != NULL) /* GMT_SHAREDIR was set */
        GMT->session.SHAREDIR = strdup (this_c);
#ifdef SUPPORT_EXEC_IN_BINARY_DIR
    else if (running_in_bindir_src)
        /* Use ${GMT_SOURCE_DIR}/share to simplify debugging and running in GMT_BINARY_DIR */
        GMT->session.SHAREDIR = strdup (GMT_SHARE_DIR_DEBUG);
#endif
    else {
        /* Still not found */
        GMT_Report (GMT->parent, GMT_MSG_NORMAL, "Error: Could not locate share directory that matches the current GMT version %s.\n",
          GMT_PACKAGE_VERSION_WITH_SVN_REVISION);
        GMT_exit (GMT, GMT_RUNTIME_ERROR); return GMT_RUNTIME_ERROR;
    }

leouieda commented 5 years ago

So this would require setting the environment variable?

GenericMappingTools / gmt

Release GMT 6.0.0rc1 #962

[x] update macports ports