v1.18.0 release

[shefty]

Update this procedure with any new providers. Copy/paste markdown as new issue for each new release, and update as necessary.

Make a perfect tarball

• For libfabric:
  • [x] For new minor releases, make stable v1.Y.x branch
  • [x] Cherry-pick relevant commits from master
  • [x] Update the libfabric version number in configure.ac to the final release version
  • [x] Update the version number in include/windows/config.h
  • Update the version numbers in include/rdma/fabric.h
  • [x] FI_VERSION_MAJOR should match first number in configure.ac
  • [x] FI_VERSION_MINOR should match second number in configure.ac
  • [x] FI_REVISION_VERSION should match the revision value (x in v1.Y.x)
  • [x] Update providers to support the version in fabric.h
  • Update ofi.h OFI_VERSION_LATEST, this will update most/all providers
  • Tip: also search for .fi_version and FI_VERSION.
  • [x] Update the shared library version number(s) in Makefile.am per the GNU Libtool shared library version number rules
  • Always check the libtool rules, but as general rules: Stable releases (v1.x.y - where y is updated) should never see API changes; stable release c:r:a versions should normally be updated to c:r+1:a. Minor releases (v1.x.0 - where x is updated) may see API additions or backwards compatible API changes. If there are no API changes, minor release c:r:a versions should be updated to c:r+10:a relative to the highest numbered stable release. The r+10 allows for up to 9 additional stable releases without resulting in a collision in the release numbers. If there are new APIs or backwards compatible changes, update c:r:a to c+1:0:a+1. Just say no to all non-compatible API changes.
  • Pro tip: run git log --stat --no-merges > log.txt as the head of the master
  • Examine this file all the way back to the Git tag for the previous release; look for changes to files in src/, include/, and include/rdma to help determine if the c:r:a version values need to change
  • Pro tip: run git log <last_version_tag>..HEAD -- include/rdma
  • Will show all changes to the external header files since the last release. The last_version_tag should be similar to v1.3.0
  • Update all documentation files (especially including dates and version numbers), including:
  • [x] README.md
  • [x] AUTHORS (via git log --pretty=format:'%aN <%aE>' | sort --ignore-case | uniq > AUTHORS)
  • [x] Update the years in the copyright notices in COPYING
  • Provider Feature Matrix (PFM)
    • [x] Add a link to this version's PFM on the main PFM wiki page
  • NEWS.md
    • [x] all providers
    • [x] copy changes from stable branch releases
    • [x] update release date
  • Man pages
    • [x] man/fi_<provider>.7.md
    • [x] man/fabric.7.md - Note: Update with any API/ABI updates
    • [x] fabtests/man/fabtests.7.md
• For fabtests:
  • Provide a release that matches the libfabric release number, even if there are no changes.
  • [x] Update the version number in configure.ac to the final release version
  • Update all documentation files (especially including dates and version numbers):
  • [x] README
  • [x] README.md
  • [x] Update the years in the copyright notices in COPYING
• [ ] make distcheck -- for final libfabric-x.y.z.bz2 tarball
• [ ] make distcheck -- for final fabtests-x.y.z.bz2 tarball

Ensure that the tarball actually works

• [ ] Ensure libfabric make distcheck passes
• Build and install libfabric (from a tarball)
  • [ ] Check that the version number and release date is correct in the installed man pages
• [ ] Build and install fabtests (from a tarball)
• [ ] Ensure fabtests make distcheck passes
• Ensure providers pass all (relevant) tests
  • [ ] Open PR with release commit, but do not merge
  • [ ] Wait for CI to complete successfully on PR, then close
• Build and install previous version of libfabric (from a tarball)
• Build and install previous version of fabtests against previous version of libfabric (from a tarball)
• rm -rf the installation of the previous version of libfabric
• Set LD_LIBRARY_PATH to point to the installation of the new libfabric
• Re-run all fabtests and ensure that they still pass

Test packaging - note external CI tests following steps

• [ ] Ensure that building a libfabric source RPM works on a RHEL system
• [ ] Ensure that building the libfabric binary RPMs (libfabric and libfabric-devel) work on a RHEL system
• [ ] Install both RPMs on an RHEL system
• [ ] Test building and running fabtests against the RPM-installed libfabric
• [ ] Test building a libfabric-based program with the output from pkg-config with the RPM-installed libfabric.pc
• [ ] Uninstall libfabric-devel and ensure libfabric programs still run correctly

Do the release

• [x] git tag -a vX.Y.Z on the libfabric master branch
• [ ] git push --tags --dry-run to libfabric. Remove --dry-run when you're convinced it is correct.
• [ ] Make a libfabric release corresponding to the git tag on https://github.com/ofiwg/libfabric/releases
• Copy the man/*.[0-9].md files from exactly the same Git hash that was used to make the official tarball to the gh-pages branch of the libfabric Git repo
  • [x] Put them in vX.Y.Z/man/
  • [x] Also copy fabtests/man/*.md here
  • [x] Also put an index.md file in that directory (copy one from the prior releases and edit it to reflect this version, and add/remove any man pages as relevant for this release)
• Update the main index.md page:
  • [x] Add links to the new libfabric and fabtests release tarballs
  • [x] Add link for the man pages for this release
• [ ] Close the relevant Github milestone in ofiwg/libfabric and ofiwg/fabtests
• [ ] Make a new milestone for the release after this one (e.g., v1.Y.x+1 or v1.Y+1.0)
• [ ] Re-target all open issues for an appropriate future milestone
• [ ] Ensure that new Github milestones exist in ofiwg/libfabric and ofiwg/fabtests for the next release
• [ ] Update the libfabric version number in configure.ac to <NEXT_VERSION>a1
• [ ] Update the fabtests version number in configure.ac to <NEXT_VERSION>a1
• [ ] Send an email to the ofiwg mailing list announcing the release

[shefty]

@wckzhang @shijin-aws -- Can someone from AWS send me a list of release notes for v1.18? I have notes for the other providers, but git log shows a number of updates to efa, and it would be better if someone more familiar with the changes could provide those to me. I can update NEWS.md; I just need the list.

[shijin-aws]

@shefty Here is the efa news

• Implement CUDA support on instance that does support GPUDirect RDMA
• Implement fi_write using device's RDMA write capability
• Enrich error messages with debug and connection info
• Implement the support of FI_OPT_EFA_USE_DEVICE_RDMA and FI_OPT_CUDA_API_PERMITTED in fi_setopt.
• Add support for neuron dma-buf
• Improve the intra-node CUDA communication performance for small messages by using gdrcopy
• Use shm provider's FI_AV_USER_ID support
• Fix bugs in efa provider’s shm info initialization procedure.

[shefty]

@shijin-aws - thanks, I've pulled those notes into NEWS