search

gazebo-tooling / release-tools

8 stars 9 forks source link

Change release.py to use the new centralized tarball approach #1048

Closed j-rivero closed 8 months ago

j-rivero commented 8 months ago

The PR changes the release.py logic to be able to use the new generation of tarball that happens on Jenkins instead of using the dev local machines. See the linked docs update for a general usage and data flow overview https://github.com/gazebosim/docs/pull/403

No changes in releasing workflow and the release.py call: Note that the tag of the project code to be built still happens in the local dev system so the workflow is the same than now: go to the source code you want to build and run release.py with the same arguments than now. Instead of calling -debbuilders there will a chain of actions composed

PR is quite long, probably easier to read change by change to understand the logic:

  1. Remove all current code related to tarball generation 5f18737cdd47d918c30ecfec5dd569f2c71d7ffc
  2. Remove current --no-generate-source-file and clean up nightly generation 7887be0975ab0b100335d1ed9f76604b1c284c89
    1. Helper function to get the github repository uri from current directory:
      1. Originally implemented in 6a712ff36af9d6f480af3f0ba7a0876a8436d1d0 but patched afterwards in https://github.com/gazebo-tooling/release-tools/commit/a870d8c87abe56f6eb4c3dc63637ad8e6290f54a#diff-9f016d3777d731463c2969cd7d16bb014363bee6bf98af9fe7ca66c4392949ad
      2. Sanity check for right format added in dc06b9724615607672f98077112b1402fdb08d92
  3. Introduce new release.py parameters --source-repo-existing-ref and --source-tarball-uri. If --source-tarball-uri:
    1. Implement the transport of the parameters in the release call e786626b3c52eb88475b302c7420e6864c007cae
    2. Explain in --help what the new parameters do 8aca8acd5655fcb7a9f52dc386976ffb91100fee
  4. Change logic to make the two modes of calling to work: db39135071fea103f092ebccd592bb5b69bab6ac
    1. Standard new workflow calls to gz-foo-source directory as entry point
    2. Nightly and --source-tarball-uri calls will trigger gz-foo-debbuilders directly since no source code needs to be generated.
      1. Make the --bump-only-revision-linux parameter to need --source-tarball-uri cbb83e2

CI Testing:

  1. Implemented an smoke test job to be added to CI
    1. Change release.py to be use _RELEASEPY_TEST_RELEASE_REPO https://github.com/gazebo-tooling/release-tools/commit/479139235e9b7218960a1d35c91130049a75f0d8#diff-9f016d3777d731463c2969cd7d16bb014363bee6bf98af9fe7ca66c4392949ad
    2. See the bash testing in https://github.com/gazebo-tooling/release-tools/commit/a870d8c87abe56f6eb4c3dc63637ad8e6290f54a#diff-9074386702f4ff2677975dcd91d17dd99887e1e2a886f8234056e4a8ebefbf9b

Manual testing: Run the whole generation pipeline with the real example of gz-utils2 version 2.2.0~pre3. Steps were as follow:

  1. Usual previous steps to prepare the -release repository and the source code modifying CMakeLists.txt
  2. cd code/gz-utils
  3. Dry run ~/code/release-tools/release.py gz-utils2 2.2.0~pre1 <token> --dry-run --upload-to-repo prerelease to see the gz-source
  4. Run release ~/code/release-tools/release.py gz-utils2 2.2.0~pre1 <token> --upload-to-repo prerelease
    1. The release.py call triggers gz-utils2-source Build Status. Which after success, called:
      1. repository_uploader_packages to upload the tarball. In the parameters can be found the new tarball name and upload target. Build Status
      2. _releasepy (that will wait until all jobs of repository_uploader_packages finish) to call the debbuilders using --source-tarball-uri that points to the new tarball. The call and output is hidden in the a workspace log for security reasons. Build Status. This job will call --debuilders and the brew bottle builder as it is done until now.