Build	Status
brew test-bot GitHub action

homebrew-simulation

Homebrew tap for osrf simulation software

To use:

brew tap osrf/simulation
brew install gz-harmonic

Bottle status

Status	Citadel	Fortress	Garden	Harmonic	Ionic
gz-cmake
gz-common
gz-fuel-tools
gz-gui
gz-launch
gz-math
gz-msgs
gz-physics
gz-plugin
gz-rendering
gz-sensors
gz-sim
gz-tools
gz-transport
gz-utils
sdformat
collection

Gazebo Classic

• gazebo-classic11

To build bottles

The https://build.osrfoundation.org jenkins instance is used for building bottles with the following job (configured in brew_release.dsl):

• https://build.osrfoundation.org/job/generic-release-homebrew_triggered_bottle_builder/

This jenkins job is triggered for pull requests when an administrator makes a comment on the pull request that includes the phrase build bottle. The job should appear in the GitHub build status interface for the latest commit:

For example, #1157 was created after running our release.py script and this comment triggered the bottle build, resulting in a successful bottle upload and a4793387.

Bottle builds are not triggered automatically for every pull request for several reasons:

• Not all pull requests require a bottle to be rebuilt (such as #1007 that added this text to the README)
• Successful bottle builds result in binary artifacts being immediately uploaded to our hosting provider so pull requests should be screened for malicious intent by administrators before triggering a bottle build.
  • This process differs from the approach taken by homebrew/homebrew-core whose bottles are hosted at GitHub Packages, which hosts files according to the hash of their contents rather than by filename. The homebrew-core CI jobs will build bottles for any incoming pull request, which will upload bottles directly to GitHub Packages, but the SHA hash associated with these bottles will not be easily available unless the pull request is merged by a homebrew maintainer.

To disable broken bottles

When a new major or minor version of a formula is merged to homebrew-core that is a dependency of formulae in this tap, it may break our bottles, requiring a rebuild (see #1728 or #1708 for example). As the osrf/simulation bottles will be broken immediately upon the merge in homebrew-core, it can be useful to disable the broken bottles while waiting for new bottles to be rebuilt. The --remove-bottle-block parameter to brew bump-revision can be used in this instance. For example, the bottle removals from 8ca61f3d in #1742 could be repeated with the following commands:

brew bump-revision --remove-bottle-block ignition-msgs1
brew bump-revision --remove-bottle-block ignition-msgs5
brew bump-revision --remove-bottle-block ignition-msgs6
brew bump-revision --remove-bottle-block ignition-msgs7
brew bump-revision --remove-bottle-block ignition-msgs8

If all broken formulae properly list an explicit dependency on the breaking formula, the following shell script loop can be used to remove all broken bottles. For example, the following snippet was used to remove broken protobuf bottles in https://github.com/osrf/homebrew-simulation/issues/2314#issuecomment-1626396384:

cd `brew --repo osrf/simulation`/Formula
for f in $(grep -l '^ *bottle do' $(grep -rlI depend.*protobuf .) | sort)
do
  brew bump-revision --remove-bottle-block --message="broken bottle" $f
done

Troubleshooting

• Does a new bottle need to be built for every homebrew pull request?

  • A new bottle is not needed for all pull requests. For example, updates to documentation or a formula's test do block do not change the installed binary and thus don't require a new bottle. Changing the tarball url, adding patches, or bumping the formula revision do require new bottles. If you aren't sure, just ask in the pull request.

• Do I need to wait for the brew test-bot / test-bot GitHub action to succeed for starting a bottle build?

  • No, you don't need to wait. You can start the bottle build as soon as the pull request is opened.

• When can I merge a pull request? Does CI need to be finished?

  • Yes, CI must be finished and successful. If a new bottle is needed, the generic-release-homebrew_triggered_bottle_builder job must be successful as well.

• I commented build bottle, but it did not start a generic-release-homebrew_triggered_bottle_builder job.

  • Confirm that the build.osrfoundation.org web page loads. If it is not accessible, an OSRF build farmer should be notified.
  • If build.osrfoundation.org is operational, confirm that you have adequate permissions. Currently, you must be a member of the github.com/ignitionrobotics org in order to use the build bottle trigger phrase (see configuration in brew_release.dsl).

• I'm updating a Formula to build from a specific commit in a git repository. How do I get the sha256 for the tarball corresponding to that commit?

  • First, make sure that you have updated the url to use the commit hash that corresponds to the commit in the repository that you'd like to use. Also be sure to update the version, if it exists (in the example linked here, 20201028~c02cd0 is the part that needs to be modified: 20201028 is the date (year-month-day), and c02cd0 is the first 6 characters of the commit was used in the updated url).
  • Now, run the command wget <url>, where <url> is the updated url that was just mentioned. Once you have the tar file downloaded, run the command sha256sum <file>, replacing <file> with the file that was downloaded via wget. The sha256 will be printed to the console, which can then be used to update the Formula's sha256.

• I ran the release.py script multiple times for the same release and commented build bottle on the pull request, but the bottle building job failed, with console output containing the text Warning: Formula reports different SHA256:.

  • It's possible that the tarball uploaded at the time the pull request was created was overwritten by a subsequent call to release.py (see ignition-tooling/release-tools#274). If so, update the sha256 field for the tarball (see #1156 and 57fa5defcce for an example).

Jenkins implementation details

The generic-release-homebrew_triggered_bottle_builder jenkins job currently builds bottles for macOS 10.15 catalina and 10.16 big_sur using the following job configurations and the homebrew_bottle_creation.bash script:

• https://build.osrfoundation.org/job/generic-release-homebrew_triggered_bottle_builder/label=osx_catalina
• https://build.osrfoundation.org/job/generic-release-homebrew_triggered_bottle_builder/label=osx_bigsur

If the bottle building job finishes without errors for each build configuration, it will trigger a subsequent repository_uploader_packages job that uploads the bottles to s3 and a generic-release-homebrew_pr_bottle_hash_updater job that commits the changes in bottle sha256 values to the pull request branch using this script.

Building bottles for newly supported macOS distributions

When we add support for a new version of macOS, we need to build bottles for that formula, while ideally keeping the existing bottles. This can be done by using the --keep-old parameter with brew test-bot and brew bottle. Since ignition-tooling/release-tools#556, bottle builds can be triggered for a specified version of macOS using --keep-old by adding special tags to the build bottle comment in a homebrew-simulation pull request. Use brew-bot-tag: along with build-for-new-distro-{distro} in the comment, where {distro} is the version string used in homebrew bottle blocks (such as catalina or big_sur). See this comment in osrf/homebrew-simulation#1694 as an example that triggered a bottle build for big_sur only. Note that the --keep-old flag only works if the pull request does not change the formula version. Adding a comment to a formula (as in osrf/homebrew-simulation#1694) is sufficient.