Bash script for continuous integration of Arduino projects. This is currently targeted for use with Travis CI but it could be easily adapted to other purposes.
The script can be used in multiple ways:
Include the latest release of the script in your project by adding the following lines to your build configuration file:
# Clone the script repository
- git clone --depth 1 https://github.com/per1234/arduino-ci-script.git "${HOME}/scripts"
- cd "${HOME}/scripts"
# Get new tags from the remote
- git fetch --tags
# Checkout the latest tag
- git checkout $(git describe --tags `git rev-list --tags --max-count=1`)
- source "${HOME}/scripts/arduino-ci-script.sh"
If you're passing a token to the script's publish report functions then best security practices would be to use a static copy of the script so you can be sure of the commands the token is used with:
- source arduino-ci-script.sh
Be sure to check for new releases of the script so that you can benefit from the ongoing development work. You can receive notifications of releases by watching the repository.
See https://github.com/per1234/WatchdogLog/blob/master/.travis.yml for an example of the script in use.
Please configure your continuous integration system to make the minimum number of downloads and sketch verifications necessary to effectively test your code. This will prevent wasting Arduino and Travis CI's bandwidth and keep the build durations short.
set_script_verbosity SCRIPT_VERBOSITY_LEVEL
Control the level of verbosity of the script's output in the Travis CI log. Verbose output can be helpful for debugging but in normal usage it makes the log hard to read and may cause the log to exceed Travis CI's maximum log size of 4 MB, which causes the job to be terminated. The default verbosity level is 0
.
0
, 1
or 2
(least to most verbosity).set_application_folder APPLICATION_FOLDER
check_code_formatting
) to. This should be set to /usr/local/share
or a subfolder of that location. The folder will be created if it doesn't already exist. The Arduino IDE will be installed in the arduino
subfolder.set_sketchbook_folder SKETCHBOOK_FOLDER
install_library
will be installed to the libraries
subfolder. Non-Boards Manager hardware packages installed via install_package
will be installed to the hardware
subfolder. This setting is only supported by Arduino IDE 1.5.6 and newer.set_board_testing BOARD_TESTING
Turn on/off checking for problems with the board definition that generate a warning message during sketch verification but don't ordinarily cause it to fail, such as missing bootloader file. If this is turned on and a problem is detected the build_sketch
command will return a non-zero exit status. This feature is off by default.
true
/false
set_library_testing LIBRARY_TESTING
Turn on/off checking for problems with libraries that generate a warning message during sketch verification but don't ordinarily cause it to fail, such as missing or invalid items in the library.properties file. If this is turned on and a problem is detected the build_sketch
command will return a non-zero exit status. This feature is off by default.
true
/false
all
: Refers to all versions of the Arduino IDE (including the hourly build). In the context of install_ide
this means all IDE versions compatible with the script (those that support the command line interface, 1.5.2 and newer). In the context of all other functions this means all IDE versions that were installed via install_ide
.oldest
: The oldest release version of the Arduino IDE. In the context of install_ide
this is the oldest of the IDE versions compatible with the script (1.5.2, the first version to have a command line interface). In the context of build_sketch this means the oldest IDE version that was installed via install_ide
.newest
: In the context of install_ide
this means the newest IDE release version. In the context of all other functions this means the newest IDE release version that was installed via install_ide
. 'newest' will only match to the hourly build if that is the only version available.hourly
: The hourly build of the Arduino IDE. Note that this IDE version is intended for beta testing only.install_ide [IDEversionList]
Install a list of Arduino IDE version(s).
'("1.6.5-r5" "1.6.9" "1.8.2")'
. If no arguments are supplied all IDE versions will be installed. The script allows you to install all IDE versions with a command line interface (1.5.2 and newer) for the sake of being complete but I don't see a good reason for testing with the 1.5.x versions of the Arduino IDE. Please only install the IDE versions you actually need for your test to avoid wasting Arduino's bandwidth. This will also result in a shorter build duration. Installation of the IDE will be skipped if it's found to already be installed in the folder specified via the set_application_folder
function so install_ide
can also be used simply to inform the script which IDE versions are available.install_ide startIDEversion [endIDEversion]
Install a range of version(s) of the Arduino IDE.
install_package
"Manually" install the hardware package from the current repository. Packages are installed to $SKETCHBOOK_FOLDER/hardware
. Assumes the hardware package is located in the root of the download or repository and has the correct folder structure.
install_package packageURL
"Manually" install a hardware package downloaded as a compressed file. Packages are installed to $SKETCHBOOK_FOLDER/hardware
. Assumes the hardware package is located in the root of the file and has the correct folder structure.
http://
, https://
) is required.install_package packageURL [branchName]
"Manually" install a hardware package by cloning from a Git repository. Packages are installed to $SKETCHBOOK_FOLDER/hardware
. Assumes the hardware package is located in the root of the repository and has the correct folder structure.
http://
, https://
) is required. The URL must end in .git
.install_package packageID [packageURL]
Install a hardware package using the Arduino IDE (Boards Manager). Only the Arduino AVR Boards package is included with the Arduino IDE installation. Packages are installed to $HOME/.arduino15/packages
. You must call install_ide
before this function. This feature is only available with Arduino IDE 1.6.4 and newer.
package name:platform architecture[:version]
. If version
is omitted the most recent version will be installed. e.g., arduino:samd
will install the most recent version of Arduino SAMD Boards.install_library
Install the library from the current repository. Assumes the library is in the root of the repository. The library is installed to the libraries
subfolder of the sketchbook folder.
install_library libraryName
Install a library that is listed in the Arduino Library Manager index. The library is installed to the libraries
subfolder of the sketchbook folder. You must call install_ide
before this function. This feature is only available with Arduino IDE 1.6.4 and newer installed.
install_library libraryURL [newFolderName]
Download a library in a compressed file from a URL. The library is installed to the libraries
subfolder of the sketchbook folder.
http://
, https://
) is required. The download file can be in any compressed file format. Assumes the library is located in the root of the file.{repository name}-{branch name}
. Library folder names that contain -
or .
are not compatible with Arduino IDE 1.5.6 and older, arduino will hang if it's started with a library using an invalid folder name installed.install_library libraryURL [branchName [newFolderName]]
Install a library by cloning a Git repository. The library is installed to the libraries
subfolder of the sketchbook folder. Assumes the library is located in the root of the repository.
http://
, https://
) is required. The URL must end in .git
.-
or .
are not compatible with Arduino IDE 1.5.6 and older, arduino will hang if it's started with a library using an invalid folder name installed. If the newFolderName
argument is specified the branchName
argument must also be specified. If you don't want to specify a branch then use ""
for the branchName
argument.set_verbose_output_during_compilation verboseOutputDuringCompilation
Turn on/off arduino
verbose output during compilation (same as the IDE's File > Preferences > Show verbose output during: > compilation). This will show all the commands arduino
runs during the process rather than just the compiler output. This is usually not very useful output and only clutters up the log. This feature is off by default.
true
/false
check_sketch_structure searchPath
Check sketches to ensure they have the correct structure.
check_library_structure basePath [depth]
Check libraries to ensure they have the correct structure. This will also run check_sketch_structure
on all sketches bundled with the library.
basePath
where the libraries are located. A depth of 0 will check the library located at basePath
. The default value is 0.check_library_properties searchPath [maximumSearchDepth]
Check library.properties library metadata files for errors.
searchPath
and no subfolders. The default value is 0.check_keywords_txt searchPath [maximumSearchDepth]
Check keywords.txt files for errors.
searchPath
and no subfolders. The default value is 0.check_library_manager_compliance libraryPath
Make some additional checks for compliance with the requirements for adding a library to the Library Manager index. This function should be used in combination with check_library_structure
and check_library_properties
to ensure full compliance with the requirements.
check_code_formatting strictness excludedPathList targetPath
Check code formatting for compliance with the Arduino code style. The Artistic Style formatter tool is used for this check. If it's not already installed, it will be installed to the astyle
subfolder of the folder specified to set_application_folder
. Note that in the Travis CI job logs, the check_code_formatting
output is "folded" to make it easier to browse. You can click the triangle in the left margin of the command to unfold the output.
1
: The Arduino IDE's auto format configuration.2
: The configuration Arduino uses to format their example sketches.3
: A custom configuration based on a study of the prevailing styles used in official Arduino code.build_sketch sketchPath boardID allowFail IDEversion
build_sketch sketchPath boardID allowFail [IDEversionList]
build_sketch sketchPath boardID allowFail startIDEversion endIDEversion
Verify/compile sketch(es). build_sketch
will echo the arduino
exit status to the log, which is documented at https://github.com/arduino/Arduino/blob/master/build/shared/manpage.adoc#exit-status. Note that in the Travis CI job logs, the compilation output is "folded" to make it easier to browse. You can click the triangles in the left margin to unfold.
package:arch:board[:parameters]
ID of the board to be compiled for. e.g., arduino:avr:uno
. Board-specific parameters are only supported by Arduino IDE 1.5.5 and newer.true
, require
, or false
. Allow the verification to fail without causing the CI build to fail. require
will cause the build to fail if the sketch verification doesn't fail.'("1.6.5-r5" "1.6.9" "1.8.2")'
. If no version list is provided all installed IDE versions will be used.display_report
Echo a tab separated report of all verification results to the log. The report is located in the ${HOME}/arduino-ci-script_report
folder and will be named according to the build number and job number. Note that Travis CI runs each build of the job in a separate virtual machine so if you have multiple jobs you will have multiple reports. The only way I have found to generate a single report for all tests is to run them as a single job. This means not setting multiple matrix environment variables in .travis.yml's env
array. See: https://docs.travis-ci.com/user/environment-variables. The report consists of one line per verification:
push
, pull_request
, api
, cron
.allowFail
argument of build_sketch
).publish_report_to_repository REPORT_GITHUB_TOKEN repositoryURL reportBranch reportFolder doLinkComment
Add the report to a repository. See the instructions for publishing job reports for details.
true
or false
Whether to comment on the GitHub thread of the commit that triggered the build with a link to the report.publish_report_to_gist REPORT_GITHUB_TOKEN REPORT_GIST_URL doLinkComment
Add the report to the report gist. See the instructions for publishing job reports for details.
true
or false
Whether to comment on the GitHub thread of the commit that triggered the build with a link to the report.The script offers the option of publishing the job result reports to a repository or GitHub gist by using the publish_report_to_repository
or publish_report_to_gist
functions. This makes it easier to view the reports or to import them into a spreadsheet program. You also have the option of having the link to the reports automatically added in a comment to the commit that triggered the build. This requires some configuration, which is described in the instructions below.
NOTE: For security reasons, reports for builds of pull requests from a fork of the repository can not be published. If the owner of that fork wants to publish reports they can create a GitHub token (and gist if using publish_report_to_gist
) and configure the Travis CI settings for their fork of the repository following these instructions.
This is required for either publishing option.
publish_report_to_gist
check gist.publish_report_to_repository
or setting the doLinkComment
argument of publish_report_to_gist
check public_repo (for public repositories only) or repo (for private and public repositories).REPORT_GITHUB_TOKEN
in the Name field and the token in the Value field. Make sure the Display value in build log switch is in the off position.An alternative to using a Travis CI hidden environment variable as described above is to define the GitHub personal access token as an encrypted environment variable: https://docs.travis-ci.com/user/environment-variables/#Encrypting-environment-variables.
This is required for use of the publish_report_to_gist
function.
REPORT_GIST_URL
in the Name field and the URL of the gist in the Value field. You can turn the Display value in build log switch to the on position. The gist URL is not secret and this will provide more information in the log.The Arduino IDE will usually try to start the GUI whenever there is an error in the command. Since the Travis CI build environment does not support this it will just hang for ten minutes until Travis CI automatically cancels the job. This means you get no useful information on the cause of the problem.
Verbose output results in a harder to read log so you should leave it off or minimized when possible but it can be useful for troubleshooting. Note that turning on verbose output for a large build may cause the log to exceed 4 MB, which causes Travis CI to terminate the job.
set_script_verbosity
documentation in the Usage section.set_verbose_output_during_compilation
documentation in the Usage section..travis.yml
file to get more details of the Travis CI build process.
- set -o verbose
- set -o xtrace
Some older versions of the Arduino IDE have bugs or limitations that may cause problems if used with this script:
--pref
), thus set_sketchbook_folder
can not be used if no newer IDE version has been installed.file that doesn't define a
core-dependencies` property. The file include is successful but compilation of sketches that use the library functions will fail.-
or .
are not allowed in sketch or library folder names. If any are present the Arduino IDE will hang indefinitely when it's executed.install_ide
. 1.6.2 is installed if it is explicitly specified in a version list.--install-boards
), thus install_package
can't be used if no newer IDE version has been installed.Pull requests or issue reports are welcome! Please see the contribution rules for instructions.