Although the action's Python code itself is currently well tested via unit tests, this does not provide any checks for
how the action interacts with the GitHub Actions framework and GitHub API.
Even though it won't provide full verification of the report, simply running the action provides a
"smoke test" as well as facilitating manual verification.
On every pull request event, the action will run and source the sketches reports test data from a local path.
When the job is triggered by an event from a fork, it will fail due to the access token not having the write permissions
required to comment on the PR thread. The workflow is triggered to still pass under these conditions. Checking the log to
see whether the action failed in the expected manner may still provide a useful validation of the PR.
When the PR is from a branch of the repository, the deltas report will be commented on the PR thread.
The workflow is also configured to run on push, schedule, workflow_dispatch, and repository_dispatch events. In this case,
it sources the sketches report from a workflow artifact. A dummy PR (https://github.com/arduino/report-size-deltas/pull/14) will be left permanently open in the repository with
the sole purpose of serving as a target for these tests.
Click for full report table
Board|examples/MIDIUSB_clock flash|%|examples/MIDIUSB_clock RAM for global variables|%|examples/MIDIUSB_loop flash|%|examples/MIDIUSB_loop RAM for global variables|%|examples/MIDIUSB_read flash|%|examples/MIDIUSB_read RAM for global variables|%|examples/MIDIUSB_write flash|%|examples/MIDIUSB_write RAM for global variables|%
-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-
arduino:avr:leonardo|0|0.0|0|0.0|0|0.0|0|0.0|0|0.0|0|0.0|-40|-0.14|-16|-0.62
arduino:sam:arduino_due_x|0|0.0|N/A|N/A|0|0.0|N/A|N/A|0|0.0|N/A|N/A|0|0.0|N/A|N/A
arduino:samd:mkrzero|84|0.03|0|0.0|0|0.0|0|0.0|-28|-0.01|0|0.0|0|0.0|32|0.1
Click for full report CSV
```
Board,examples/MIDIUSB_clock flash,%,examples/MIDIUSB_clock RAM for global variables,%,examples/MIDIUSB_loop flash,%,examples/MIDIUSB_loop RAM for global variables,%,examples/MIDIUSB_read flash,%,examples/MIDIUSB_read RAM for global variables,%,examples/MIDIUSB_write flash,%,examples/MIDIUSB_write RAM for global variables,%
arduino:avr:leonardo,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,-40,-0.14,-16,-0.62
arduino:sam:arduino_due_x,0,0.0,N/A,N/A,0,0.0,N/A,N/A,0,0.0,N/A,N/A,0,0.0,N/A,N/A
arduino:samd:mkrzero,84,0.03,0,0.0,0,0.0,0,0.0,-28,-0.01,0,0.0,0,0.0,32,0.1
```
Although the action's Python code itself is currently well tested via unit tests, this does not provide any checks for how the action interacts with the GitHub Actions framework and GitHub API.
Even though it won't provide full verification of the report, simply running the action provides a "smoke test" as well as facilitating manual verification.
On every pull request event, the action will run and source the sketches reports test data from a local path.
When the job is triggered by an event from a fork, it will fail due to the access token not having the write permissions required to comment on the PR thread. The workflow is triggered to still pass under these conditions. Checking the log to see whether the action failed in the expected manner may still provide a useful validation of the PR.
When the PR is from a branch of the repository, the deltas report will be commented on the PR thread.
The workflow is also configured to run on
push,schedule,workflow_dispatch, andrepository_dispatchevents. In this case, it sources the sketches report from a workflow artifact. A dummy PR (https://github.com/arduino/report-size-deltas/pull/14) will be left permanently open in the repository with the sole purpose of serving as a target for these tests."Golden" sketches reports for integration tests were generated from the repository attached here: https://github.com/arduino/report-size-deltas/pull/8#issue-493741941
Expected size deltas report resulting from the integration test workflow:
Memory usage change @ 651f05f4d4aca30ac359e972c01568f873112d43
Click for full report table
Board|examples/MIDIUSB_clockflash|%|examples/MIDIUSB_clock
RAM for global variables|%|examples/MIDIUSB_loop
flash|%|examples/MIDIUSB_loop
RAM for global variables|%|examples/MIDIUSB_read
flash|%|examples/MIDIUSB_read
RAM for global variables|%|examples/MIDIUSB_write
flash|%|examples/MIDIUSB_write
RAM for global variables|% -|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|- arduino:avr:leonardo|0|0.0|0|0.0|0|0.0|0|0.0|0|0.0|0|0.0|-40|-0.14|-16|-0.62 arduino:sam:arduino_due_x|0|0.0|N/A|N/A|0|0.0|N/A|N/A|0|0.0|N/A|N/A|0|0.0|N/A|N/A arduino:samd:mkrzero|84|0.03|0|0.0|0|0.0|0|0.0|-28|-0.01|0|0.0|0|0.0|32|0.1
Click for full report CSV
``` Board,examples/MIDIUSB_clockflash,%,examples/MIDIUSB_clock
RAM for global variables,%,examples/MIDIUSB_loop
flash,%,examples/MIDIUSB_loop
RAM for global variables,%,examples/MIDIUSB_read
flash,%,examples/MIDIUSB_read
RAM for global variables,%,examples/MIDIUSB_write
flash,%,examples/MIDIUSB_write
RAM for global variables,% arduino:avr:leonardo,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,-40,-0.14,-16,-0.62 arduino:sam:arduino_due_x,0,0.0,N/A,N/A,0,0.0,N/A,N/A,0,0.0,N/A,N/A,0,0.0,N/A,N/A arduino:samd:mkrzero,84,0.03,0,0.0,0,0.0,0,0.0,-28,-0.01,0,0.0,0,0.0,32,0.1 ```