search

golioth / golioth-firmware-sdk

Firmware SDK enabling any IoT device to connect to Golioth - the Universal Connector for IoT
https://golioth.io
Apache License 2.0
53 stars 11 forks source link

HIL: adjust OTA test timing #483

Closed szczys closed 1 month ago

szczys commented 1 month ago

Adjust OTA test timing to reduce tests that fail due to propagation delay.

https://github.com/golioth/firmware-issue-tracker/issues/568

github-actions[bot] commented 1 month ago

Visit the preview URL for this PR (updated for commit 4f1e82f):

https://golioth-firmware-sdk-doxygen-dev--pr483-szczys-ota-hil-7xfm26tj.web.app

(expires Tue, 14 May 2024 16:46:10 GMT)

🔥 via Firebase Hosting GitHub Action 🌎

Sign: a9993e61697a3983f3479e468bcb0b616f9a0578

github-actions[bot] commented 1 month ago

Code Coverage

Type Coverage
lines 55.2% (1037 of 1878 lines)
functions 57.7% (94 of 163 functions)
sam-golioth commented 1 month ago

Have you seen tests fail at the margins of these timeouts? Or is this more of a guess and check?

szczys commented 1 month ago

More of a guess and check. I have not been able to observe a status update sent to the server fail to update when I check manually so I am guessing that the 1.5s timeout pytest uses from seeing the log message to running the API call is the culprit. I doubled that to 3 seconds and built in some space for that in the C code timing.

I'd prefer not to bank on these timings, but the alternative that comes to mind is introducing an RPC to tell the device to go to the next status. That doesn't seem like a good idea.