Testrun automates specific test cases to verify network and security functionality in IoT devices. It is an open source tool which allows manufacturers of IP capable devices to test their devices for the purposes of Device Qualification within the BOS program.
Without tools like Testrun, test labs and engineers may need to maintain a large and complex network coupled with dynamic configuration files and constant software updates. The major issues which can and should be solved are: 1) The complexity of managing a testing network 2) The time required to perform testing of network functionality 3) The accuracy and consistency of testing network functionality
Testrun creates an isolated and controlled network environment on a linux machine. This removes the necessity for complex hardware, advanced knowledge and networking experience whilst enabling test engineers to validate device behaviour against Google’s Building Operating System requirements.
Two modes are supported by Testrun:
Once you have met the hardware and software requirements, you can get started with Testrun by following the Get started guide. Further docs are available in the docs directory
Testrun will constantly evolve to further support end-users by automating device network behaviour against industry standards. For further information on upcoming features, check out the Roadmap.
We are proud of our tool and strive to provide an enjoyable experience for all of our users. Testrun goes through rigorous accessibility testing at each release. You can read more about Google and Accessibility here. You are welcome to submit a new issue and provide feedback on our implementations. To find out how Testrun implements accessibility features, you can view a short video here.
If the application has come across a problem at any point during setup or use, please raise an issue under the issues tab. Issue templates exist for both bug reports and feature requests. If neither of these are appropriate for your issue, raise a blank issue instead.
The contributing requirements can be found in CONTRIBUTING.md. In short, checkout the Google CLA site to get started. After that, check out our developer documentation.
1) I have an issue whilst installing/upgrading Testrun, what do I do?
Sometimes, issues may arise when installing or upgrading Testrun - this may happen due to one of many reasons due to the nature of the application. However, most of the time, it can be resolved by following a full Testrun re-install by using these commands:
sudo docker system prune -a
sudo apt install ./testrun-*.deb
2) What device networking functionality is validated by Testrun?
Best practices and requirements for IoT devices are constantly changing due to technological advances and discovery of vulnerabilities. The current expectations for IoT devices on Google deployments can be found in the Application Security Requirements for IoT Devices. Testrun aims to automate as much of the Application Security Requirements as possible.
3) What services are provided on the virtual network?
The following are network services that are containerized and accessible to the device under test though are likely to change over time:
4) Can I run Testrun on a virtual machine?
Testrun can be virtualized if the 2x ethernet adapters are passed through to a VirtualBox VM as a USB device rather than managed network adapters. You can view the guide to working on a virtual machine here.