Izuma Client E2E tests verify that a target platform can perform essential Device Management Client operations. The library is designed to be used with the pytest test framework.
Python 3.8 or later.
$ git clone https://github.com/PelionIoT/client-e2e-python-test-library.git
$ pip install wheel
$ python3 setup.py bdist_wheel
$ cd dist/
$ pip install -I client_test_lib*.whl
Build the Device Management Client example application for your board and flash it.
Set the CLOUD_API_KEY
environment variable with your access key.
export CLOUD_API_KEY=<access_key_here>
set CLOUD_API_KEY=<access_key_here>
Default API address is https://api.us-east-1.mbedcloud.com
. You can change this by defining CLOUD_API_GW
environment variable in similar way as CLOUD_API_KEY
is done above.
Test run will create temporary API key for the WebSocket callback channel by default. If you want to prevent that and use only the exported API key, add --use_one_apikey
startup argument.
Tests use Mbed LS to select the board from the serial port.
mbedls
to check the target board's ID, and use it in the test run's argument --target_id=[id]
.$ mbedls
+---------------+----------------------+-------------+--------------+--------------------------------------------------+-----------------+
| platform_name | platform_name_unique | mount_point | serial_port | target_id | daplink_version |
+---------------+----------------------+-------------+--------------+--------------------------------------------------+-----------------+
| K64F | K64F[0] | D: | /dev/ttyACM0 | 0240000032044e4500257009997b00386781000097969900 | 0244 |
+---------------+----------------------+-------------+--------------+--------------------------------------------------+-----------------+
You can also run tests with the Linux build of the Device Management Client by passing the compiled binary in the --local_binary=./mbedCloudClientExample.elf
argument.
To run a test set for Device Management Client use the command:
pytest tests/dev-client-tests.py
Read later instructions how to setup the update test to go with the test set.
To run a single test from the set, use the -k
argument to set the test name as a keyword:
pytest tests/dev-client-tests.py -k get_resource
Before running the update test, make sure you create update-related configuration and initialize the developer environment properly, as describe the Device Management Client example tutorial.
The update test uses the manifest-dev-tool
utility to create the actual manifest for update campaigns.
To run the update test, provide the mandatory update image path and optional manifest-dev-tool init path arguments at startup:
--update_bin=/home/user/mbed-cloud-client-example_update.bin
absolute path for the update image--manifest_tool=/home/user/mbed-os-example-pelion
absolute path where manifest-dev-tool init is executed - defaults to the current working directory.To leave the firmware image, manifest and campaign in your account after the test, add the --no_cleanup
startup argument.
Manifest tool 2.0.0 supports two manifest schema versions: v1
and v3
. By default, the update test creates v3
manifests, but you can create v1
manifests by passing the --manifest_version=v1
startup argument.
Add the startup arguments to adjust the generated output:
--log-cli-level=DEBUG
adds more details--html=results.html
generates an HTML report--junitxml=junit.xml
provides output for CI systems, for example JenkinsThe library also writes a separate client.log
file from the Device Management Client output.
There are many ways to configure the test runs. Refer to the full pytest documentation for more information.
Test name | Main functions | Notes |
---|---|---|
test_01_get_device_id |
Verify that the device is registered. | |
test_02_get_resource |
Verify that the device responds to GET. | Uses Resource /1/0/1 |
test_03_put_resource |
Verify that the device responds to PUT. | Uses Resource /1/0/1 |
test_04_subscribe_resource |
Verify the notification from the subscribed resource. | Uses Resource /1/0/1 |
test_05_factory_reset |
Verify the client's factory reset behaviour. | |
test_06_update_device |
Verify the device firmware update. |
See the license agreement.