Machine learning: Neural network profiler

This code example demonstrates how to run through the ModusToolbox™ machine learning (ModusToolbox™-ML) development flow with PSoC™ 6 MCU, where the end user has a pre-trained neural network (NN) model, which can be profiled and validated at the PC and target device.

You can import a pre-trained model using the ModusToolbox™-ML configurator tool, create an embedded, optimized version of this model, and validate the performance on the PC. After this, the validation data files can be integrated with this code example or streamed to the device (default option), so you can run the same validation data and profile performance when the ML model is deployed on the PSoC™ 6 MCU.

Figure 1. ModusToolbox™-ML development flow

For details about the ModusToolbox™ machine learning solution, see this link.

View this README on GitHub.

Provide feedback on this code example.

Requirements

• ModusToolbox™ software v3.0 or later (tested with v3.0)
• ModusToolbox™ Machine Learning Pack v2.0 or later (tested with v2.0)
• PSoC™ 6 Board support package (BSP) minimum required version: 4.0.0
• Programming language: C and C++
• Associated parts: All PSoC™ 6 MCU parts

Supported toolchains (make variable 'TOOLCHAIN')

• GNU Arm® embedded compiler v10.3.1 (GCC_ARM) - Default value of TOOLCHAIN
• Arm® compiler v6.16 (ARM)
• IAR C/C++ compiler v9.30.1 (IAR)

Note: TensorFlow Lite for Microcontrollers (TFLM) does not support IAR compiler.

Supported kits (make variable 'TARGET')

• PSoC™ 6 Wi-Fi Bluetooth® prototyping kit (CY8CPROTO-062-4343W) - Default value of TARGET
• PSoC™ 6 Wi-Fi Bluetooth® pioneer kit (CY8CKIT-062-WIFI-BT)
• PSoC™ 62S2 Wi-Fi Bluetooth® pioneer kit (CY8CKIT-062S2-43012)
• PSoC™ 62S1 Wi-Fi Bluetooth® pioneer kit (CYW9P62S1-43438EVB-01)
• PSoC™ 62S1 Wi-Fi Bluetooth® pioneer kit (CYW9P62S1-43012EVB-01)
• PSoC™ 62S3 Wi-Fi Bluetooth® prototyping kit (CY8CPROTO-062S3-4343W)
• PSoC™ 64 "Secure Boot" Wi-Fi Bluetooth® pioneer kit (CY8CKIT-064B0S2-4343W)
• PSoC™ 62S2 evaluation kit (CY8CEVAL-062S2, CY8CEVAL-062S2-LAI-4373M2, CY8CEVAL-062S2-MUR-43439M2)

Hardware setup

This example uses the board's default configuration. See the kit user guide to ensure that the board is configured correctly.

Note: The PSoC™ 6 Bluetooth® LE pioneer kit (CY8CKIT-062-BLE) and the PSoC™ 6 Wi-Fi Bluetooth® pioneer kit (CY8CKIT-062-WIFI-BT) ship with KitProg2 installed. The ModusToolbox™ software requires KitProg3. Before using this code example, make sure that the board is upgraded to KitProg3. The tool and instructions are available in the Firmware Loader GitHub repository. If you do not upgrade, you will see an error like "unable to find CMSIS-DAP device" or "KitProg firmware is out of date".

Software setup

Install a terminal emulator if you do not have one. Instructions in this document use Tera Term.

Install the Machine Learning pack using this link. Alternatively, use the Infineon Developer Center (IDC) launcher and search for ModusToolbox Machine Learning Pack and install it.

Use the ModusToolbox™-ML configurator tool (from {ModusToolbox™ software install directory}/packs/ModusToolbox-Machine-Learning-Pack/tools/ml-configurator/) to load a pre-trained NN model and generate C files to be used with this code example. Alternatively, you can launch the ML configurator tool in Eclipse IDE for ModusToolbox™ from the Quick Launch window; or using the make ml-configurator command in the console. For more information, see the ModusToolbox™ machine learning user guide.

By default, the Makefile uses a model that comes with the code example. The pre-trained NN model is located in the pretrained_models folder. You can use the ModusToolbox™-ML configurator to link to this file, or load another model file and generate C files for the target device.

By default, the output files location is set to mtb_ml_gen. The project name is set to TEST_MODEL. If you change any of these default settings, edit the following Makefile parameters of this code example:

Note: The tflm and tflm_less inference engines only support float and int8x8.

To validate the given model with the ModusToolbox™-ML configurator tool, click the Validate in Desktop tab. By default, the project uses testing data stored in the file located at test_data/test_data.csv.

Alternatively, you can use a random dataset structure to validate the model. You can click the Validate button to initiate the validation. Note that this will generate the regression data files, which can be stored in the internal memory of the target device. Because of that, limit the sample count to 100 in the ModusToolbox™-ML configurator tool.

You can also stream the test data to the device by clicking the Validate on Target. In this scheme, the regression data files are not stored in the internal memory, so you can work with a larger dataset. It uses the UART to stream the data.

This code example sets the UART baudrate based on the operating system used to build the firmware. It sets to 115.2 Kbps for MacOS and 1 Mbps for Windows or Linux. ModusToolbox™-ML configurator tool also sets the baudrate based on the operating system. If the code example is built on a different operating system then the one running the tool, check if the baudrate matches. If not, rebuild the firmware by setting the appropriate value in the main.c when calling cy_retarget_io_init();

Using the code example

Create the project and open it using one of the following:

Operation

If using a PSoC™ 64 "Secure" MCU kit (like CY8CKIT-064B0S2-4343W), the PSoC™ 64 device must be provisioned with keys and policies before being programmed. Follow the instructions in the "Secure Boot" SDK user guide to provision the device. If the kit is already provisioned, copy-paste the keys and policy folder to the application folder.

1. Connect the board to your PC using the provided USB cable through the KitProg3 USB connector.

2. If using local regression data, in main.c, set the REGRESSION_DATA_SOURCE to USE_LOCAL_DATA. Then open a terminal program and select the KitProg3 COM port. Set the serial port parameters to 8N1 and 115200 baud.

3. Program the board using one of the following:

   Using Eclipse IDE for ModusToolbox™ software

   1. Select the application project in the Project Explorer. 2. In the **Quick Panel**, scroll down, and click **\ Program (KitProg3_MiniProg4)**.
   Using CLI

   From the terminal, execute the `make program` command to build and program the application using the default toolchain to the default target. The default toolchain and target are specified in the application's Makefile but you can override those values manually: ``` make program TARGET= TOOLCHAIN= ``` Example: ``` make program TARGET=CY8CPROTO-062-4343W TOOLCHAIN=GCC_ARM ```

4. After programming, the application starts automatically. If using regression local data, confirm that "Neural Network Profiler", model information, profiling data and accuracy results are printed on the UART terminal.

5. If using streaming regression data (default option), open the ModusToolbox™-ML configurator tool. Open the design.mtbml file and click Validate on Target. Select the KitProg COM port (ensure that no other software has the COM port open) and check the Quantization that matches the NN_TYPE set in the Makefile. Click Validate.

6. If using the ModusToolbox™-ML configurator tool, the results of the validation are printed on the tool's console. Log files for the profiling are stored in the following files (located at NN_MODEL_FOLDER/info_target/ folder):

   • device_log.txt: All messages exchanged between the configurator tool and the device.
   • profiler_frame_log.txt: Per-frame profile information
   • profiler_info_log.txt: Per-model profile information
   • profiler_output_log.txt: Per-frame output log (not currently supported, but output log can be viewed in the device_log.txt).

   You can choose the type of profiling/debugging data to be printed by setting the PROFILE_CONFIGURATION macro in main.c.

   Note: Depending on the chosen quantization, validation results might return a failure if the accuracy is not greater than or equal to 98%.

Debugging

You can debug the example to step through the code. In the IDE, use the \ Debug (KitProg3_MiniProg4) configuration in the Quick Panel. For details, see the "Program and debug" section in the Eclipse IDE for ModusToolbox™ software user guide.

Note: (Only while debugging) On the CM4 CPU, some code in main() may execute before the debugger halts at the beginning of main(). This means that some code executes twice – once before the debugger stops execution, and again after the debugger resets the program counter to the beginning of main(). See KBA231071 to learn about this and for the workaround.

Design and implementation

In this example, you must provide a pre-trained NN model with the weights, regression data, and the model parameters. The ModusToolbox™-ML configurator tool can generate such data based on the standard Keras H5 format or TFLite format. By default, these files are located in the ./mtb_ml_gen folder. The code examples also provide an ModusToolbox™-ML configurator tool project file - design.mtbml, which points to the pre-trained NN model available in the pretrained_models folder.

If you change the output file location in the ModusToolbox™-ML configurator tool, you must also reflect the change in the Makefile (the NN_MODEL_FOLDER parameter). The model data is stored as a header file or as a binary file (used for filesystem applications). This example uses header files. Depending the type of the NN model chosen in the Makefile (the NN_TYPE parameter), the application uses the files and variables from the following table, where (KEY is the output file prefix defined by the ModusToolbox™-ML configurator tool):

Table 1. Inference Engine: ifx

Table 2. Inference Engine: tflm

Table 3. Inference Engine: tflm_less

In the Makefile, set the NN_MODEL_NAME parameter based on the output file prefix chosen in the ModusToolbox™-ML configurator tool.

You can also change the type of inference engine to run by setting the NN_INFERENCE_ENGINE in the Makefile. We have three options: 1) ifx: Infineon inference engine. 2) tflm: TensorFlow Lite Micro inference engine with runtime interpreter 3) tflm_less: TensorFlow Lite Micro inference engine without interpreter (interpreter-less)

Note: If you are using TensorFlow Lite Micro, only int8x8 and float NN_TYPE are supported.

After updating the Makefile, all the model files are built into the application automatically, allowing the NN inference engine to be initialized and fed with the regression data.

This application has an option to choose the source of the regression data in the main.c file. You can set the REGRESSION_DATA_SOURCE to one of the following:

• USE_STREAM_DATA – Uses the ModusToolbox™-ML Configurator tool to stream the regression data
• USE_LOCAL_DATA – Uses the files located in mtb_ml_gen/mtb_ml_regression_data for the regression data

Note: Some devices from the supported kits might not have enough memory to run some of the configurations above, specially if using local regression data. If that occurs, pick another kit with larger memory device, or refer to the ML user guide on how to define the CY_ML_MODEL_MEM macro.

Note: When using a TFLM int8x8 model with local regression data, the output of the model is compared to quantized reference int8x8 model results. If using the streamed data, the output of the model is compared to a float reference model results. That means the accuracy results might differ between using local data and streamed data.

The application also has an option to choose what type of profiling/debugging data to print/stream. You can set the PROFILE_CONFIGURATION to one of the following:

If using the local regression data, the application automatically loads the regression data generated by the ML configurator tool. The regression data consists of a collection of inputs (X) and a collection of outputs (Y). Once the inference engine processes X, it outputs the result. Then, the firmware compares the result with the desired value, Y. If these match, the firmware contributes to the accuracy calculation.

If using the ModusToolbox™-ML configurator tool, the same regression data is streamed over the UART. The following figure shows the communication sequence diagram between the tool and the device.

Figure 2. Communication sequence diagram

Files and folders

|-- mtb_ml_gen/               # Contains the model and regression files
|-- pretrained_models/        # Contains the Keras-H5 and TFlite models (used by the ML configurator tool)
|-- test_data/                # Contains a CSV file with the test data
|-- source                    # Contains the source code files for this example
   |- elapsed_timer.c/h       # Implements a system tick timer
   |- ml_local_regression.c/h # Implements a local regression flow
|-- design.mtbml              # ModusToolbox™-ML configurator tool project file

Resources and settings

Table 4. Application resources

Related resources

Other resources

Infineon provides a wealth of data at www.infineon.com to help you select the right device, and quickly and effectively integrate it into your design.

For PSoC™ 6 MCU devices, see How to design with PSoC™ 6 MCU - KBA223067 in the Infineon community.

Document history

Document title: CE232551 - Machine learning: Neural network profiler

© Cypress Semiconductor Corporation, 2020-2022. This document is the property of Cypress Semiconductor Corporation, an Infineon Technologies company, and its affiliates ("Cypress"). This document, including any software or firmware included or referenced in this document ("Software"), is owned by Cypress under the intellectual property laws and treaties of the United States and other countries worldwide. Cypress reserves all rights under such laws and treaties and does not, except as specifically stated in this paragraph, grant any license under its patents, copyrights, trademarks, or other intellectual property rights. If the Software is not accompanied by a license agreement and you do not otherwise have a written agreement with Cypress governing the use of the Software, then Cypress hereby grants you a personal, non-exclusive, nontransferable license (without the right to sublicense) (1) under its copyright rights in the Software (a) for Software provided in source code form, to modify and reproduce the Software solely for use with Cypress hardware products, only internally within your organization, and (b) to distribute the Software in binary code form externally to end users (either directly or indirectly through resellers and distributors), solely for use on Cypress hardware product units, and (2) under those claims of Cypress’s patents that are infringed by the Software (as provided by Cypress, unmodified) to make, use, distribute, and import the Software solely for use with Cypress hardware products. Any other use, reproduction, modification, translation, or compilation of the Software is prohibited.
TO THE EXTENT PERMITTED BY APPLICABLE LAW, CYPRESS MAKES NO WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, WITH REGARD TO THIS DOCUMENT OR ANY SOFTWARE OR ACCOMPANYING HARDWARE, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. No computing device can be absolutely secure. Therefore, despite security measures implemented in Cypress hardware or software products, Cypress shall have no liability arising out of any security breach, such as unauthorized access to or use of a Cypress product. CYPRESS DOES NOT REPRESENT, WARRANT, OR GUARANTEE THAT CYPRESS PRODUCTS, OR SYSTEMS CREATED USING CYPRESS PRODUCTS, WILL BE FREE FROM CORRUPTION, ATTACK, VIRUSES, INTERFERENCE, HACKING, DATA LOSS OR THEFT, OR OTHER SECURITY INTRUSION (collectively, "Security Breach"). Cypress disclaims any liability relating to any Security Breach, and you shall and hereby do release Cypress from any claim, damage, or other liability arising from any Security Breach. In addition, the products described in these materials may contain design defects or errors known as errata which may cause the product to deviate from published specifications. To the extent permitted by applicable law, Cypress reserves the right to make changes to this document without further notice. Cypress does not assume any liability arising out of the application or use of any product or circuit described in this document. Any information provided in this document, including any sample design information or programming code, is provided only for reference purposes. It is the responsibility of the user of this document to properly design, program, and test the functionality and safety of any application made of this information and any resulting product. "High-Risk Device" means any device or system whose failure could cause personal injury, death, or property damage. Examples of High-Risk Devices are weapons, nuclear installations, surgical implants, and other medical devices. "Critical Component" means any component of a High-Risk Device whose failure to perform can be reasonably expected to cause, directly or indirectly, the failure of the High-Risk Device, or to affect its safety or effectiveness. Cypress is not liable, in whole or in part, and you shall and hereby do release Cypress from any claim, damage, or other liability arising from any use of a Cypress product as a Critical Component in a High-Risk Device. You shall indemnify and hold Cypress, including its affiliates, and its directors, officers, employees, agents, distributors, and assigns harmless from and against all claims, costs, damages, and expenses, arising out of any claim, including claims for product liability, personal injury or death, or property damage arising from any use of a Cypress product as a Critical Component in a High-Risk Device. Cypress products are not intended or authorized for use as a Critical Component in any High-Risk Device except to the limited extent that (i) Cypress’s published data sheet for the product explicitly states Cypress has qualified the product for use in a specific High-Risk Device, or (ii) Cypress has given you advance written authorization to use the product as a Critical Component in the specific High-Risk Device and you have signed a separate indemnification agreement.
Cypress, the Cypress logo, and combinations thereof, WICED, ModusToolbox, PSoC, CapSense, EZ-USB, F-RAM, and Traveo are trademarks or registered trademarks of Cypress or a subsidiary of Cypress in the United States or in other countries. For a more complete list of Cypress trademarks, visit cypress.com. Other names and brands may be claimed as property of their respective owners.