Issue with QEMU Unsupported Machine Type for ESP32S3 in CI/CD Workflow

hayschan commented 10 months ago

Hello Hanxi Fu,

I'm encountering an issue with QEMU in the GitHub Actions CI/CD workflow when using the hfudev/esp-idf-build-and-test-ci-template for testing ESP32S3. The problem arises during the "Run Test App on Target (QEMU)" stage, where QEMU fails to recognize the esp32s3 machine type.

When I idf.py build and pytest -s --embedded-service esp,idf the test_app, the unit testing went successfully.

pytest_aip1629.py:

import pytest
from pytest_embedded import Dut

@pytest.mark.supported_targets
@pytest.mark.generic
def test_aip1629(dut: Dut)-> None:
    dut.run_all_single_board_cases()

Error Details:

Error Message: qemu-system-xtensa: unsupported machine type and pexpect.exceptions.TIMEOUT.
Context: This error occurs while trying to execute the test script components/aip1629/test_apps/pytest_aip1629.py which relies on QEMU to emulate the ESP32S3 environment.

Request for Assistance

I would greatly appreciate your expertise and insights on the following:

Root Cause Analysis: Any insights into the root cause of why QEMU is not recognizing the esp32s3 machine type in this particular setup.
Solution Guidance: Recommendations or steps to resolve this issue to successfully emulate ESP32S3 in the GitHub Actions CI/CD workflow.

I am attaching the relevant section of my GitHub Actions workflow configuration and the error logs for your reference.

GitHub Actions workflow configuration

name: Build and Test Application

on:
  pull_request:
    paths:
      - "src/**/*.c"
  push:
    branches:
      - main
  workflow_dispatch: # Can enable manually

env:
  IDF_PATH: /opt/esp/idf
  test_dirs: components/aip1629

defaults:
  run:
    shell: bash

jobs:
  build:
    name: Build Test App
    strategy:
      fail-fast: false
      matrix:
        # choose the version of ESP-IDF to use for the build
        idf-branch:
          - release-v5.0
          - release-v5.1
          - latest
        # choose the target to build for
        target:
          - esp32s3
          # - esp32c3
    runs-on: ubuntu-22.04
    container:
      image: espressif/idf:${{ matrix.idf-branch }}
    steps:
      - uses: actions/checkout@v3
      - name: Install Python Dependencies
        run: |
          . $IDF_PATH/export.sh
          python -m pip install idf-build-apps
      - name: Build Test Application with ESP-IDF
        run: |
          . $IDF_PATH/export.sh
          idf-build-apps build \
            -p ${{ env.test_dirs }} \
            --target ${{ matrix.target }} \
            --recursive \
            --build-dir build_${{ matrix.target }}_${{ matrix.idf-branch }}
      - name: Upload files to artifacts for run-target job
        uses: actions/upload-artifact@v3
        with:
          name: built_binaries_${{ matrix.target }}_${{ matrix.idf-branch }}
          path: |
            **/build**/bootloader/bootloader.bin
            **/build**/partition_table/partition-table.bin
            **/build**/*.bin
            **/build**/*.elf
            **/build**/flasher_args.json
          if-no-files-found: error
  target-test:
    name: Run Test App on Target (QEMU)
    needs: build
    strategy:
      fail-fast: false
      matrix:
        # choose the version of ESP-IDF to use for the build
        idf-branch:
          - release-v5.0
          - release-v5.1
          - latest
        # choose the target to build for
        target:
          - esp32s3
          # - esp32c3
    runs-on: ubuntu-22.04
    container:
      image: hfudev/qemu:main
    steps:
      - uses: actions/checkout@v3
      - uses: actions/download-artifact@v2
        with:
          name: built_binaries_${{ matrix.target }}_${{ matrix.idf-branch }}
      - name: Install Python packages
        run: |
          . $IDF_PATH/export.sh
          pip install \
            pytest-embedded-idf \
            pytest-embedded-qemu
      - name: Run Test App on target
        run: |
          . $IDF_PATH/export.sh
          pytest ${{ env.test_dirs }} \
            --target ${{ matrix.target }} \
            --embedded-services idf,qemu \
            --junit-xml test_${{ matrix.target }}_${{ matrix.idf-branch }}.xml \
            --build-dir build_${{ matrix.target }}_${{ matrix.idf-branch }}
      - uses: actions/upload-artifact@v2
        if: always()
        with:
          name: test_${{ matrix.target }}_${{ matrix.idf-branch }}_junit
          path: test_${{ matrix.target }}_${{ matrix.idf-branch }}.xml

Error log

Detecting the Python interpreter
Checking "python3" ...
Python 3.8.10
"python3" has been detected
Checking Python compatibility
Checking other ESP-IDF version.
Adding ESP-IDF tools to PATH...
Checking if Python packages are up to date...
Python requirements are satisfied.
Requirement files:
 - /opt/esp/idf/tools/requirements/requirements.core.txt
Python being checked: /opt/esp/python_env/idf5.2_py3.8_env/bin/python
Added the following directories to PATH:
  /opt/esp/idf/components/espcoredump
  /opt/esp/idf/components/partition_table
  /opt/esp/idf/components/app_update
  /opt/esp/tools/xtensa-esp-elf-gdb/12.1_20221002/xtensa-esp-elf-gdb/bin
  /opt/esp/tools/riscv32-esp-elf-gdb/12.1_20221002/riscv32-esp-elf-gdb/bin
  /opt/esp/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin
  /opt/esp/tools/xtensa-esp32s2-elf/esp-12.2.0_20230208/xtensa-esp32s2-elf/bin
  /opt/esp/tools/xtensa-esp32s3-elf/esp-12.2.0_20230208/xtensa-esp32s3-elf/bin
  /opt/esp/tools/riscv32-esp-elf/esp-12.2.0_20230208/riscv32-esp-elf/bin
  /opt/esp/tools/esp32ulp-elf/2.35_20220830/esp32ulp-elf/bin
  /opt/esp/tools/cmake/3.24.0/bin
  /opt/esp/tools/openocd-esp32/v0.12.0-esp32-20230419/openocd-esp32/bin
  /opt/esp/python_env/idf5.2_py3.8_env/bin
  /opt/esp/idf/tools
Done! You can now compile ESP-IDF projects.
Go to the project directory and run:

  idf.py build

============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.4.4, pluggy-1.3.0 -- /opt/esp/python_env/idf5.2_py3.8_env/bin/python
cachedir: .pytest_cache
rootdir: /__w/esp-project-component-test/esp-project-component-test
configfile: pytest.ini
plugins: embedded-1.6.1
collecting ... collected 1 item

components/aip1629/test_apps/pytest_aip1629.py::test_aip1629 
-------------------------------- live log setup --------------------------------
2024-01-12 14:37:12 esptool.py v4.6.2
2024-01-12 14:37:12 Wrote 0x200000 bytes to file /__w/esp-project-component-test/esp-project-component-test/components/aip1629/test_apps/build_esp32s3_release-v5.0/flash_image.bin, ready to flash to offset 0x0
2024-01-12 14:37:12 
2024-01-12 14:37:12 INFO Executing qemu-system-xtensa -nographic -machine esp32s3 -qmp tcp:localhost:4489,server,wait=off -drive file=/__w/esp-project-component-test/esp-project-component-test/components/aip1629/test_apps/build_esp32s3_release-v5.0/flash_image.bin,if=mtd,format=raw
2024-01-12 14:37:12 qemu-system-xtensa: unsupported machine type
2024-01-12 14:37:12 Use -machine help to list supported machines
FAILED---------------------------- live log sessionfinish ----------------------------
2024-01-12 14:37:42 INFO Merged junit report dumped to /__w/esp-project-component-test/esp-project-component-test/test_esp32s3_release-v5.0.xml

=================================== FAILURES ===================================
_________________________________ test_aip1629 _________________________________
/opt/esp/python_env/idf5.2_py3.8_env/lib/python3.8/site-packages/pytest_embedded/dut.py:75: in wrapper
    index = func(self, pattern, *args, **kwargs)  # noqa
/opt/esp/python_env/idf5.2_py3.8_env/lib/python3.8/site-packages/pytest_embedded/dut.py:141: in expect_exact
    return self.pexpect_proc.expect_exact(pattern, **kwargs)
/opt/esp/python_env/idf5.2_py3.8_env/lib/python3.8/site-packages/pexpect/spawnbase.py:432: in expect_exact
    return exp.expect_loop(timeout)
/opt/esp/python_env/idf5.2_py3.8_env/lib/python3.8/site-packages/pexpect/expect.py:181: in expect_loop
    return self.timeout(e)
/opt/esp/python_env/idf5.2_py3.8_env/lib/python3.8/site-packages/pexpect/expect.py:144: in timeout
    raise exc
E   pexpect.exceptions.TIMEOUT: <pytest_embedded.log.PexpectProcess object at 0x7f4dd2aa00a0>
E   searcher: searcher_string:
E       0: b'Press ENTER to see the list of tests'
E   <pytest_embedded.log.PexpectProcess object at 0x7f4dd2aa00a0>
E   searcher: searcher_string:
E       0: b'Press ENTER to see the list of tests'

The above exception was the direct cause of the following exception:
components/aip1629/test_apps/pytest_aip1629.py:21: in test_aip1629
    dut.run_all_single_board_cases()
/opt/esp/python_env/idf5.2_py3.8_env/lib/python3.8/site-packages/pytest_embedded_idf/unity_tester.py:428: in run_all_single_board_cases
    for case in self.test_menu:
/opt/esp/python_env/idf5.2_py3.8_env/lib/python3.8/site-packages/pytest_embedded_idf/unity_tester.py:243: in test_menu
    self._test_menu = self._parse_test_menu()
/opt/esp/python_env/idf5.2_py3.8_env/lib/python3.8/site-packages/pytest_embedded_idf/unity_tester.py:124: in _parse_test_menu
    self.expect_exact(ready_line)
/opt/esp/python_env/idf5.2_py3.8_env/lib/python3.8/site-packages/pytest_embedded/dut.py:82: in wrapper
    raise e.__class__(debug_str) from e
E   pexpect.exceptions.TIMEOUT: Not found "Press ENTER to see the list of tests"
E   Bytes in current buffer (color code eliminated): ine help to list supported machines
E   Please check the full log here: /tmp/pytest-embedded/2024-01-12_14-37-11-917788/test_aip1629/dut.txt
=========================== short test summary info ============================
FAILED components/aip1629/test_apps/pytest_aip1629.py::test_aip1629 - pexpect.exceptions.TIMEOUT: Not found "Press ENTER to see the list of tests"
Bytes in current buffer (color code eliminated): ine help to list supported machines
Please check the full log here: /tmp/pytest-embedded/2024-01-12_14-37-11-917788/test_aip1629/dut.txt
============================== 1 failed in 30.25s ==============================

Thank you in advance for your assistance and time. Your guidance will be invaluable in helping resolve this critical issue.

hayschan commented 10 months ago

Later, I found out that ESP32-S3 is probably not supported by Espressif QEMU. It only lists the support of ESP32 and ESP32-C3 here.

I changed the target (at both locations, e.g. Build Test App and Run Test App on Target (QEMU)) back to esp32 and esp32c3:

    target:
-             - esp32s3
+             - esp32
+             - esp32c3

Weirdly, I failed on the first stage this time - Build Test App.

Run actions/upload-artifact@v3
/usr/bin/docker exec  388c95248a36e714275eb2474c40e3faf2af50f11003940093499ecad87ac991 sh -c "cat /etc/*release | grep ^ID"
Error: No files were found with the provided path: **/build**/bootloader/bootloader.bin
**/build**/partition_table/partition-table.bin
**/build**/*.bin
**/build**/*.elf
**/build**/flasher_args.json. No artifacts will be uploaded.

hfudev commented 10 months ago

Hi @hayschan could you provide more logs? for example the build logs? If the build succeeded, these files should exist.

hfudev / esp-idf-build-and-test-ci-template