espressif / esptool

Espressif SoC serial bootloader utility
https://docs.espressif.com/projects/esptool
GNU General Public License v2.0
5.48k stars 1.36k forks source link

Adopt `pytest` (ESPTOOL-462) #747

Closed kloczek closed 1 year ago

kloczek commented 2 years ago

Operating System

Linux/x86_64

Version

latest master

Python Version

3.8.13

Full Esptool Command Line that Was Run

N/A

Esptool Output

N/A

What is the Expected Behaviour?

pytest should pass without errors.

More Information

I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

Here is pytest output:

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/esptool-4.1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/esptool-4.1-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra =========================================================================== test session starts ============================================================================ platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0 rootdir: /home/tkloczko/rpmbuild/BUILD/esptool-4.1 collected 184 items / 4 errors ================================================================================== ERRORS ================================================================================== ______________________________________________________ ERROR collecting test/efuse_scripts/esp32/test_efuse_script.py ______________________________________________________ /home/tkloczko/rpmbuild/BUILD/esptool-4.1/test/efuse_scripts/esp32/test_efuse_script.py:3: in espefuse(esp, efuses, args, "burn_efuse JTAG_DISABLE 1 DISABLE_SDIO_HOST 1 CONSOLE_DEBUG_DISABLE 1") E NameError: name 'espefuse' is not defined _____________________________________________________ ERROR collecting test/efuse_scripts/esp32/test_efuse_script2.py ______________________________________________________ /home/tkloczko/rpmbuild/BUILD/esptool-4.1/test/efuse_scripts/esp32/test_efuse_script2.py:3: in espefuse(esp, efuses, args, "burn_efuse JTAG_DISABLE 1 DISABLE_SDIO_HOST 1 CONSOLE_DEBUG_DISABLE 1") E NameError: name 'espefuse' is not defined _____________________________________________________ ERROR collecting test/efuse_scripts/esp32xx/test_efuse_script.py _____________________________________________________ /home/tkloczko/rpmbuild/BUILD/esptool-4.1/test/efuse_scripts/esp32xx/test_efuse_script.py:3: in espefuse(esp, efuses, args, 'burn_efuse DIS_FORCE_DOWNLOAD 1 DIS_CAN 1 DIS_DOWNLOAD_MODE 1') E NameError: name 'espefuse' is not defined ____________________________________________________ ERROR collecting test/efuse_scripts/esp32xx/test_efuse_script2.py _____________________________________________________ /home/tkloczko/rpmbuild/BUILD/esptool-4.1/test/efuse_scripts/esp32xx/test_efuse_script2.py:3: in espefuse(esp, efuses, args, 'burn_efuse DIS_FORCE_DOWNLOAD 1 DIS_CAN 1 DIS_DOWNLOAD_MODE 1') E NameError: name 'espefuse' is not defined ========================================================================= short test summary info ========================================================================== ERROR test/efuse_scripts/esp32/test_efuse_script.py - NameError: name 'espefuse' is not defined ERROR test/efuse_scripts/esp32/test_efuse_script2.py - NameError: name 'espefuse' is not defined ERROR test/efuse_scripts/esp32xx/test_efuse_script.py - NameError: name 'espefuse' is not defined ERROR test/efuse_scripts/esp32xx/test_efuse_script2.py - NameError: name 'espefuse' is not defined !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 4 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ============================================================================ 4 errors in 0.34s ============================================================================= ```

Other Steps to Reproduce

No response

kloczek commented 2 years ago

And after add those four files to --ignore list

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/esptool-4.1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/esptool-4.1-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra --ignore test/efuse_scripts/esp32/test_efuse_script2.py --ignore test/efuse_scripts/esp32/test_efuse_script.py --ignore test/efuse_scripts/esp32xx/test_efuse_script2.py --ignore test/efuse_scripts/esp32xx/test_efuse_script.py =========================================================================== test session starts ============================================================================ platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0 rootdir: /home/tkloczko/rpmbuild/BUILD/esptool-4.1 collected 184 items test/test_espefuse.py FFFFFF [ 3%] test/test_espefuse_host.py FFFFFFFFsFFsFF.FFFsFFFFFssFsssssssssFFsssssssssssssssssssssssFFFFssFFF [ 41%] test/test_espsecure.py FFFFE..FFFFFFFFFFFFFFF.. [ 53%] test/test_esptool.py sssFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFsFFFFFFFFFsFFFFFsFFsFFFFFFFF [ 87%] test/test_imagegen.py .............. [ 95%] test/test_merge_bin.py ........ [ 99%] test/test_modules.py . [100%] ================================================================================== ERRORS ================================================================================== ____________________________________________________ ERROR at teardown of SigningTests.test_generate_and_extract_key_v2 ____________________________________________________ self = , test_case = , isTest = False @contextlib.contextmanager def testPartExecutor(self, test_case, isTest=False): old_success = self.success self.success = True try: > yield /usr/lib64/python3.8/unittest/case.py:60: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python3.8/unittest/case.py:717: in doCleanups self._callCleanup(function, *args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , function = , args = ('/tmp/tmpqb8asebj',), kwargs = {} def _callCleanup(self, function, /, *args, **kwargs): > function(*args, **kwargs) E OSError: [Errno 39] Directory not empty: '/tmp/tmpqb8asebj' /usr/lib64/python3.8/unittest/case.py:639: OSError --------------------------------------------------------------------------- Captured stdout call --------------------------------------------------------------------------- RSA 3072 private key in PEM format written to /tmp/tmpqb8asebj/key.pem /tmp/tmpqb8asebj/key.pem public key extracted to /tmp/tmpwv2w4604 ECDSA NIST192p private key in PEM format written to /tmp/tmpqb8asebj/key.pem ================================================================================= FAILURES ================================================================================= ________________________________________________________________ TestBurnKey.test_burn_key_34_coding_scheme ________________________________________________________________ self = def setUp(self): # reset and zero efuses > serialport.dtr = False E AttributeError: 'NoneType' object has no attribute 'dtr' /home/tkloczko/rpmbuild/BUILD/esptool-4.1/test/test_espefuse.py:46: AttributeError ________________________________________________________________ TestBurnKey.test_burn_key_no_coding_scheme ________________________________________________________________ self = def setUp(self): # reset and zero efuses > serialport.dtr = False E AttributeError: 'NoneType' object has no attribute 'dtr' /home/tkloczko/rpmbuild/BUILD/esptool-4.1/test/test_espefuse.py:46: AttributeError _____________________________________________________________ TestBurnBlockData.test_burn_block_data_34_coding _____________________________________________________________ [..] --------------------------------------------------------------------------- Captured stdout call --------------------------------------------------------------------------- ************************************************** Serial port None ========================================================================= short test summary info ========================================================================== SKIPPED [1] test/test_espefuse_host.py:314: system parameters efuse read-protection is supported only by esp32, other chips protect whole blocks SKIPPED [1] test/test_espefuse_host.py:247: when the purpose of BLOCK2 is set SKIPPED [1] test/test_espefuse_host.py:416: 3/4 coding scheme is only in esp32 SKIPPED [1] test/test_espefuse_host.py:654: 3/4 coding scheme is only in esp32 SKIPPED [1] test/test_espefuse_host.py:666: 3/4 coding scheme is only in esp32 SKIPPED [1] test/test_espefuse_host.py:512: IO pins 30 & 31 cannot be set for SPI flash only on esp32 SKIPPED [1] test/test_espefuse_host.py:711: The test only for esp32c2 SKIPPED [1] test/test_espefuse_host.py:680: The test only for esp32 SKIPPED [1] test/test_espefuse_host.py:855: 512 bit keys are only supported on ESP32-S2 and S3 SKIPPED [1] test/test_espefuse_host.py:873: 512 bit keys are only supported on ESP32-S2 and S3 SKIPPED [1] test/test_espefuse_host.py:920: 512 bit keys are only supported on ESP32-S2 and S3 SKIPPED [1] test/test_espefuse_host.py:734: The test only for esp32c2 SKIPPED [1] test/test_espefuse_host.py:824: 3/4 coding scheme is only in esp32 SKIPPED [1] test/test_espefuse_host.py:768: Only chip with 6 keys SKIPPED [1] test/test_espefuse_host.py:1002: The test only for esp32c2 SKIPPED [1] test/test_espefuse_host.py:1188: 3/4 coding scheme is only in esp32 SKIPPED [1] test/test_espefuse_host.py:1209: 3/4 coding scheme is only in esp32 SKIPPED [1] test/test_espefuse_host.py:971: The test only for esp32 SKIPPED [1] test/test_espefuse_host.py:1025: Only chip with 6 keys SKIPPED [1] test/test_espefuse_host.py:1130: The test only for esp32c2 SKIPPED [1] test/test_espefuse_host.py:1158: Only chip with 6 keys SKIPPED [1] test/test_espefuse_host.py:1099: The test only for esp32 SKIPPED [1] test/test_espefuse_host.py:1242: The test only for esp32, supports 2 key blocks SKIPPED [1] test/test_espefuse_host.py:1279: The test only for esp32, supports 2 key blocks SKIPPED [1] test/test_espefuse_host.py:1262: The test only for esp32, supports 2 key blocks SKIPPED [1] test/test_espefuse_host.py:1292: The test only for esp32c2, supports one key block SKIPPED [1] test/test_espefuse_host.py:1307: The test only for esp32c2, supports one key block SKIPPED [1] test/test_espefuse_host.py:1322: The test only for esp32c2, supports one key block SKIPPED [1] test/test_espefuse_host.py:1337: The test only for esp32c2, supports one key block SKIPPED [1] test/test_espefuse_host.py:1359: Supports 6 key blocks SKIPPED [1] test/test_espefuse_host.py:1397: Supports 6 key blocks SKIPPED [1] test/test_espefuse_host.py:1449: The test only for esp32c2 SKIPPED [1] test/test_espefuse_host.py:1430: The test only for esp32 SKIPPED [1] test/test_espefuse_host.py:1468: Only chip with 6 keys SKIPPED [1] test/test_espefuse_host.py:1502: 3/4 coding scheme is only in esp32 SKIPPED [1] test/test_espefuse_host.py:1523: Tests are only for esp32. (TODO: add for all chips) SKIPPED [1] test/test_espefuse_host.py:1560: Tests are only for esp32. (TODO: add for all chips) SKIPPED [1] test/test_espefuse_host.py:1695: For this chip, FE and SB keys go into one BLOCK SKIPPED [1] test/test_espefuse_host.py:1714: For this chip, FE and SB keys go into one BLOCK SKIPPED [1] test/test_esptool.py:269: ESP32 only SKIPPED [1] test/test_esptool.py:290: ESP32 only SKIPPED [1] test/test_esptool.py:335: ESP32 only SKIPPED [1] test/test_esptool.py:607: ESP8266 only SKIPPED [1] test/test_esptool.py:692: ESP8266 only SKIPPED [1] test/test_esptool.py:768: Bootloader header needs to be modifiable - without sha256 SKIPPED [1] test/test_esptool.py:828: ESP8266 only ERROR test/test_espsecure.py::SigningTests::test_generate_and_extract_key_v2 - OSError: [Errno 39] Directory not empty: '/tmp/tmpqb8asebj' FAILED test/test_espefuse.py::TestBurnKey::test_burn_key_34_coding_scheme - AttributeError: 'NoneType' object has no attribute 'dtr' FAILED test/test_espefuse.py::TestBurnKey::test_burn_key_no_coding_scheme - AttributeError: 'NoneType' object has no attribute 'dtr' FAILED test/test_espefuse.py::TestBurnBlockData::test_burn_block_data_34_coding - AttributeError: 'NoneType' object has no attribute 'dtr' FAILED test/test_espefuse.py::TestBurnBlockData::test_burn_block_data_normal - AttributeError: 'NoneType' object has no attribute 'dtr' FAILED test/test_espefuse.py::TestBurnEfuse::test_burn_efuses - AttributeError: 'NoneType' object has no attribute 'dtr' FAILED test/test_espefuse.py::TestBurnBit::test_burn_bits - AttributeError: 'NoneType' object has no attribute 'dtr' FAILED test/test_espefuse_host.py::TestReadCommands::test_adc_info - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestReadCommands::test_check_error - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestReadCommands::test_dump - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestReadCommands::test_get_custom_mac - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestReadCommands::test_help - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestReadCommands::test_help2 - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestReadCommands::test_summary - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestReadCommands::test_summary_json - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestReadProtectionCommands::test_read_protect_efuse - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestReadProtectionCommands::test_read_protect_efuse2 - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestReadProtectionCommands::test_read_protect_efuse4 - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestWriteProtectionCommands::test_write_protect_efuse - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestBurnCustomMacCommands::test_burn_custom_mac - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestBurnCustomMacCommands::test_burn_custom_mac2 - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestBurnCustomMacCommands::test_burn_custom_mac3 - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestSetFlashVoltageCommands::test_set_flash_voltage_1_8v - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestSetFlashVoltageCommands::test_set_flash_voltage_3_3v - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestSetFlashVoltageCommands::test_set_flash_voltage_off - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestSetFlashVoltageCommands::test_set_flash_voltage_off2 - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestBurnEfuseCommands::test_burn_efuse - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestBurnEfuseCommands::test_burn_mac_custom_efuse - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestBurnBlockDataCommands::test_burn_block_data_check_args - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestBurnBlockDataCommands::test_burn_block_data_check_errors - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestExecuteScriptsCommands::test_execute_scripts_nesting - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestExecuteScriptsCommands::test_execute_scripts_with_check - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestExecuteScriptsCommands::test_execute_scripts_with_check_that_only_one_burn - FileNotFoundError: [Errno 2] No such file or director... FAILED test/test_espefuse_host.py::TestExecuteScriptsCommands::test_execute_scripts_with_index_and_config - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestMultipleCommands::test_burn_bit - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestMultipleCommands::test_multiple_cmds_help - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espefuse_host.py::TestMultipleCommands::test_not_burn_cmds - FileNotFoundError: [Errno 2] No such file or directory: 'python' FAILED test/test_espsecure.py::ESP32SecureBootloaderTests::test_digest_bootloader - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/256bit_key.bin' FAILED test/test_espsecure.py::ESP32SecureBootloaderTests::test_digest_rsa_public_key - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/rsa_secure... FAILED test/test_espsecure.py::SigningTests::test_extract_binary_public_key - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/ecdsa_secure_boot_si... FAILED test/test_espsecure.py::SigningTests::test_generate_and_extract_key_v2 - ValueError: ('Could not deserialize key data. The data may be in an incorrect format, it ... FAILED test/test_espsecure.py::SigningTests::test_sign_v1_data - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/ecdsa_secure_boot_signing_key.pem' FAILED test/test_espsecure.py::SigningTests::test_sign_v1_data_pkcs8 - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/ecdsa_secure_boot_signing_k... FAILED test/test_espsecure.py::SigningTests::test_sign_v2_append_signatures - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/rsa_secure_boot_sign... FAILED test/test_espsecure.py::SigningTests::test_sign_v2_append_signatures_multiple_steps - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/rsa_s... FAILED test/test_espsecure.py::SigningTests::test_sign_v2_data - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/rsa_secure_boot_signing_key.pem' FAILED test/test_espsecure.py::SigningTests::test_sign_v2_multiple_keys - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/rsa_secure_boot_signing_... FAILED test/test_espsecure.py::SigningTests::test_verify_signature_public_key - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/ecdsa_secure_boot_... FAILED test/test_espsecure.py::SigningTests::test_verify_signature_signing_key - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/ecdsa_secure_boot... FAILED test/test_espsecure.py::ESP32FlashEncryptionTests::test_encrypt_decrypt_app - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/hello-world-s... FAILED test/test_espsecure.py::ESP32FlashEncryptionTests::test_encrypt_decrypt_bootloader - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/bootlo... FAILED test/test_espsecure.py::ESP32FlashEncryptionTests::test_encrypt_decrypt_non_default_conf - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/... FAILED test/test_espsecure.py::AesXtsFlashEncryptionTests::test_encrypt_decrypt_app - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/hello-world-... FAILED test/test_espsecure.py::AesXtsFlashEncryptionTests::test_encrypt_decrypt_app_512_bit_key - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/... FAILED test/test_espsecure.py::AesXtsFlashEncryptionTests::test_encrypt_decrypt_bootloader - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/bootl... FAILED test/test_espsecure.py::AesXtsFlashEncryptionTests::test_padding - FileNotFoundError: [Errno 2] No such file or directory: 'secure_images/256bit_key.bin' FAILED test/test_esptool.py::TestFlashing::test_adjacent_flash - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_adjacent_independent_flash - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_compressed_nostub_flash - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_compressible_file - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_compressible_non_trivial_file - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_correct_offset - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_erase_range_messages - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_highspeed_flash - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_length_not_aligned_4bytes - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_length_not_aligned_4bytes_no_compression - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_no_compression_flash - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_partition_table_then_bootloader - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_partition_table_then_bootloader_no_compression - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_partition_table_then_bootloader_nostub - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_repeated_address - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_short_flash - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_single_byte - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_write_no_overlap - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_write_overlap - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_write_sector_overlap - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashing::test_zero_length - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashSizes::test_flash_size_keep - AssertionError: unsupported chip for test: --ignore FAILED test/test_esptool.py::TestFlashSizes::test_high_offset - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashSizes::test_high_offset_no_compression - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashSizes::test_invalid_size_arg - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashSizes::test_large_image - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashSizes::test_large_no_compression - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashSizes::test_write_no_compression_past_end_fails - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashSizes::test_write_past_end_fails - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestFlashDetection::test_flash_id - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestStubReuse::test_stub_reuse_with_synchronization - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestErase::test_chip_erase - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestErase::test_large_region_erase - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestErase::test_region_erase - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestSectorBoundaries::test_end_sector - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestSectorBoundaries::test_end_sector_uncompressed - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestSectorBoundaries::test_overlap - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestVerifyCommand::test_verify_failure - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestVerifyCommand::test_verify_success - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestVerifyCommand::test_verify_unaligned_length - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestReadIdentityValues::test_read_mac - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestMemoryOperations::test_memory_dump - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestMemoryOperations::test_memory_read - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestMemoryOperations::test_memory_write - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestKeepImageSettings::test_detect_size_changes_size - KeyError: '--ignore' FAILED test/test_esptool.py::TestKeepImageSettings::test_keep_does_not_change_settings - KeyError: '--ignore' FAILED test/test_esptool.py::TestLoadRAM::test_load_ram - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestBootloaderHeaderRewriteCases::test_flash_header_no_magic_no_rewrite - TypeError: sequence item 5: expected str instance, NoneType found FAILED test/test_esptool.py::TestBootloaderHeaderRewriteCases::test_flash_header_rewrite - KeyError: '--ignore' FAILED test/test_esptool.py::TestAutoDetect::test_auto_detect - TypeError: sequence item 3: expected str instance, NoneType found FAILED test/test_esptool.py::TestVirtualPort::test_auto_detect - IndexError: tuple index out of range FAILED test/test_esptool.py::TestVirtualPort::test_auto_detect_virtual_port - IndexError: tuple index out of range FAILED test/test_esptool.py::TestVirtualPort::test_highspeed_flash_virtual_port - IndexError: tuple index out of range FAILED test/test_esptool.py::TestReadWriteMemory::test_read_write_memory_rom - KeyError: '--ignore' FAILED test/test_esptool.py::TestReadWriteMemory::test_read_write_memory_stub - KeyError: '--ignore' =========================================================== 110 failed, 28 passed, 46 skipped, 1 error in 9.93s ============================================================ ```
radimkarnis commented 2 years ago

Hi @kloczek,

The first four scripts (the ones you added to --ignore list) are not meant to be run directly (more info here).

Nevertheless, esptool.py project tests were not written with pytest in mind.

We are taking a lo-fi approach of running the test python scripts one by one, see here. Most of the test are also meant to be run with real hardware connected. Just running pytest will therefore not work out of the box.

kloczek commented 2 years ago

Nevertheless, esptool.py project tests were not written with pytest in mind.

Nevertheless I would try gentrly encorage you to try to have look closer on pytest as it is de facto standard test suite framework fo +8/9 of all modules 😄 I'm not hiding that as I'm working on some Linux/Solaris distro (now with +940 modules packaged) I'm trying to standardise all tests around pytest because it allow not only test generated packages but allow as well use build procedure to evaluate/audit all packaged python code. None of other approaches to python modules testing provides such useful platform. If for example pytest would be useable to it is easy to install for example 'pytest-blackin build env and by execute 'pytest --black to have integrated with regular test suite black module tests. The same is with many, many other pytest extensions allowing keep python code better.

Nevertheless as I found some units working I've added all failing units to --deselect list to have now some baseline for regression tests between versions. In other words I have now kind of solution but it would be good to improve pytest usability 😋

kloczek commented 1 year ago

Here is pytest output for exptool 4.3 (new units started failing esptool-pytest.txt