CE-Programming / CEmu

Third-party TI-84 Plus CE / TI-83 Premium CE emulator, focused on developer features
https://ce-programming.github.io/CEmu/
Other
1.1k stars 78 forks source link

cemu-autotester CLI does not find correct hashes; running program through the CEmu autotester dialog does #430

Closed captain-calc closed 2 years ago

captain-calc commented 2 years ago

What's wrong, and with what software version?

Operating System: Fedora 36 CEmu version: CEmu v2.0dev (git: 28575a4) Describe your issue:

The CLI-based testing script does not get the correct hashes for a particular test. Homescreen output appears as expected when running the test manually. Putting the test through the CEmu Autotester dialog reports that all correct hashes were found. The CLI autotester and the one in the CEmu GUI are not reporting the same results.

The testing framework requires a Linux distribution (only tested under Fedora), Python 3.9+, and Bash.

What are the steps to reproduce this issue?

  1. Clone the MemoryClassTest repo, available here.
  2. Set the TI84PCE_ROM_PATH and PROGRAM_SRC_PATH environment variables following the instructions in the repo's README.
  3. Run bash runtests.sh tests/. This script calls a Python program that will automatically update every autotest.json file with the ROM path you provided in TI84PCE_ROM_PATH.
  4. The script will compile all of the tests and then run them. The code that calls the cemu-autotester in the testing script is here.
  5. The test memory_ReadOnlyMemory_read_seq_4 will fail with 3 out of 3 mismatched hashes.

Any logs, error output, screenshot, other comments...?

The testing script's output on the failed test (using cemu-autotester -d): autotester_error

The output for the CEmu Autotester dialog: CEmu_autotester_output

jacobly0 commented 2 years ago

You are hashing the following images:

failure_hash1_num1_dump

failure_hash2_num2_dump

failure_hash3_num3_dump