Create unit tests

[Ahsoka]

Description

If possible at some point it would be really nice if the software had to unit tests to make sure everything is working properly.

Here is the ideal unit test coverage:

• Test when an error occurs in the program the appropriate error handlers work as expected. Specifically:
  • [ ] Error is logged, see here for reference.
  • [ ] Code correctly identifies which module produced the error, see here for reference.
  • [ ] Error debugging info is sent to sentry_sdk, see here, here, and here for reference.
  • [ ] Code is able to retrieve the source code of dependent packages, see source.py for reference.
  • [ ] Identify if URLs used for retrieving source code of dependent packages are correct, see source.py for reference.
  • [ ] Make sure when there is a segmentation fault the program can still log the traceback of the segmentation fault, see here for reference (this one might be a bit tricky to test).
• Test automatic updates work correctly, specifically:
  • [ ] When there is not an update nothing happens, see here for reference.
  • [ ] When there is an update make sure the latest version of the app is installed (that are not pre-release versions), see here for reference.
  • [ ] When a download of a new version of the app is interrupted the app is able to recognized that the download is a corrupted file by comparing the hash of the corrupted file to the hash listed in the sha256-hashes folder, see here for reference.
  • [ ] When a new version of app is installed make sure the toaster notification is shown to the user, see here for reference.
  • [ ] When the user clicks the Update button on toaster notification the app shutdowns correctly and the setup exe for the new version opens, see here for reference.
  • [ ] When the user clicks the What's New? button in the toaster notification the app opens the correct URL for the newest version in the user's default web browser, see here for reference.
  • [ ] When the user clicks the What's New? button in the toaster notification the app display the secondary toaster notification, see here for reference.
• Test various aspects of the program use of the settings file, specifically:
  • [ ] If the settings file is missing the app will automatically create a new settings file, see here for reference.
  • [ ] If the folder that is supposed to contain the settings file is missing the program can create both the folder and file, see here for reference.
  • [ ] If the settings file cannot be decoded the program will create a new settings file, see here for reference.
  • [ ] If the program calls the __setitem__ method of the settings object the settings file will be updated according, see here for reference.
• Test driver related functions, specifically:
  • [ ] When there is no SEAL kit connected the program correctly recognizes this, see here, here, and here for reference.
  • [ ] When there are no drivers on the computer the program correctly recognizes this (side note: this one will probably be kind of hard to test, will have to brainstorm how to do this), see here for reference.
• UI related stuff, these are not as important but would still be really nice:
  • [ ] Test focus order is as expected (in all the pages in pages.py); this test would need to be custom tailored to each one of the previously mentioned pages/popups and if we make it here I will elaborate on exactly what the correct order should be.
  • [ ] Test that certain key presses don't cause weird behavior within the spin boxes, see #15 and https://github.com/Ahsoka/beskar/commit/c586c8b65fde4870b0dfc8b196072ec4ac59ffba for reference.
  • [ ] Test forward and back button work as expected in StartUpPopup, will elaborate on this if we make to this test, see here, here, here, and here for reference.
  • [ ] Test animation in refresh button works properly (not sure exactly how we would test this), see this.
  • [ ] Test animations in StepProgressBar work as expected (not sure exactly how we would this test), see this for reference.
  • NoSEALKitPage:
  • [ ] Test the link to install drivers is still valid, see here for reference.
  • VoltageOffsetPage:
    • [ ] Test when the user moves onto the ApplyVoltagePage the selected offset is stored in the settings file, see here for reference.
  • ApplyVoltagePage:
    • [ ] Test if when user selects an extreme voltage the "This voltage may not be applied accurately..." message appears, see here for reference.
    • [ ] Test when the previously selected voltage is not in the new acceptable voltage range the program automatically sets the voltage to zero, see here for reference.
    • [ ] Test when the user applies the selected voltage the program updates the settings file, see here for reference.
  • DarkCurrentPage:
    • [ ] Test when the user clicks the Refresh button the values on the dark current page update, here and here for reference.
  • ScanPage:
    • [ ] Test that only one scan can be initiated at a time.
    • [ ] Test that when saving a scan it does not interrupt a scan currently in progress (for example, the scan stops until the file dialog is closed).
    • [ ] Test that when dark mode is enabled the file dialog the "dark mode version" of the icon is used in the file dialog (not sure how we would test this), see here for reference.
    • [ ] Test when the user selects the Screenshot option the warning dialog shows up, see here for reference.
    • [ ] Test that dragging the window while a scan is in progress does not halt the scanning, see #13 for reference.
    • [ ] Test that when many zeros are read during scanning the QLabel indicating this shows up, see here and here for reference.
    • [ ] Test that files are in fact saved when using the file dialog, see here for reference.
    • [ ] Test trying to save files to locations that require elevated privileges to write to (such as C:\Program Files or /root).
    • [ ] Test that when a scan is finished the Save button is shown (instead of the Start button) and that the Start button is now enabled for "unscanned tabs".
• Test interaction with the SEAL kit work, this would be hard to automate as you need a human to verify that these worked, however, they might be nice to implement:
  • [ ] Test the apply_voltage function, see here for reference.
  • [ ] Test the interact_with_LED function, see here for reference.

[Ahsoka]

Hey there, @kerrande, thanks for taking an interest in this issue. I saw that you forked my repo and created some skeleton unit tests. If possible, I would like to use pytest as the unit test framework as opposed to the built-in unittest framework. I have updated the previous comment so you can see exactly what coverage the unit tests should have. This is obviously a lot so please don't feel pressured to implement this all. If you have any follow up questions please let me know!

[karnaerrande]

Cool

On Sun, Sep 26, 2021, 6:14 PM Andres Torres @.***> wrote:

Hey there, @kerrande https://github.com/kerrande, thanks for taking an interest in this issue. I saw that you forked my repo and created some skeleton unit tests. If possible, I would like to use pytest https://docs.pytest.org/en/latest/ as the unit test framework as opposed to the built-in unittest framework. I have updated the previous comment so you can see exactly what coverage the unit tests should. This obvious a lot so please don't feel pressured to implement this all. If you have any follow up questions please let me know!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Ahsoka/beskar/issues/8#issuecomment-927426134, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJKMSPJ527MRT2JTKYMW3PLUD7APZANCNFSM47TMM72A .

[Ahsoka]

Also if this is a bit overwhelming, please feel free to check out the other issues as well. Those are probably a bit easier than this one. Once again I greatly appreciate your interested in this! 💜