Set up test framework

[m3brown]

• Bootstrap pytest in requirements-test.txt and Dockerfile.test
• Add a couple test files in the tests/ directory
• Update readme with instructions for running tests
• Add .gitignore (source) to ensure we aren't committing unwanted files (venv, pyclass, etc.)

Note: #2 adds another test, which can run on its own but builds on top of the bullet points above. Also, the refactoring in #2 will probably break tests/tests/test_stratifyStack.py... If one of these PRs is merged I'm happy to rebase the other and resolve conflicts.

[tenekev]

Can you explain, in laymen terms, the purpose of these tests? They are well above my level since I'm not a developer. I understand they test individual functions in the script against possible inputs. I do this already manually. Are they a part of the image or they are just for debugging? Can they be incorporated in a Github Action step to ensure a working image.

[m3brown]

Can you explain, in laymen terms, the purpose of these tests? They are well above my level since I'm not a developer. I understand they test individual functions in the script against possible inputs. I do this already manually.

Sure!

I'll provide some helpful links within my comment. Since you said you're not a dev, I'm not making assumptions about what you already know.

The tests I created use a library called pytest, and serve to confirm the individual functions work the way the developer expects them to. The term for this type of testing is Unit Testing, which is the foundational layer of the test pyramid.

The main reason I felt compelled to add unit tests is because I wanted to change some of the functionality without introducing new problems. Consider this workflow:

1. Write test for existing functionality, confirming how it currently works
2. Modify the functionality, which breaks some tests
3. I can review the broken tests and confirm they broke for reasons that make sense (because duh, I changed things) or because I accidentally broke something I didn't want to
   • 3a. go back to my code to get tests passing again, if they weren't supposed to break
   • 3b. update the tests (or delete them, if they're no longer applicable) for tests that I expected to break due to the changes I made.

I imagine your workflow for manual validation is effective for the way you work.

• As the amount of code grows over time, you'll probably find that it's hard for you to mentally keep track of what to validate.
• Manual validation is a big challenge when working as a team, where everyone has different knowledge. Test automation really becomes valuable here, where tests
  • confirm it works a certain way and
  • acknowledge that we meant for it to work that way (i.e. documentation as code)

Are they a part of the image or they are just for debugging?

Technically the tests aren't part of the published docker container. I did add a second Dockerfile (Dockerfile.test) for running tests locally. You could also create a virtualenv and run tests locally without docker. To answer your question -- they're a helpful tool for developers, and do not provide any value to "users" of the tool you've built.

Can they be incorporated in a Github Action step to ensure a working image.

Yes!! A good modern practice is to have unit tests run automatically. This is an important part of Continuous Integration (CI). Github Actions is one way to do this, and a pretty cool way.

For example, I see you already have checks when a pull request is created. We can run the tests every time a PR is created and if any tests fail, the Merge button is grayed out, forcing the developer to reconcile why the tests are failing unexpectedly.

[tenekev]

Thanks for the explanation. I've written quite a lot of python code for personal use but never dabbled into automated testing. Not even on a conceptual level. I've been going through the code to grasp the changes and how to utilize them in the future. I'll probably steal them for my other Immich repo. And I still need to test the changes, that's why I haven't replied.

Again, I'm really happy someone found my project helpful and even contributed.