leo-arch / clifm

The shell-like, command line terminal file manager: simple, fast, extensible, and lightweight as hell.
https://github.com/leo-arch/clifm/wiki
GNU General Public License v2.0
1.35k stars 39 forks source link

Automated testing #199

Open Slackadays opened 1 year ago

Slackadays commented 1 year ago

Is your feature request related to a problem? Please describe. This isn't related to any specific problem, but rather a whole class of them caused by lack of code testing. Logic issues are all too common in software with a lot of branches and conditions like clifm, and integration/unit tests are a perfect way to catch these bugs as they appear. For example, #113.

Describe the solution you'd like Add automated testing through integration test scripts or unit tests and provide badges for test results and optionally code coverage in the readme.

Describe alternatives you've considered Have tests done locally

Additional context For Clipboard, adding a couple basic scripts to test functionality revealed some extremely insidious bugs that I only would have found much later in advance when I try to copy binary data with a certain format. I think clifm would benefit heavily from similar scripts so that you don't have to go bug hunting later. However, I just realized that it might be tricky to get this working in a script as clifm is all about interactive terminals while CB has scriptability baked right in.

leo-arch commented 1 year ago

Yes, you rightly spotted the difficulty: automated tests work great when it comes to command line arguments, but I wasn't able to find yet a way to make these tests inside clifm's interactive shell.

EDIT: On the other side, this might be dangerous, insofar as badly designed test input (I'm thinking about fuzzy testing, which is the best technique to spot input processing errors) might end up destroying your file system, since clifm can do that if instructed to do so.

Slackadays commented 1 year ago

I wasn't able to find yet a way to make these tests inside clifm's interactive shell

One way I can think of to combat this is to make a "fake terminal" for clifm using a couple standard Linux commands: https://stackoverflow.com/questions/52187/virtual-serial-port-for-linux https://man7.org/linux/man-pages/man3/openpty.3.html and then write to this fake terminal using the same data that a keyboard would give. Then, check the output by reading from that fake tty and you're good to go.

since clifm can do that if instructed to do so

Clipboard can easily do the same (ok, only for the cut action) but this hasn't been an issue as all tests take place in their own special temporary folder and special temporary files are made to play with that don't impact anything else. Additionally, if you really want some armor, Docker or jails could be helpful.

leo-arch commented 1 year ago

I'll give this a try. Thanks.

spenserblack commented 1 year ago

On this subject, since you document that clang-format is required, might make sense to have a workflow to run that and ensure that PRs adhere to preferred styling.