search

tsoding / porth

It's like Forth but in Python
628 stars 50 forks source link

Implement binary format for test outputs and errors #52

Closed kolumb closed 2 years ago

kolumb commented 2 years ago

Tried to implement binary format that was described in #50

drocha87 commented 2 years ago

@kolumb since you are working on the tests I would like to point something. There are some tests which have a long output (like stack.porth for example), and then is a little bit hard to spot where the error is (imagine a more complex output).

I don't know if is possible or simple to implement a way to tell where the output has diverged, something like running a diff (if the user has it installed) and then printing the result to output, or something like this.

Maybe one way to solve this is breaking the tests in small pieces and have more detailed tests somehow.

drocha87 commented 2 years ago

And I think the test.py should have a flag to stop the tests as soon as it fails or maybe a flag to run the tests silently and only output the broken tests, what you think?

kolumb commented 2 years ago

I think that it could be reported as separate issue that this PR introduces. Because, if I understand correctly, right now main priority that author has is to rewrite compiler and test script in porth. So I tried to keep everything as simple as possible. I contributed only because I thought that this PR could be helpful for rewriting process.

rexim commented 2 years ago

The fact that binary files are not displayed properly in diffs is definitely an issue. I'll need to think about a better simple format.

rexim commented 2 years ago

Something like this. What do you guys think?

rexim commented 2 years ago

@kolumb thank you for the contribution! :+1: