Zendro-dev / graphql-server-model-codegen

Command line utility to auto-generate the structure files for a graphql server
MIT License
1 stars 2 forks source link

Improve error messages in the unit tests #135

Open asishallab opened 4 years ago

asishallab commented 4 years ago

Currently the unit tests when failing print a non readable error message that is too long and without white-spaces and thus is completely useless. Improve this

Use linux shell's diff and grep

Assume you have the full generated code (generated.txt) and the expected code (expected.txt) and want to see common lines and where the expected code deviates from what has been generated. Using *nix command line tools we can easily highlight which expected lines are missing from the generated code. Consider the following example:

diff -u expected.txt generated.txt | grep -P "^[\s-]" | sed -e '1d'

which generates the following example output

   input searchTranscript_countInput {
     field: transcript_countField
     value: typeValue
     operator: Operator
-    searchChanged: [searchTranscript_countInput]
   }

The - sign indicates the expected line of code missing from the generated code.

Helper function to solve this

Implement a simple test helper function that receives two arguments, the expected and the generated code including white-spaces. The helper then invokes the system and uses the above combination of commands to generate a comprehensive error message. This can then be passed to the mocha unit tests as custom error messages.

Security check

Investigate whether node is aware of the underlying operating system. If so, the above helper function should fall back to the default mocha error messages on Windows systems. We don't really support Win OS, do we?

Adapt existing test cases

All tests that actually compare expected with generated code should be updated to use this new comprehensive custom error messages.

TimRehberg commented 4 years ago

It took a bit of fiddling to get exactly the output we want, but I've achieved the desired results with jsdiff in a standalone proof-of-concept (~20 lines of code), so we're platform independent.

We can also colour code the changes (makes it easier to read in my opinion) and/or hilight individual words/indentifiers (might be useful on long lines? Will test.)

TimRehberg commented 4 years ago

I've had to spend a little more time on the helper because a not-syntax-aware diff is of limited use if you combine a lot of duplicated code with modified function signatures. I'll merge the improved version into the issue135 branch when I'm done fixing the Cassandra unit tests.