search

eykrehbein / strest

⚡️ CI-ready tests for REST APIs configured in YAML
MIT License
1.74k stars 59 forks source link

Feature: Logfiles #79

Closed eykrehbein closed 5 years ago

eykrehbein commented 6 years ago

Add an option to the cli command to create a log file. Formatted as Json array with an entry for each request where response code, data, headers and a possible error is logged?

jgroom33 commented 6 years ago

Seems reasonable. There's this to help: https://github.com/winstonjs/winston/blob/master/README.md Are you thinking we just cut everything to logging and abandon the cli output?

eykrehbein commented 6 years ago

@jgroom33 I think the cli output should stay as it is because this quick response is a key factor in my opinion. We can just make the file output optional.

jgroom33 commented 6 years ago

I'm assuming logging could output to stdout and optionally a file. this might simplify the code. but, the fancy colorized output would no longer exist

eykrehbein commented 6 years ago

What is more attractive for professional usage? The more descriptive logging style as it is rn or just blank stdout logs?

jgroom33 commented 6 years ago

typically, I think json is useful when there is a need for another system to analyze the data. the primary use I'm concerned with is for CI. for that purpose, the most important things are: Is there an ability to return exit code of 0 or non 0? when the exit code is not 0, what data can be displayed to a human to assist with identifying the issue? so, for this use, json structure does not provide additional value (since a human is using the data). therefore, whatever is easiest to develop is best. at the same time, a json report might be beneficial to show metadata (number of tests, execution time)

the existing cli output is very nice. I think it helps for casual use (non CI) and provides the average postman user with a friendly output.

if we assume both outputs (cli and json) are useful, then we should ask: are they both useful simultaneously? an implementation option is --format=(colorized, json)

perhaps we look at the newman usage of --report and somewhat mimic that.

jgroom33 commented 6 years ago

this feature will help when a vs code extension is developed. It will allow saving the previous executions and loading the saved result. I'm assuming the api is needed for this, but it might be possible without the api.