Closed asummers closed 3 years ago
Hmm, so you'd basically like to run as usual but also produce the artifact / log with summary - i.e. info about passed and failed checks? Sounds reasonable, but there are a few questions to answer:
Aside from that, there are a few things that I don't understand in your described use case:
Please elaborate :)
For JSON, I the general pattern is to do a check for e.g. Jason before spitting it out, but in the interest of making it extensible, maybe allowing a user to specify an output format module would be a good future goal if need arises, so someone could export as HTML, JSON, XML, whatever. For my purposes, a plain text comment is completely sufficient, and I don't need any external dependencies.
Having ex_check write that to a file would be useful. Then it's just snaking that output file into postprocessing as needed and eventual curl calls. For credo, I was more talking about seeing the visual feedback in the output on the PR of "skipped due to not having credo", but I realize I can simply take off the detect
to make it run always. Nonetheless, having the users be able to stay on GitHub and see the failures without having to go to CI and see the summary would be useful to others besides myself, I think =)
Sorry for taking some time, but I'm coming back with a good news. I've added support for producing manifest file in #24. Since it's plaintext that includes list of all passed, failed and skipped checks, and since --manifest
command line option allows to specify the file path, it should cover your needs from this issue.
Btw, my motivation for doing this was to allow to only run failed tools with --failed
command line option, just like ExUnit allows. But it's great to kill two birds with one shot :)
I'll merge and release v0.13 with this addition as well as all others from you in next few days.
My use case is that I would like to lift the end summary of which checks ran/passed/failed and post them into a comment back to the PR to alleviate the needs for developers to log into CI to see what failed. Because I run my CI checks centrally for many projects, I worry that one or more projects will be in a state where a skipped check will not be apparent to users from e.g. not having
credo
installed.Would you be interested in something like a e.g.
--summary-to-file
opt that allowed me to redirect this to a file/env var or some mechanism by which I can export this data somewhere?