Optional filter on buffer data before write

[sirmspencer]

It would be great if we could filter the output instead of dumping it all. For example, each of our tests reconnects to postgres with a fixture. This logs some debug data that isn't relevant to the test itself. With many tests, it makes sorting through that output to find the actual errors is extra work.

I am not sure if would go with the buffer: https://github.com/lambdaisland/kaocha/blob/v0.0-118/src/kaocha/plugin/capture_output.clj

Or with the reporter https://github.com/lambdaisland/kaocha/blob/v0.0-118/src/kaocha/report.clj#L126

[alysbrooks]

Hi @sirmspencer! That could definitely be very useful. I think what we want to add that feature to capture_output, perhaps by doing something similar to the bypass macro.

I also wonder if it could be hooked into a logging library like log4j to filter entries in a more structured way.

Another thing to consider is that you may want to change the logging at the source when you're running tests, e.g., change the log level to a higher one or disable logging from certain namespaces. I suspect filtering from Kaocha would be most useful for noisy libraries or when debugging a failing test. There might be other cases where the kind of filtering you want to do is very test-specific, and you don't want to change how you do production logging for the sake of tests. I don't know if you have any thoughts on that based on your experience.

Finally, is this something you would want to attempt a PR for? (I'm not sure whether you're asking about the buffer and reporter just to document some implementation options or because you want to take a crack at it.)

[sirmspencer]

I have not worked with buffers very often, so I don't know where to begin for that. If you said in the reporter, its a pretty simple string operation to split, apply a pred, and rejoin.

It would be nice if I could programmatically turn off certain logging, but that's also something I have never seen. That got me thinking a little more though. If I created a fixture to snag all of stdout, then filter and dump at the end, would koacha see it? (ie does the capture end before or after a fixture is closed) edit: not progress here, with my knowledge anyway.

[alysbrooks]

Note that you can certainly create a reporter in your own namespace and do the splitting, predicate application, and rejoining and use it in your own project's tests.edn. Let me know if you want help with that. I think keeping the filtering logic in the output capture plugin makes more sense for our official solution, however.

(If you're curious what that kind of filtering looks like, you can look at timbre: https://github.com/ptaoussanis/timbre#advanced-minimum-levels-and-namespace-filtering. I believe the more sophisticated/complicated Java logging frameworks also have similar functionality.)

I believe captures starts before fixtures, but I'd have to look.

[sirmspencer]

Note that you can certainly create a reporter in your own namespace

Good point! Its funny, I do this by default for react packages (most are bloated), but forget to do so in clojure. I think this solves it for me, if no one else wants to take on the buffer filtering.