WallarooLabs / wally

Distributed Stream Processing
https://www.wallaroolabs.com
Apache License 2.0
1.48k stars 68 forks source link

Make a Logger available end to user computation steps #106

Open SeanTAllen opened 8 years ago

SeanTAllen commented 8 years ago

so instead of something like

fun apply(input: U64)

we have:

fun apply(input: U64, logger: Logger)


Initially this can be a single shared logger. We should be designing with the idea of being able to provide different loggers to each "app"/"pipeline".

As part of this pass, we need to be able to set a LogLevel for the logger.


as part of this, we would:

SeanTAllen commented 8 years ago

I was thinking about this. I think we need to supply a logger per pipeline out of the gate. But perhaps pipeline is the wrong level to be thinking at, perhaps it should be "app" which means right now that a shared logger would be fine because you cant run a more than 1 app per process right now.

Mixing logging across topologies was one of the things that made debugging Storm via log files awful. Twitter mentions this in the Heron paper. Ideally you don't want to be debugging via log files but it can be a very effective method for some problems therefore we should make it as