search

bakdata / aws-lambda-r-runtime

Serverless execution of R code on AWS Lambda
https://medium.com/bakdata/running-r-on-aws-lambda-9d40643551a6
MIT License
143 stars 52 forks source link

Can the run time printing the event to console be controlled on the user side? #33

Closed DaveParr closed 5 years ago

DaveParr commented 5 years ago

I believe (though I may be wrong) that https://github.com/bakdata/aws-lambda-r-runtime/blob/0788f5d82d864c6e66342036b9c744b684429694/runtime/src/runtime.R#L31 is logging the entire event to cloudwatch.

I can see the value in doing this in many cases, but in the case where you may be passing a large data set in the body of the json, this makes the logs, and the in console test return, pretty impossible to use.

Is it possible to control this in your 'default' R runtime layers, without the user forking the project and rolling their own?

I've naively included logging::loglevel("WARN") at the start of script.R, though this doesn't seem to make an impact.

An ideal hypothetical for me would be to be able to specifically not log the body of json An acceptable one would be to not capture logs from the runtime.R at all Another acceptable one would be to control the log level of runtime.R, though this shouldn't impact the main script.R log level

I'm happy to do my best to look into this as well (I can see your project is picking up steam 🚂 !) but I might need a bit of guidance about where to start and how to implement.

philipp94831 commented 5 years ago

Hi @DaveParr, I think it would be best to set the log level of these messages to debug (https://cran.r-project.org/web/packages/logging/logging.pdf). Then only if the logging config prints debug messages, they are printed to the console. This can be controlled in the user script I guess. Feel free to submit a PR :)

DaveParr commented 5 years ago

I've done the trivial easy part, i.e. the modify the logging level and PR is incoming. I'm still curious as to a method to control logging from script.R and how it might be possible. Working on it atm.

philipp94831 commented 5 years ago

Hi @DaveParr, I also looked into it and made it work: https://github.com/bakdata/aws-lambda-r-runtime/compare/feature/debug-logging-in-runtime To change the log level it suffices to do basicConfig(level='DEBUG'). The long log output generated by large payloads also is the final fix (among others) for #29 Do you mind if I merge my PR and decline yours for now?

DaveParr commented 5 years ago

Totally fine, thank's a lot for doing it :)