search

sasjs / cli

Command line interface for creating, compiling, and building SAS® projects
https://cli.sasjs.io
MIT License
37 stars 5 forks source link

feature request: stream log to a file as the flow is running #587

Closed mehtaanirudh closed 3 years ago

mehtaanirudh commented 3 years ago

Currently, SASJS only fetches the log after the flow has run. It would be good to add a feature to show the log as the flow is running. It will not only help with debugging but also take away some anxiety!

tmoody commented 3 years ago

+1. On two occasions over the past few days this would have been valuable for determining the cause of a SAS job running well past its expected duration, but SASjs has failed to spot the end of the long-running process that was killed and the log is now no longer available.

krishna-acondy commented 3 years ago

@mehtaanirudh / @tmoody There's an implementation of this feature nearly ready for release now. It saves the log to a file as we're polling to check for the job's status. Would it be useful at all to show the logs in the console as well as saving them to a file?

allanbowe commented 3 years ago

Whilst welcoming @mehtaanirudh / @tmoody 's perspectives also, I would say that it would not be useful - the logs can be very large, and the first thing you would typically do is search it for ERRORs / WARNINGs (if there was a non zero return code).

I guess that's something that WOULD be useful - if we could display (in the console) the first occurrence of a line that begins with WARNING or ERROR.

mehtaanirudh commented 3 years ago

@allanbowe looking for errors/warnings is just one of the many things an engineer would do. SASjs retrieves the log post completion of a job anyway. However, while the code is running, a user has no idea what is going on inside the code. A job in production can run for hours and there is no way to identify whether it's running as expected or not. Sometimes, you can have a feel of the run by looking at number of observations printed in the log or time taken by a particular step.

The size of the log is irrelevant because SASjs would fetch the log after the end of the run anyway. So if we stream it to a file then the size should not be an issue. Streaming to console could become a problem.

allanbowe commented 3 years ago

The location of the log would be useful thing to print (immediately) to the console