[RFE] Improve Log Manager

StykMartin commented 4 years ago

In Restraint 0.3.0 new feature was introduced. Log manager is keeping logs until the task is finished/aborted and then it will upload it back to Beaker.

This may be suitable in the case where everything is running by automation, however, user has no chance to check the status if needed.

What should be considered -

User can opt-in and define that wanna get a feed from the task
Periodically upload task output back to Beaker ( with some reasonable timing)

StykMartin commented 4 years ago

@beaker-project/restraint-maintainers please provide your opinions.

cbouchar commented 4 years ago

If you allow user to opt-in, I suspect they will all opt-in which defeats what you were trying to achieve with the Log Mgr change. Also it's also difficult trying to triage issues without log data. I'm thinking periodic upload may be the best way to go.

AloisMahdal commented 4 years ago

While we're at it, it would be great if user could (from runtest.sh) opt-in for additional "hot" files to upload. We often do eg.

leapp upgrade .. >leapp.out 2>leapp.err

which can sometimes take 1-2 hours, so (within some conservative perf. related default timing) it would be useful to be able to see not just taskout.log but also other files updated regularly.

Some use cases for checking taskout.log or other files ad-hoc:

Seeing if my test perhaps already failed in earlier step.

Eg.: When using upstream frameworks that don't talk to the lab, seeing failure earlier in taskout.log can save an expensive re-run.

Eg.2: After I rewrote runtest.sh and/or beaker library, on next run I could see any potential errors.
When a test is taking looong, being able to guess progress.

Eg. My upgrade is taking too long and I don't know why; from output i could make a good guess if it's CDN being slow, machine being slow or something stuck. And make educated decision whether to re-run or wait.

AloisMahdal commented 4 years ago

But feel free to ignore my scope creep. :-)

Just bringing back taskout.log would be better. Even if it was eg. every 5 mins. (Not sure what was the frequency before.)

JohnVillalovos commented 4 years ago

Being able to have the log as before via a config setting would be great. As we have no issues with our log servers so for us it is only a negative to not have log messages during the job run.

restraint-harness / restraint

[RFE] Improve Log Manager #152