dlumbrer
commented
5 years ago I'm trying to get the worker log (that contains the logs of the jobs) in order to filter it by a Handler and save them in the corresponding meta log (just for testing, this will not be the code):
class MetaJobLogHandler(StreamHandler):
def __init__(self, job):
StreamHandler.__init__(self)
self.job = job
self.job.meta['log'] = ""
self.current_job = None
def emit(self, record):
msg = self.format(record)
if self.job.id in msg or not self.current_job:
self.current_job = self.job
self.current_job.meta['log'] += "{} \n".format(msg)The problem that I faced is that the logger (logger = logging.getLogger('rq.worker')) does not contain the logs from the Perceval job, I have to find in what logger are the Perceval logs writing.
Any idea @sduenas ?
sduenas
We should find a way to store the log messages produced by each job. These messages contain valuable information to keep track of the status of a job and to debug any possible error. So far these messages are only printed by the workers.
The easiest way to add this feature is to store the logs into log files but that will make difficult to access them using the rest API that
arthurdprovides. Maybe the best way is to store them within the jobs in RQ and Redis.