mozilla-services / updatebot

Automation for updating third party libraries for Firefox
Mozilla Public License 2.0
8 stars 5 forks source link

Improve Logging to include library name #347

Closed tomrittervg closed 4 months ago

tomrittervg commented 5 months ago

If you're looking at a long string of log output like

[task 2024-03-19T06:13:00.735Z] [Info]    VendorTaskRunner: Processing aom for a new upstream revision 69975b155791efa10f7b1dd01e28d9c57360a26d, the most recent job is 369.
[task 2024-03-19T06:13:00.735Z] [Debug]   VendorTaskRunner: ================================================
[task 2024-03-19T06:13:00.735Z] [Info]    VendorTaskRunner: Beginning VendorTaskRunner._process_new_job
[task 2024-03-19T06:13:00.735Z] [Debug]   VendorTaskRunner:  Arguments: (<tasktypes.vendoring.VendorTaskRunner object at 0x7f51a0359790>, {{ aom  Core  Audio/Video: Playback  11631186b36e96afce18808ebebb17cc23a010ef  https://aomedia.googlesource.com/aom  False  regular  padenot@mozilla.com  padenot  None  None  None  media/libaom/moz.yaml [{vendoring True None [] [] release linux None None None }] }}, {vendoring True None [] [] release linux None None None }, '69975b155791efa10f7b1dd01e28d9c57360a26d', '2024-03-25 00:00:00', <Job id: 369 library: aom>) {}
[task 2024-03-19T06:13:00.735Z] [Debug]   VendorTaskRunner: ================================================
[task 2024-03-19T06:13:00.735Z] [Info]    VendorTaskRunner: Beginning BaseTaskRunner._should_process_new_job
[task 2024-03-19T06:13:00.735Z] [Debug]   VendorTaskRunner:  Arguments: (<tasktypes.vendoring.VendorTaskRunner object at 0x7f51a0359790>, {{ aom  Core  Audio/Video: Playback  11631186b36e96afce18808ebebb17cc23a010ef  https://aomedia.googlesource.com/aom  False  regular  padenot@mozilla.com  padenot  None  None  None  media/libaom/moz.yaml [{vendoring True None [] [] release linux None None None }] }}, {vendoring True None [] [] release linux None None None }, '69975b155791efa10f7b1dd01e28d9c57360a26d') {}
[task 2024-03-19T06:13:00.735Z] [Debug]   MySQLDatabase: ================================================
[task 2024-03-19T06:13:00.735Z] [Info]    MySQLDatabase: Beginning MySQLDatabase.get_all_jobs_for_library
[task 2024-03-19T06:13:00.735Z] [Debug]   MySQLDatabase:  Arguments: (<components.db.MySQLDatabase object at 0x7f51a178c490>, {{ aom  Core  Audio/Video: Playback  11631186b36e96afce18808ebebb17cc23a010ef  https://aomedia.googlesource.com/aom  False  regular  padenot@mozilla.com  padenot  None  None  None  media/libaom/moz.yaml [{vendoring True None [] [] release linux None None None }] }}) {}
[task 2024-03-19T06:13:00.761Z] [Debug]   MySQLDatabase: Function returned a list of 2 objects
[task 2024-03-19T06:13:00.761Z] [Info]    MySQLDatabase: Ending MySQLDatabase.get_all_jobs_for_library
[task 2024-03-19T06:13:00.761Z] [Debug]   VendorTaskRunner: Function returned False
[task 2024-03-19T06:13:00.761Z] [Info]    VendorTaskRunner: Ending BaseTaskRunner._should_process_new_job
[task 2024-03-19T06:13:00.761Z] [Info]    VendorTaskRunner: Because of the task's frequency restrictions (release) we are not processing this new revision now.
[task 2024-03-19T06:13:00.761Z] [Debug]   VendorTaskRunner: Function returned None
[task 2024-03-19T06:13:00.761Z] [Info]    VendorTaskRunner: Ending VendorTaskRunner._process_new_job
[task 2024-03-19T06:13:00.761Z] [Debug]   VendorTaskRunner: ================================================
[task 2024-03-19T06:13:00.761Z] [Info]    VendorTaskRunner: Beginning VendorTaskRunner._reset_for_new_job
[task 2024-03-19T06:13:00.761Z] [Debug]   VendorTaskRunner:  Arguments: (<tasktypes.vendoring.VendorTaskRunner object at 0x7f51a0359790>,) {}
[task 2024-03-19T06:13:00.761Z] [Info]    VendorTaskRunner: Removing any outgoing commits before moving on.

It's very easy to get lost in where you are.

I think we should amend the Logging so the 'category' may include the library we are looking at, and also may include the job_id (if we have one).

https://github.com/mozilla-services/updatebot/blob/78a8ed3b827b7527fa0a4621b138a0c81f80cc87/components/logging.py#L80

Presently the category gets initialized from the classname at https://github.com/mozilla-services/updatebot/blob/78a8ed3b827b7527fa0a4621b138a0c81f80cc87/components/providerbase.py#L82 and then is omitted.

I think the way I would do this is to add methods to LoggerInstance:

def setContext(self, library, job_id=None):
   self.context = library
   if job_id:
      self.context += " job_id=" + str(job_id)

def clearContext(self):
   self.context = ""

And then in the individual log() functions, add the context to where we put the category.