dchaley / deepcell-imaging

Tools & guidance to scale DeepCell imaging on Google Cloud Batch
8 stars 2 forks source link

Log entries are duplicated #279

Closed dchaley closed 2 months ago

dchaley commented 2 months ago

Follow-up to #220 : now that we're outputting with the new handler, we're getting duplicate log entries. Example: [job]

2024-07-11 14:45:25.745 PDT Ran prediction in 3.0 s; success: True
2024-07-11 14:45:25.745 PDT INFO:deepcell_imaging:Ran prediction in 3.0 s; success: True
2024-07-11 14:45:25.745 PDT Saving raw predictions output to gs://deepcell-batch-jobs_us-central1/job-runs/j3d36718b-a686-4bd5-9035-fb6c59e6c68a/raw_predictions.npz.gz
2024-07-11 14:45:25.745 PDT INFO:deepcell_imaging:Saving raw predictions output to gs://deepcell-batch-jobs_us-central1/job-runs/j3d36718b-a686-4bd5-9035-fb6c59e6c68a/raw_predictions.npz.gz
2024-07-11 14:45:26.251 PDT Saved output in 0.51 s
2024-07-11 14:45:26.251 PDT INFO:deepcell_imaging:Saved output in 0.51 s

Screenshot:

Image

We have 2 options:

  1. Remove the handler from the default, top-level logger
  2. Change the deepcell logger to not propagate

I lean toward option 1, with the further change of adding the GCP-formatted handler to the top-level logger so all logs get handled properly.