NOAA-GSL / VxIngest

Other
2 stars 0 forks source link

move the Begin a_time log message into the VxIngest class where it should have been all along. That is what was breaking the scraper, along with the reformatted log message preamble. #295

Closed randytpierce closed 8 months ago

randytpierce commented 8 months ago

The Begin a_time log message was incorrectly placed in the parse routine of the run_ingest, instead of in the runit routine. This in combination with the longer log preamble was breaking the scraper.

randytpierce commented 8 months ago

I think embedding the metrics is a good idea and better than scraping but I think it's another whole issue.

On Tue, Jan 9, 2024 at 8:15 AM Ian McGinnis @.***> wrote:

@.**** approved this pull request.

That's a good catch. It sounds like since we're now calling runit directly and not main on these run_ingest_threads.py modules, the scraper wasn't getting the output it expected.

Looks good!

Longer term, I wonder if outputting a JSON doc with the required metadata would make the scraper process more robust - it can be tricky to tell what information it requires.

— Reply to this email directly, view it on GitHub https://github.com/NOAA-GSL/VxIngest/pull/295#pullrequestreview-1811373483, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGDVQPVMS7UEHE4PFSLCMU3YNVNITAVCNFSM6AAAAABBSHJSXKVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTQMJRGM3TGNBYGM . You are receiving this because you were assigned.Message ID: @.***>

-- Randy Pierce