search

nsidc / granule-metgen

Metadata generator for direct-to-Cumulus era
Other
0 stars 0 forks source link

Create and write audit trail log to a configurable destination #42

Open lisakaser opened 3 months ago

lisakaser commented 3 months ago

3 story points

acceptance criteria 1: Given: When: run MetGenC Then: every granule has a log output summarizing metrics (success/fail; when/how long; error reason)

acceptance criteria 2: Given: When: run MetGenC Then: show a summary of number of granules, how many success/fail, error reason

acceptance criteria 3: Given: When: run MetGenC Then: for every error log all detailed information for possible cause of error

NOTE: This story can conclude with a functional example of each three acceptance criteria - details can be added along the way when we find more errors/metrics that need output or tracking

lisakaser commented 1 month ago

Ops preference: log file plus output directly while running Suggestion to add a quiet option - should be separate issue for later.

eigenbeam commented 1 week ago

To test, you can run metgenc and inspect the stdout info that will scroll by, and also inspect the logfile that will get created named, creatively, metgenc.log.

afitzgerrell commented 4 days ago

Hi, I've tested this and approve almost all facets of all the Acceptance Criteria. And, "almost" only pertains to the error messaging. Logging/summary-wise, I think everything is on a good path.

The only thing I was left wondering after diving in to test the error reporting is if some of the errror messaging can be made a little more intuitive to uses when they don't realize they've made a mistake?

In testing error messages, I first tried adding the option "--number -5". The error I receive is: Unable to process data: Indices for islice() must be None or an integer: 0 <= x <= sys.maxsize. I know what I’ve done, thus I can noodle out what the error message means…but two questions:

  1. Can it instead report “Unable to process data: number specified must be an integer: 0 <= x <= sys.maxsize.” so it’s more intuitive?
  2. My interest was sparked by the “<= sys.maxsize”, what is the max number of granules that can be processed?

I next tested not having data in the data directory, and received: Unable to process data: list index out of range Could this instead say something like: No data found in data directory?

Then I tested an .ini file devoid of a response to “overwrite_ummg = “. I received: Unable to process data: 'Exception' object has no attribute 'show' Could the error message instead point to the problem in the .ini file: I need to know what to do with the umm-g files, Dave

All my other tests went swimmingly: I tested incorrectly naming the data dir in my .ini file and in that case, the error message was clear: The configuration is invalid: * The data_dir does not exist. Subsequent similar tests: making the output dir name incorrect and not including the config file location also led to similarly intuitive error messages.