Warnings verbosity

[joeflack4]

Overview

I noticed recently that sometimes warnings are overly verbose. For example, sometimes for a given type of warning, it prints a full warning for each instance of an offending class or mapping, rather than printing some text about the type of warning and then providing a concise list of all the offending cases.

Current behavior

Example

In [my instance](https://github.com/monarch-initiative/mondo-ingest/pull/412#discussion_r1493469285), this warning repeated 100's of times, each time with a different mapping object, but the same error message. Here's what it looks like with just 3 entries (the latter 2 are fake / just examples). ``` WARNING:sssom.parsers:One mapping in the mapping set is not well-formed, and therfore not included in the mapping set ({'mapping_justification': 'semapv:UnspecifiedMatching', 'subject_id': 'Orphanet:3080', 'predicate_id': 'oboInOwl:hasDbXref', 'subject_label': 'Intellectual disability, Wolff type'}). Error: object_id must be supplied WARNING:sssom.parsers:One mapping in the mapping set is not well-formed, and therfore not included in the mapping set ({'mapping_justification': 'semapv:UnspecifiedMatching', 'subject_id': 'Orphanet:3081', 'predicate_id': 'oboInOwl:hasDbXref', 'subject_label': 'LABEL'}). Error: object_id must be supplied WARNING:sssom.parsers:One mapping in the mapping set is not well-formed, and therfore not included in the mapping set ({'mapping_justification': 'semapv:UnspecifiedMatching', 'subject_id': 'Orphanet:3082', 'predicate_id': 'oboInOwl:hasDbXref', 'subject_label': 'LABEL'}). Error: object_id must be supplied ```

Suggested behavior

Example

``` WARNING:sssom.parsers:Non well-formed mappings detected in mapping set, and therefore were excluded. Error: object_id must be supplied Offending mappings: ({'mapping_justification': 'semapv:UnspecifiedMatching', 'subject_id': 'Orphanet:3080', 'predicate_id': 'oboInOwl:hasDbXref', 'subject_label': 'Intellectual disability, Wolff type'}). ({'mapping_justification': 'semapv:UnspecifiedMatching', 'subject_id': 'Orphanet:3081', 'predicate_id': 'oboInOwl:hasDbXref', 'subject_label': 'LABEL'}). ({'mapping_justification': 'semapv:UnspecifiedMatching', 'subject_id': 'Orphanet:3082', 'predicate_id': 'oboInOwl:hasDbXref', 'subject_label': 'LABEL'}). ```

Additional details

Related

• 549

[matentzn]

I agree with this, but isnt this going to be print the same number of rows?

What about truncating the output and only print the first 10 cases, and then saying "There are 1230 similar ones, omitted for brevity"?

[joeflack4]

Same number of rows, but text reduced by like 1/3 or so because it won't print the warning message for each line.

I think printing such a sample is also a good idea. Could change behavior to print some / all based on verbosity settings, or save more to a logfile, at some point in the future.

[matentzn]

I explored catching the warnings and basically it will require a huge amount of refactoring, because the warning is written a few methods down from where I could collect the erroneous mappings, and that method is called in dozens of places.

in #502 I have

1. Drastically reduced the verbosity of the error your are particularly concerned with
2. Devised a generic strategy for truncating logs when in a low log level is set (WARNING or below) by extending the root logger and customising it. What it does, in essence, is documenting the source of a logging attempt. If that error has been printed more time than the limit (configurable), it just does not print it. In the end of a command execution, a log summary is printed if and only if an error was truncated.

What do you think?

[joeflack4]

Wow you got right to this. Yeah I think this seems pretty good!