Pipeline fails due to ontobio errors wrt MGI

[kltm]

[2023-11-21T12:12:12.007Z]     result = rule.value.run_test(active_annotation, config, group=group)
[2023-11-21T12:12:12.007Z]   File "/usr/local/lib/python3.10/dist-packages/ontobio/io/qc.py", line 137, in run_test
[2023-11-21T12:12:12.007Z]     result = self._run_if_context(annotation, config, group=group)
[2023-11-21T12:12:12.007Z]   File "/usr/local/lib/python3.10/dist-packages/ontobio/io/qc.py", line 132, in _run_if_context
[2023-11-21T12:12:12.007Z]     result = self.test(annotation, config, group=group)
[2023-11-21T12:12:12.007Z]   File "/usr/local/lib/python3.10/dist-packages/ontobio/io/qc.py", line 365, in test
[2023-11-21T12:12:12.007Z]     only_go = [t for conjunctions in withfrom for t in conjunctions.elements if (t.namespace == "GO" and go_id.namespace == "GO" and t.identity != go_id.identity)] # Filter terms that aren't GO terms and different from GO ID
[2023-11-21T12:12:12.007Z] TypeError: 'NoneType' object is not iterable
[2023-11-21T12:12:16.129Z] make: *** [Makefile:84: target/groups/mgi/mgi.group] Error 1
[2023-11-21T12:12:16.129Z] make: *** Waiting for unfinished jobs....

[kltm]

https://github.com/biolink/ontobio/pull/660

[kltm]

Will no one rid me of this turbulent python?

[dustine32]

We isolated this error to this GPAD line in noctua_mgi-src.gpad:

MGI     MGI:1916040     located_in      GO:0005634      PMID:23369715   ECO:0000305     GO:0031061              20200518        MGI             contributor=https://orcid.org/0000-0001-7476-6306|noctua-model-id=gomodel:MGI_MGI_1916040|contributor=https://orcid.org/0000-0003-2689-5511|model-state=production

The with/from field value GO:0031061 was recently obsoleted (no replaced_by) and the ontobio code to validate this field was erroneously blanking out the with/from with a None rather than an iterable empty list []. See this line.

@mugitty I think the return None at this line should really be a continue so that the loop finishes checking other with/from terms and returns empty list if still needed.

[mugitty]

The terms in the with/from field can be a list delimited with 'AND'. For example, GO:123 AND GO:456 AND GO:789. If GO:456 is obsolete, the parser should not remove it. If the clause is of the form GO:123 OR GO:456 OR GO:789 and one of the terms is obsolete, the parser should be able to remove it. This is a question for @pgaudet. For replacement or alternate terms, the parser can make changes, however, should the parser update the meaning of the annotation?

[kltm]

Thanks to @mugitty , this should hopefully clear soon.

[kltm]

New expressions with:

[2023-11-30T16:21:04.576Z] 2023-11-30 16:21:02 [INFO] [PID:10255 TID:139913818542080] [assocparser.py:541 in `association_generator`]  Parsed 135426 assocs from 135452 lines. Skipped: 0
[2023-11-30T16:21:04.576Z] Writing ttl to disk
[2023-11-30T16:21:11.057Z] Traceback (most recent call last):
[2023-11-30T16:21:11.057Z]   File "/usr/local/bin/validate.py", line 710, in <module>
[2023-11-30T16:21:11.057Z]     cli(obj={})
[2023-11-30T16:21:11.057Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__
[2023-11-30T16:21:11.057Z]     return self.main(*args, **kwargs)
[2023-11-30T16:21:11.057Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
[2023-11-30T16:21:11.057Z]     rv = self.invoke(ctx)
[2023-11-30T16:21:11.057Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1688, in invoke
[2023-11-30T16:21:11.057Z]     return _process_result(sub_ctx.command.invoke(sub_ctx))
[2023-11-30T16:21:11.057Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
[2023-11-30T16:21:11.057Z]     return ctx.invoke(self.callback, **ctx.params)
[2023-11-30T16:21:11.057Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
[2023-11-30T16:21:11.057Z]     return __callback(*args, **kwargs)
[2023-11-30T16:21:11.057Z]   File "/usr/local/lib/python3.10/dist-packages/click/decorators.py", line 33, in new_func
[2023-11-30T16:21:11.057Z]     return f(get_current_context(), *args, **kwargs)
[2023-11-30T16:21:11.057Z]   File "/usr/local/bin/validate.py", line 573, in produce
[2023-11-30T16:21:11.057Z]     end_gaf = mixin_a_dataset(valid_gaf, mixin_metadata_list, group_metadata["id"], dataset, absolute_target,
[2023-11-30T16:21:11.057Z]   File "/usr/local/bin/validate.py", line 459, in mixin_a_dataset
[2023-11-30T16:21:11.057Z]     mixin_gaf = produce_gaf(mixin_dataset_id, mixin_src, ontology, gpipaths=gpipaths, paint=True, group=mixin_metadata["id"], rule_metadata=rule_metadata, format=format, rule_contexts=context, gaf_output_version=gaf_output_version)[0]
[2023-11-30T16:21:11.057Z]   File "/usr/local/lib/python3.10/dist-packages/ontobio/validation/tools.py", line 11, in wrapper
[2023-11-30T16:21:11.057Z]     output_file = file_function(*args, **kwargs)
[2023-11-30T16:21:11.057Z]   File "/usr/local/bin/validate.py", line 247, in produce_gaf
[2023-11-30T16:21:11.057Z]     gafwriter.write_assoc(assoc)
[2023-11-30T16:21:11.057Z]   File "/usr/local/lib/python3.10/dist-packages/ontobio/io/assocwriter.py", line 87, in write_assoc
[2023-11-30T16:21:11.057Z]     vals = self.as_tsv(assoc)
[2023-11-30T16:21:11.057Z]   File "/usr/local/lib/python3.10/dist-packages/ontobio/io/assocwriter.py", line 204, in as_tsv
[2023-11-30T16:21:11.057Z]     return assoc.to_gaf_2_2_tsv()
[2023-11-30T16:21:11.057Z]   File "/usr/local/lib/python3.10/dist-packages/ontobio/model/association.py", line 593, in to_gaf_2_2_tsv
[2023-11-30T16:21:11.057Z]     ConjunctiveSet.list_to_str(self.evidence.with_support_from),
[2023-11-30T16:21:11.057Z]   File "/usr/local/lib/python3.10/dist-packages/ontobio/model/association.py", line 329, in list_to_str
[2023-11-30T16:21:11.057Z]     return "|".join([conj.display(conjunct_to_str=conjunct_to_str) for conj in conjunctions])
[2023-11-30T16:21:11.057Z] TypeError: 'NoneType' object is not iterable
[2023-11-30T16:21:14.285Z] make: *** [Makefile:84: target/groups/mgi/mgi.group] Error 1
[2023-11-30T16:21:14.285Z] make: *** Waiting for unfinished jobs....

@mugitty and @dustine32 on the scene.

[kltm]

I just wanted to check if the fix was "in" for this? (To move from "In progress" to Clearing")

[mugitty]

@kltm - This fix has been released in ontobio. You can close the ticket

[kltm]

Cheers!