Biolink 3.1.1 migration (related but separate from qualifiers)

colleenXu commented 2 years ago

https://github.com/biolink/biolink-model/compare/v2.4.8...v3.0.3 https://github.com/biolink/biolink-model/blob/master/Migration_3.0_Guide.md

Major:

New predicates:
- children of physically_interacts_with starting line 2112: physically_directly_interacts_with, binds, indirectly_physically_interacts_with
- mode_of_inheritance_of and its inverse starting line 5779 (odd. seems better as a node or edge attribute)
- removed causes_adverse_event and its inverse, replaced with (starting line 6151. Also discusses in lines 10178-10210):
  - has_adverse_event and its inverse
  - has_side_effect and its inverse
- inverses:
  - location_of_disease (line 5084)
  - primarily_composed_of (line 5251)

Minor:

Clarify predicates:
- associated_with is for statistical correlations weaker than correlated_to (correlation analysis method)
  - so...semmeddb's ASSOCIATED_WITH....can stay mapped to this? or go to related_to?
- ~associated_with_likelihood_of is for statistical correlations. Discusses in lines 10260-10283~
- try not to use interacts_with directly! use a child term of it.
Clarify EPC:
- only give 1 primary knowledge source (use aggregator for all others). EDIT 2022-10-28: not an issue for x-bte annotation
- new options: supporting data source, supporting data set, etc. (lines 6585-6832 and 6964-7010)
deprecate "original knowledge source" -> ~replace all instances of this in x-bte annotation (edge-attribute mapping)? And~ check on code. EDIT 2022-10-28: not an issue for x-bte annotation
deprecate "source" -> replace all instances of this in x-bte annotation (edge-attribute mapping)? And check on code
New edge-attributes:
- FDA adverse event level (line 1157, range FDA_IDA_adverse_event_enum line 12021)
- has supporting study result (line 6509)

New categories (node types)?

stuff on "study result" (lines 7275-7313)
genetic inheritance (line 8452)
virus (line 8489)
cellular organism (line 8497)

Other "qualifiers"?

population context qualifier (line 1085)
temporal context qualifier (line 1099)
temporal interval qualifier (line 1106)
frequency qualifier (line 1359)
severity qualifier (line 1365)
sex qualifier (line 1372)
clinical modifier qualifier (line 1386)
sequence variant qualifier (line 1393)
quantifier qualifier (line 1399)
catalyst qualifier (line 1412)
stage qualifier (line 1422)

tokebe commented 2 years ago

@colleenXu If I understand this list of changes, as far as implementation within BTE logic I see a few steps:

Update biolink file
Ensure all tests are up to date with new predicates/etc
Check over any hardcoded attributes for compatibility, update as necessary
Check/implement support for qualifier hierarchy traversal in biolink-model package
Write appropriate qualifier hierarchy tests

Are there any other things on my end that I'm missing? I'm aware this will entail a lot of x-bte annotation editing on your end. Regarding that, I'm wondering if it would make sense for you to take notes on what feels repetitive and we could later look into possibly making a new issue about increasing automation around x-bte annotation generation using further templating/etc to lessen the time it takes for you to make changes?

colleenXu commented 1 year ago

Noting: for semmeddb, changing mapping of SEMMEDDB:ASSOCIATED_WITH from associated_with (now a statistical correlation) to related_to https://github.com/NCATS-Tangerine/translator-api-registry/commit/5e51b26a77ce9b427701304da71086773ee8c0c2

EDIT: leaving the mapping to interacts_with: SEMMEDDB:INTERACTS_WITH and idisk drug-supplement interactions

colleenXu commented 1 year ago

EDIT: revising list based on changes to master branch

Basically DONE on the x-bte annotation side of things.

Involved commits from here to here

The APIs (biothings unless noted "external")with adjusted predicates/edge-attributes/comments were:

iDISK
MyChem
semmeddb

Note that I didn't find anything to adjust for Multiomics or Text-Mining APIs. However, they may want to do their own adjustments.

colleenXu commented 1 year ago

EDIT 2022-12-13: actually not changing the title anymore.

~Noting that I changed the title of the GO MF API....so we'd want to change the title in the BTE config list too. Is there a branch of bte-trapi that the qualifier work is going into? @tokebe~

tokebe commented 1 year ago

All qualifier work is currently under branches named add-qualifiers.

colleenXu commented 1 year ago

new branch made in bte-trapi with the little changes (BTE's own smartapi yaml) and go-mf api name https://github.com/biothings/BioThings_Explorer_TRAPI/tree/add-qualifiers

colleenXu commented 1 year ago

sierra moxon decided to make biolink "interacts_with" not abstract, so we can use it again. Changed biolink/monarch and dgidb yamls for biolink3 branch, so they use interacts_with again https://github.com/NCATS-Tangerine/translator-api-registry/commit/47f3054ffd6ee565e0b4d490e3b908e58951825e

https://github.com/biolink/biolink-model/commit/c256cee9487e4eac2186f6463ad9baf7acf7f638.

colleenXu commented 1 year ago

Discussing moving to 3.1.1 rather than 3.0.3...shouldn't be a big deal but I want to do a review.

Also we know BTE would ignore deprecated predicates since they are no longer in the yaml...

https://github.com/biolink/biolink-model/compare/v3.0.3...v3.1.1

colleenXu commented 1 year ago

Review of 3.1.1 done.

Don't see any big deal (Jackson may only need to do testing / any updating of automated tests to migrate to biolink 3.1.1)
CX has some things to check on and do

To-do, for CX

Check on what is the correct prefix to use? (Asked Sierra, see github post)
- Orphanet Disease IDs. We use ORPHANET in MyDisease. But I see ORPHA (3.0.3) -> orphanet (3.1.1, line 133) and ORPHANET -> orphanet (3.1.1, line 7026)
- CLINVAR variant IDs. We don't use these IDs right now. 3.1.1 seems to use CLINVAR (lines 7712, 7743, 9217-9218, 9419)

EDIT: DONE:

commit. Run SEMMEDDB notebook for auto-generating x-bte, using 3.1.1.
- Edit code as needed.
- Likely going to keep manual mappings to use qualifiers
- This file is provided with AUGMENTS, STIMULATES, INHIBITS mappings but it doesn't match my mappings for STIMULATES and INHIBITS - activity vs activity_or_abundance.
Translator Slack post. Tell Sierra of oddities in 3.1.1:
- predicates
- ~lines 1865-1867 and 1878-1880: both associated with likelihood of and its inverse are canonical predicates??~
- lines 2949-2950 and 2969-2970: gene associated with condition and its inverse. The domain and range for both are swapped and now incorrect?
- lines 3916-3917, 3924-3925, 3929-3930, 3937-3938: domain and range for these predicates seems off (nucleic acid entities are not the only things that catalyze, molecular activities don't have substrates).
- qualifiers:
- lines 1203-1215: case of example values, typo with "analong", mismatch between examples and notes
- lines 1226-1241: case of example values, mismatch between examples and notes
- lines 1268-1274: mismatch between examples and notes
- line 10441: typo absorbtion

Fully deleted

Deprecated predicates:

has population context
has temporal context
has real world evidence of association with
chemically interacts with
molecularly interacts with
- decreases molecular interaction (+ inverse)
- increases molecular interaction (+ inverse)
many descendants of affects ( 17 * 6 + 1 = 103 total)
- affects/increases/decreases abundance of (+ inverses)
- affects/increases/decreases activity of (+ inverses)
- affects/increases/decreases expression of (+ inverses)
- affects/increases/decreases folding of (+ inverses)
- affects/increases/decreases localization of (+ inverses)
- affects/increases/decreases metabolic processing of (+ inverses)
- affects/increases/decreases molecular modification of (+ inverses)
- affects/increases/decreases synthesis of (+ inverses)
- affects/increases/decreases degradation of (+ inverses)
- affects/increases/decreases mutation rate of (+ inverses)
- affects/increases/decreases splicing of (+ inverses)
- affects/increases/decreases stability of (+ inverses)
- affects/increases/decreases transport of (+ inverses)
- affects/increases/decreases secretion of (+ inverses)
- affects/increases/decreases uptake of (+ inverses)
- process regulates/positively regulates/negatively regulates process (+ inverses)
- entity regulates/positively regulates/negatively regulates entity (+ inverses)
- affects expression in
positively regulates (+ inverse)
negatively regulates (+ inverse)
directly interacts with

Deprecated categories: nutrient, macronutrient, micronutrient, vitamin

Noting but no action needed from us

New predicates: member of / has member lines 1821 - 1842
Added inverses to: transcribed to -> transcribed from,translates to->translation of,precedes->preceded by,is side effect of->has side effect`
Changes to categories:
- Macromolecular complex isn't a mixin anymore (lines 7335-7342)
- Added NucleosomeModification, PosttranslationalModification
Added info on predicates: has active component lines 1925-1929, assesses and is assessed by lines 2126+2140, has predisposing factor line 3004
Change attribute supporting document to supporting documents lines 5113-5119
Remove attributes like source, original knowledge source
Deprecated the mixin predicates increases amount or activity of (+ inverse), decreases amount or activity of (+ inverse)

tokebe commented 1 year ago

Deployed to prod 🚀

biothings / biothings_explorer