search

RobokopU24 / ORION

Code that parses datasets from various sources and converts them to load graph databases.
MIT License
12 stars 13 forks source link

Added aggregator_knowledge_sources for Automat-CAM-KP #209

Closed gaurav closed 5 months ago

gaurav commented 7 months ago

Automat-CAM-KP sources currently look like this:

[ 
     { 
         "resource_id": "infores:go-cam", 
         "resource_role": "primary_knowledge_source", 
         "upstream_resource_ids": None, 
     }, 
     { 
         "resource_id": "infores:automat-cam-kp", 
         "resource_role": "aggregator_knowledge_source", 
         "upstream_resource_ids": ["infores:go-cam"], 
     }, 
 ]

However, we would like to clarify that infores:cam-kp should be treated as an intermediate aggregator knowledge source between the primary KS and Automat. So we would like the output to look like this:

[ 
     { 
         "resource_id": "infores:go-cam", 
         "resource_role": "primary_knowledge_source"
     }, 
     { 
         "resource_id": "infores:cam-kp", 
         "resource_role": "aggregator_knowledge_source", 
         "upstream_resource_ids": ["infores:go-cam"], 
     }, 
     { 
         "resource_id": "infores:automat-cam-kp", 
         "resource_role": "aggregator_knowledge_source", 
         "upstream_resource_ids": ["infores:cam-kp"], 
     }, 
 ]

I've tried to do that in this PR, but I have three questions:

  1. Is this the correct way to use kgxedge.aggregator_knowledge_sources? I've tried to follow the usage in e.g. the MonarchKG parser, but I might have got something wrong.
  2. Do I need to specify infores:automat-cam-kp as an aggregator source, or is that added automatically?
  3. Is it possible to get each aggregator to reference the previous one in the upstream_resource_ids as I've specified above? Or will they all have "upstream_resource_ids": ["infores:go-cam"] as the sole primary knowledge source?

You can see our discussion about this issue at https://github.com/ExposuresProvider/cam-pipeline/issues/118#issuecomment-1908569248

EvanDietzMorris commented 5 months ago

This was included in a different PR, and is in graphs now. The automat- infores is appended in plater, so we only needed to add infores:cam-kp as an aggregator in the parser.