bgyori
commented
4 years ago Hi Laura,
Thanks for your questions!
- Statement assembly (de-duplication, hierarchical refinement resolution, belief calculation, etc.) is not done immediately when processing some output from a reading system. For instance in
from indra.sources import eidos ep = eidos.process_json_file('...') stmts = ep.statementsstmtswill be a list of "raw" Statements, each of which will have a single piece of evidence in itsevidencelist. At this point, two Statements can be duplicates of each other (i.e., the same causal relationship extracted from multiple sentences/documents) and their beliefs are not yet determined.
You have to call INDRA's assembly modules to do further processing. These are all documented, for instance, the Preassembler (which deals with de-duplication and finding hierarchical refinements) is documented here: https://indra.readthedocs.io/en/latest/modules/preassembler/preassembler.html. As a user, probably the easiest way to run INDRA assembly is through the functional interface of the indra.tools.assemble_corpus tool (documented at https://indra.readthedocs.io/en/latest/modules/tools/index.html#module-indra.tools.assemble_corpus). The assemble_corpus tool gives you access to various assembly functions and filters over a list of statements. Continuing the example above, you could do
from indra.tools import assemble_corpus as ac
assembled_stmts = ac.run_preassembly(stmts, **extra_args)here the optional extra_args are things you can configure to change the way assembly is done (documented here: https://indra.readthedocs.io/en/latest/modules/tools/index.html#indra.tools.assemble_corpus.run_preassembly).
At the end of this process, you will have "assembled" statements in the assembled_stmts list that can have multiple pieces of evidence aggregated in their evidence list, and will have belief scores calculated according to overall support.
- INDRA's built-in default belief scorer starts with empirical estimates of the prior random and systematic error rates of reading systems. It first calculates the individual belief of each assembled statement (as mentioned above, statements that have gone through assembly have one or more evidences associated with them, potentially from multiple reading systems) by calculating the joint probability that given all the pieces of evidence, the statement is not incorrect. The code that does this is here:
https://github.com/sorgerlab/indra/blob/master/indra/belief/__init__.py#L110-L154
For each source (eidos, hume, etc.) that a Statement has evidence from, an error component
r^e + sis calculated whereris the random error rate for the given source,sis the systematic error rate for the given source andeis the number of evidences from that source. The product of these error terms across all sources becomes the error probability, and one minus this error probability becomes the prior belief. (For simplicity, I did not detail how negative evidences are handled - they are taken into account as evidence for a statement not being true; you can see this in the code).
After the prior belief calculation, the refinement graph of Statements (given an ontology) is used to propagate these beliefs from more specific to less specific statements. For instance, S1: "rainfall causes floods" is less specific than S2: "a large increase in rainfall causes a small increase in floods", and so S2's evidences will be propagated to S1's pool of evidences when calculating S1's final belief.
To your last question in 2.a: the identity of specific source documents is not taken into account in this default belief scorer. However, the belief calculation is fully configurable and you can implement your own BeliefScorer class according to your own assumptions about a probability model of statement correctness (just subclass the BeliefScorer class https://github.com/sorgerlab/indra/blob/master/indra/belief/__init__.py#L27 and pass the instance of your scorer as the belief_scorer argument to ac.run_preassembly).
- That error is generated by the PMC web service when the paper with the given ID is not available in PMC's native NXML format. INDRA's PMC client seems to be operational currently, e.g.,
from indra.literature import pmc_client pmc_client.get_xml('4322985')returns the NXML corresponding to https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4322985/. If you provide more specifics on what ID's you are trying to access, I can try to help more.
As a final note, I couldn't fully determine based on your questions what knowledge sources you are using with INDRA (Eidos? or some biology-specific sources?) and what your use case is. We are happy to help more and give more specific advice tailored to your use case if needed.
guerrerosimonl
My team encountered a couple of problems and unclear concepts during our work and we would like feedback to help us understanding better.
INDRA statements: we could not find information on how de-duplication of statements is done. Every statement is supposed to have a list of evidence objects (a.k.a. list of sentences from literature?) but we are only able to retrieve one object for each statement. Thus, what happened with the rest? Does the algorithm pick one? Is it a new sentence build from an ensemble of all of the evidence objects?
Belief scores: we were not able to unravel how belief scores are calculated. a. What we know: the prior probability of each statement is calculated based on the number of evidences it has and their sources. What we want to know: How does this exactly work? What happen if two or more of the retrieved evidences come from the same source (e.g. they are citations from the same article)? b. How are the belief scores calculated and what is the relation with the prior probability (further than the INDRA documentation explains, because it is not clear for us from there)?
We could not solve the WARNING message “indra.literature.pmc_client - PMC client returned with error cannotDisseminateFormat: The meta data format ‘pmc’ is not supported by the item or by the repository”.
Thank you very much in advance for your help!
Laura