search

callahantiff / PheKnowLator

PheKnowLator: Heterogeneous Biomedical Knowledge Graphs and Benchmarks Constructed Under Alternative Semantic Models
https://github.com/callahantiff/PheKnowLator/wiki
Apache License 2.0
159 stars 29 forks source link

TODO: Finalize KG Construction Survey #54

Closed callahantiff closed 4 years ago

callahantiff commented 4 years ago

I am working on the qualitative component of our evaluation and am requesting your review of a Google Form I created to help organize this information.

TODO: Please take a look at the Google Form (link to form can be found here) and let me know if you have any edits by 11:59pm on May 22, 2020?

LEHunter commented 4 years ago

Pointer was to the paper, not a form.

On May 21, 2020, at 2:54 PM, Tiffany J. Callahan notifications@github.com<mailto:notifications@github.com> wrote:

I am working on the qualitative component of our evaluation and am requesting your review of a Google Form I created to help organize this information.

TODO: Please take a look at the Google Form (link to form can be found herehttps://docs.google.com/document/d/1ykgMNMFBLUOoWIChidtBJDPKf_GojxuXIU7MJoff4hM/edit?disco=AAAAGiqskGE) and let me know if you have any edits by 11:59pm on May 22, 2020?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/callahantiff/PheKnowLator/issues/54, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AACWZKNZ4L42TOEVPETFXPDRSWIHRANCNFSM4NHE5MRA.

LEHunter commented 4 years ago

I found it. Looks quite comprehensive. A few thoughts: In section 3, some systems don’t have explicit releases. In section 5, do you want to check for clinical data sources? I’m not sure what the Scale question means — perhaps something about Maximum size of inputs? Also maybe number of inputs (can it assemble multiple ontologies into a coherent KG)? In section 6 I might add cloud compatibility as well as OS. Section 9 should include licensing terms more generally than just open/not. Not sure what you are getting at in section 8 — is this about having a docker container? In section 10, do any have a literate programming (e.g. Jupyter / R Markdown) tutorial?

L

callahantiff commented 4 years ago

I found it. Looks quite comprehensive. A few thoughts: In section 3, some systems don’t have explicit releases. In section 5, do you want to check for clinical data sources? I’m not sure what the Scale question means — perhaps something about Maximum size of inputs? Also maybe number of inputs (can it assemble multiple ontologies into a coherent KG)? In section 6 I might add cloud compatibility as well as OS. Section 9 should include licensing terms more generally than just open/not. Not sure what you are getting at in section 8 — is this about having a docker container? In section 10, do any have a literate programming (e.g. Jupyter / R Markdown) tutorial? L

Thanks @LEHunter!

I went ahead and made the following changes:

Better?

LEHunter commented 4 years ago

Yes, quite comprehensive. .

On May 21, 2020, at 4:23 PM, Tiffany J. Callahan notifications@github.com<mailto:notifications@github.com> wrote:

I found it. Looks quite comprehensive. A few thoughts: In section 3, some systems don’t have explicit releases. In section 5, do you want to check for clinical data sources? I’m not sure what the Scale question means — perhaps something about Maximum size of inputs? Also maybe number of inputs (can it assemble multiple ontologies into a coherent KG)? In section 6 I might add cloud compatibility as well as OS. Section 9 should include licensing terms more generally than just open/not. Not sure what you are getting at in section 8 — is this about having a docker container? In section 10, do any have a literate programming (e.g. Jupyter / R Markdown) tutorial? L

Thanks @LEHunterhttps://github.com/LEHunter!

I went ahead and made the following changes:

Better?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/callahantiff/PheKnowLator/issues/54#issuecomment-632376593, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AACWZKKJ55C3MDLIHM3RWMLRSWSUJANCNFSM4NHE5MRA.

bill-baumgartner commented 4 years ago
callahantiff commented 4 years ago

  • Section 1: Most recent date of interaction with code repository

    • do we need a ">1 year" response?

  • Section 2, step 1

    • is this separate from instructions for how to download data? Must it be an automatic process?

  • Section 2, steps 2/3

    • can't an edge list or edge set be considered a knowledge graph? What's the distinction?

  • Section 5

    • do you want to capture the data sources used by each method?

  • Section 9

    • I see "docker" listed in section 8, but I think it applies to section 9 too. Similarly, if it's python code you might ask if it's in PyPI. If it's Java code, is the code available in a Maven repository, etc. I guess what I'm really getting at is perhaps a question related to the ease of installation of the system.

Thanks @bill-baumgartner! These are great suggestions! I went ahead and

Do you want to see it again before I start the evaluation?

callahantiff commented 4 years ago

@LEHunter and @bill-baumgartner Last task, I'd like to run the list of methods I will apply the survey to by you (n=25 methods). You can find the list with links here. Please be patient when opening this link, it works but may take a few seconds to load.

bill-baumgartner commented 4 years ago

I was confused until I scrolled and saw the Eligible column. That is quite a list! Looks good overall. I see you have one called Metabolomics-in-SPOKE. Should you also include SPOKE itself?

callahantiff commented 4 years ago

I was confused until I scrolled and saw the Eligible column. That is quite a list! Looks good overall. I see you have one called Metabolomics-in-SPOKE. Should you also include SPOKE itself?

I would, but they don't seem to have a public repository. I'm happy to list that as the reason they are not included though.

LEHunter commented 4 years ago

Yep, do say that. They say they are going to release a COVID SPOKE, but I haven’t seen that either.

On May 26, 2020, at 3:53 PM, Tiffany J. Callahan notifications@github.com<mailto:notifications@github.com> wrote:

I was confused until I scrolled and saw the Eligible column. That is quite a list! Looks good overall. I see you have one called Metabolomics-in-SPOKE. Should you also include SPOKEhttps://spoke.ucsf.edu/ itself?

I would, but they don't seem to have a public repository. I'm happy to list that as the reason they are not included though.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/callahantiff/PheKnowLator/issues/54#issuecomment-634300499, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AACWZKLQ3MJJ2AJNMZJW5FLRTQ26NANCNFSM4NHE5MRA.

callahantiff commented 4 years ago

OK, I think we can close this now. I will work on completing the survey over the next week. Thank you both for your help!