search

snpeek / snpeek.github.io

https://snpeek.github.io/
MIT License
2 stars 3 forks source link

Dynamically update mps-data.json from SNPedia #15

Open livtanong opened 1 month ago

livtanong commented 1 month ago

To the best of my understanding, mps-data.json describes what variants on which to focus attention, and how to group them. This is currently statically defined.

This means that keeping mps-data.json updated is a manual task. We can improve this workflow.

I see two approaches:

Full automation

  1. Maintain another json file lists genes to focus on
  2. The app ingests this json file, makes requests against SNPEdia to get the list of variants.
  3. The app then builds the dynamic equivalent of mps-data.json

Script

  1. Script performs the functions of [Full Automation] but writes a mps-data.json to disk.
  2. Developer must commit this new file to git, and push.
  3. This can be made more automated in the future via CI/CD

Why Script is better than Full Automation

Why Full Automation is better than Script

I'm almost certain a script is the way to go, because updates are unlikely to happen with enough frequency to warrant pulling this information with each run. But additional research is needed. I'm not even sure if SNPedia has an API that we can access for free. At the very least, scraping would be possible.

livtanong commented 1 month ago

Welp, that's a non-starter. Can't create an account at SNPedia. Which is weird because I'm not on a proxy or anything.

image

It seems like other users are having the same issue. https://www.reddit.com/r/SNPedia/comments/11t2ojr/still_accepting_users/

livtanong commented 1 month ago

An alternative source of information is dbSNP https://clinicaltables.nlm.nih.gov/apidoc/snps/v3/doc.html

But it'll be a whole thing to understand what's going on. Just leaving this here for future reference. For now, I'll just implement the data transformation for #17 manually. (It needs to be manual because additional information regarding the phenotypes and the magnitude are being added, and I worry about being banned if I use a scraper)