search

choderalab / fah-xchem

Tools and infrastructure for automated compound discovery using Folding@home
MIT License
6 stars 3 forks source link

Update activity data daily #126

Closed jchodera closed 2 years ago

jchodera commented 3 years ago

Instead of freezing the activity data in the JSON file, we should update the activity data daily so the Retrospective tab is always up to date

dotsdl commented 2 years ago

I know this is high-value @jchodera; actively working on it now.

dotsdl commented 2 years ago

@jchodera I've looked through the scripts on mskcc1:/home/server/server2/projects/available/covid-moonshot/synthetic-enumeration/sprint-10, but it's not exactly clear how we got the activity data in /home/server/server2/projects/available/covid-moonshot/synthetic-enumeration/sprint-10/activity-data.

Can you share how we do this currently so I can build the component that does this as part of the continuous process?

dotsdl commented 2 years ago

@jchodera do you have a response for this? I want to capture this in our CLI so it can be performed more frequently instead of at the start of the pipeline.

jchodera commented 2 years ago

@dotsdl : Apologies for the delay in response!

In the past, I've been retrieving the current activity data from the link on the COVID Moonshot activity data webpage. The direct link is https://covid.postera.ai/covid/activity_data.csv

I believe that CSV file comes from this GitHub repo, but I'm not sure which file. This repo hasn't been updated since Aug 24, however, so I'm patting in @mc-robinson to help provide guidance on how to get the most up-to-date data. I've also pinged him in slack.

jchodera commented 2 years ago

@dotsdl : It looks like the Moonshot webpage and GitHub repo haven't been updated since Aug 24. @mc-robinson is working on an update in this branch, but it has been giving issues with merge conflicts (something that has been problematic for a while).

@mc-robinson suggests using the code from this script to pull the data from CDD directly. This requires a CDD API key, which @mc-robinson has kindly generated for me.

I've sent you a copy of the API key by slack. We can keep it on the FAH server and also set it up as an org/repo secret, (e.g. COVID_MOONSHOT_CDD_API_KEY) for testing.