[Epic] Scheduled Prediction Runs

[D1M1TR10S]

Summary ⭐️

Run scheduled prediction runs for all models in Ersilia's database. Analyzing patterns on prediction runs helps their research. Build an Actions workflow that triggers prediction runs for all models in the database. Make sure it's performant and scalable – and less likely to incur heavy consumption costs on AWS.

Objective 🎯

We want to answer the question: how long does it take each model to run a prediction for each molecule? (starting with the most common molecules). This way we have metrics around prediction times and other data points related to the performance of models.

Supporting Details ✏️

Dependencies 🖇

• GitHub Actions
• AWS Cloud Service of choice - compute costs could get high if we're not careful. Could store it in S3 or DynamoDB, @miquel is exploring options.

✅ Tasks

M1: Gather requirement/prototyping/Spikes (2 Weeks)

• [x] #301
• [x] #302

  M2: Implementation (~2 weeks)

• [x] #396
• [x] #595
• [ ] Test the predict action workflow when Dynamo DB is api feature is integrated in Ersillia model.

Team 🏀

DRI: @honeyankit Backup: @megamanics

Timeline 🗓

Documentation 📓

[miquelduranfrigola]

Hi @honeyankit, I have implemented a sample command. Responding now to #302 to document it!

[D1M1TR10S]

There’s a bug in the Scheduled Prediction Runs Workflow. Usually happens when there’s a YAML syntax issue.

[D1M1TR10S]

Update

The cron job is done. Picks a model randomly and runs every night.

Can we specify the model ID as a parameter? Then we can test a model of choice at any time. Should be easy to implement so we’re working on adding that to the workflow. @GrantBirki @Lehcar

[honeyankit]

Can we specify the model ID as a parameter?

@D1M1TR10S : This PR has taken care of adding model Id as a input to predict ersillia model input https://github.com/ersilia-os/ersilia/pull/476

The cron job is done. Picks a model randomly and runs every night.

This is not done and is pending.

[miquelduranfrigola]

Thanks @honeyankit - apologies, it might have been me who have the wrong update in the meeting. I thought the cron action was done but not active.

[D1M1TR10S]

Pretty much done. DynamoDB integration – credentials to submit results into DynamoDB. We have a library to do that. Just need to bundle that in Ersilia. Just add a parameter that says "save" or "don't save".

Once DynamoDB is done we want to test it in the Actions workflow.

[GemmaTuron]

@miquelduranfrigola will activate it as a cron job to test it is working despite the results won't be uploaded to DynamoDB @Lehcar and @honeyankit were working on a parallelization option

[honeyankit]

@GemmaTuron If I remember correctly, the parallelization work is already completed.

[honeyankit]

Adding corn job and testing the integration of Dynnamo DB + Ersilia with Prediction run is left. Estimated time to complete: ~1 week

[D1M1TR10S]

Status of integration with DynamoDB: hitting a wall. We have the library already to do so. Haven't been able to do so in Ersilia yet. Contractor hasn't completed this step yet. Once the integration is complete, just add the cron job to trigger the tests.

@miquelduranfrigola will follow up and complete this on his own, once DynamoDB is ready. All we need is the cron job.

Two workflows:

• Automated schedule – add a cron job to this one
• Manual trigger

Cron job duration interval: Counting installation of Ersilia. Between 10-20 minutes. => Every 15 minutes, trigger testing workflow.

@honeyankit will close this out once done – by Monday (2/6) at the latest

[honeyankit]

Cron job duration interval of 15 minutes to the Ersilia prediction schedule run is added.

[honeyankit]

The last action item ( Test the predict action workflow when Dynamo DB is api feature is integrated in Ersillia model. ) will be completed by Ersilia as discussed in our last meeting: