✍️ Contribution period: Tsion Zeleke

Tsion-Zeleke commented 4 months ago

Week 1 - Get to know the community

[X] Join the communication channels
[X] Open a GitHub issue (this one!)
[x] Install the Ersilia Model Hub and test the simplest model
[x] Install Docker if needed, and test another model
[x] Write a motivation statement to work at Ersilia
[x] Submit your first contribution to the Outreachy site

Week 2 - Get Familiar with Machine Learning for Chemistry

[x] Select a model from the list suggested in GitBook
[x] Download and serve the model via the Ersilia Model Hub to ensure it works
[x] Open a repository on your GitHub user with all the necessary files
[x] Select and clean a dataset of 1000 molecules (example notebook 1)
[x] Run predictions for the molecules on the selected model and evaluate the results

Week 3 - Validate a Model in the Wild

[x] Find a suitable dataset with sufficient experimental results
[x] Clean and standardize the dataset
[x] Run predictions and calculate metrics.

Week 4 - Prepare your final application

[x] Submit the final application in the Outreachy website

Tsion-Zeleke commented 4 months ago

I installed all the pre-requisite

(base) tsion@LAPTOP-MRB3DIHJ:~$ gcc --version
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

(base) tsion@LAPTOP-MRB3DIHJ:~$ git --version
git version 2.34.1

(base) tsion@LAPTOP-MRB3DIHJ:~$ conda list
# packages in environment at /home/tsion/miniconda3:

I installed and activated git-lfs

(base) tsion@LAPTOP-MRB3DIHJ:~$ git-lfs install
Git LFS initialized.

I create ersilia env with python version 3.7 using `conda create -n ersilia python=3.7` and activated it using `conda activate ersilia`.

(base) tsion@LAPTOP-MRB3DIHJ:~$ conda activate ersilia
(ersilia) tsion@LAPTOP-MRB3DIHJ:~$

I keep getting the below error when trying to install isaura using `python -m pip install isaura==0.1`

ERROR: Could not find a version that satisfies the requirement isaura==0.1 (from versions: none)
ERROR: No matching distribution found for isaura==0.1

based on messages on slack I tried to install Isaura after installing Ersilia and the installation was sucessful

(ersilia) tsion@LAPTOP-MRB3DIHJ:~/ersilia$ python -m pip install isaura==0.1
  ...
Successfully installed h5py-3.7.0 isaura-0.1

I successfully installed Docker

(base) tsion@LAPTOP-MRB3DIHJ:~/ersilia$ service docker status
 * Docker is running
(base) tsion@LAPTOP-MRB3DIHJ:~/ersilia$

Tsion-Zeleke commented 4 months ago

I installed Ersilia and tested the model. The model run successfully with the below output

(ersilia) tsion@LAPTOP-MRB3DIHJ:~/ersilia$ ersilia -v api run -i "CCCC"
...
23:13:49 | DEBUG    | Rearranging and returning
23:13:49 | DEBUG    | Done with unique posting
{
    "input": {
        "key": "IJDNQMDRQITEOD-UHFFFAOYSA-N",
        "input": "CCCC",
        "text": "CCCC"
    },
    "output": {
        "mw": 58.123999999999995
    }
}
(ersilia) tsion@LAPTOP-MRB3DIHJ:~/ersilia$

I tested a second model using docker. The model run successfully with the below output

🚀 Serving model eos4wt0: morgan-fps
...
(ersilia) tsion@LAPTOP-MRB3DIHJ:~$ ersilia -v api run -i "CCCC"
...
{
    "input": {
        "key": "IJDNQMDRQITEOD-UHFFFAOYSA-N",
        "input": "CCCC",
        "text": "CCCC"
    },
    "output": {
        "outcome": [
            0.0,
            0.0,
            0.0,
            0.0,
            0.0,
            0.0,
            0.0,
            0.0,

Tsion-Zeleke commented 4 months ago

Motivation Letter

Self Introduction

I'd like to introduce myself as a person who is enthusiastic about automation and its impacts. In addition to studying computer science, I completed a 1 year intensive software engineering program with a focus on backend development at ALX. Over the past few years, I have worked in various sectors, including full-stack development and data analysis, utilizing my research and analytical skills to improve smallholder farmers' lives.

How I was Introduced to Machine Learning

I was first introduced to ML when working on a project on developing a REST API for a Y-Maze test. It was based on a pretrained model aiming to automate laboratory researchers' tasks. The project initially seemed intimidating, but with intensive research, I learned a lot and am grateful for the exposure.

Ersilia

It was fascinating to see that Ersilia has already begun the path of utilizing currently released technologies to help the world. I think that technology has the potential to change people's lives in different sectors, but that potential is not fully utilized. There is yet to be discovered. Research centers and labs are not as fully automated as they could be. It was quite exciting when I ran the previous (ersila week 1) task with minimal commands and setups. I was able to run a test, which might have taken a significant amount of time, energy, and skill. Ersilia, in my opinion, is achieving what most software engineers and I hope to achieve, which is using various technologies to improve the world and serve as an inspiration for future innovations. I believe we are in the perfect time where initiatives and ideas can be easily brought to light, but many people still lack knowledge and tools. Growing open-source projects like Ersilia can assist individuals learn and contribute to the community.

Conclusion

It would be a great pleasure to join a nonprofit organization that aims to assist experts in discovering new drugs for treating infectious and neglected diseases using the latest technologies, making it convenient and time-saving. My long-term goal is to work on a project that improves people's lives. I have the vision to enhance my country and the world by coming up with new solutions to various problems. I would love to be a part of the team so that I can give back to the community. I'm eager to contribute to and learn from Ersilia!

Best, Tsion Zeleke

Tsion-Zeleke commented 4 months ago

Week 2

I selected Esilia's model eos30gr

After conducting some research on HERG, HERG Blockage, and the risks for the heart, I found the model that works on the classification of hERG blockers and non-blockers (eos30gr) to be interesting.

Intersting Facts about HERG

"HERG is a gene that affects how the heart beats, and studying it has led to important discoveries about heart rhythm disorders and medication safety."
Discovery: The HERG gene, linked to long QT syndrome, was uncovered in 1994. 🧬
Drug Development: Drugs targeting the hERG channel can inadvertently disrupt heart rhythms. ⚡️💊
Structural Insights: Understanding hERG's structure aids in developing safer drugs. 🔬💊
Cardiac Repolarization: hERG channel issues can disrupt heart muscle relaxation, causing arrhythmias. ❤️⚡️
Research Impact: HERG research has enhanced arrhythmia understanding, leading to new treatments. 🚀

Tsion-Zeleke commented 4 months ago

I decided to use codespace to run the selected model following below steps:
- I re-installed all the prerequisites on Codespace
  
  I really enjoyed using CodeSpace instead of my Ubuntu terminal. 🚀 Compared to last week's installation on Ubuntu, this week's installation prerequisites on CodeSpace were super fast and time-saving. ⏱
After Fetching the eos30gr Model, prediction for smiles keeps giving null as output
I create a GIT repo with the necessary files here

1- Testing using Ersilia Hub
- After running prediction using Ersilia hub. I got null output Here is the output log file
  2- Testing using docker
- I pulled the image using docker pull ersiliaos/eos4wt0:latest and ran but still got null as the output
- I tried to remove the image using docker system prune -a and pull the image again but still output is null
- I changed the smiles molecules and tested the model but output is still null
  3- Testing using python
- Based on the guidebook here. I imported ErsiliaModelclass in my Jupyter Notebook and run the model which still outputs a null
- The Jupyter notebook I used can be found here and the output for the notebook here
  4- Changing the python version and reinstalling the pre-request
  - I reinstalled Ersilia env with different python version listed below and tested the output. but the output is still null
```
conda create -n ersilia python=3.8 
conda create -n ersilia python=3.10
conda create -n ersilia python=3.11
```
    Finally, i tried to install the prerequisites inside the Ersilia env after activating it but issue persists test
    
    5 Additional debugging tried
- I tried to explore both the model repo found [here]() and the Ersilia repo that could be found [here]() I saw that some changes were made on the repo since last week
_I used `git reset --hard and reset back to the point of time where these changes were made but output is still null
I create conda env with the specific python versoin 3.10.10 but output is still null
I used pip install e instead on pip install e . but output is still null

Tsion-Zeleke commented 4 months ago

@DhanshreeA I keep getting null as output despite trying all the above method.

GemmaTuron commented 3 months ago

Hi @Tsion-Zeleke Sorry to hear you are still facing trouble. I cannot comment on the log as I need you to run and save them with the -v command when fetching the model, please could you run in your CLI:

ersilia -v fetch eos30gr > out.log 2>&1

And, in paralel: open a codespace from Ersilia's main repository (Ersilia is automatically installed there) and try to do the fecthing in CodeSpaces? Each github user has free Codespace time, so that should be fine for you to try. Remember to close the codespace once finished.

Hope this is helpful

Tsion-Zeleke commented 3 months ago

@GemmaTuron thank you for the comments!! I tried to run codespace from Ersilia main repo. but out put is still "null"

I fetch the model and saved the log file as mentioned here
The log file for serving the model can also be found here
I used the molecule "CCCC" and directly ran the model via ersilia -v api run -i "CCCC" output is null. out put is uploaded here
I tested it again this with 10 different smiles molecules using ersilia api run -i input.csv -o run_eos30gr_using_file.csv please find log file here with empty output for all 10 molecules

GemmaTuron commented 3 months ago

Does it happen with any other model on Ersilia? Did you run eos3b5e first to check the ersilia installation?

GemmaTuron commented 3 months ago

Please make sure you have ersilia properly installed, using a simple model as test (eos3b5e) as the model works fine for me simply following the standard Ersilia commands:

(ersilia) gturon@pujarnol:~$ ersilia run -i "CCCCC"
{
    "input": {
        "key": "OFBQJSOFQDEBGM-UHFFFAOYSA-N",
        "input": "CCCCC",
        "text": "CCCCC"
    },
    "output": {
        "outcome": [
            0.4752475247524752
        ]
    }
}

Tsion-Zeleke commented 3 months ago

@GemmaTuron Yes the other models work perfectly fine. Here is the output for the model eos3b5e and eos30gr that i just run

eos3b5e output


(ersilia) root@codespaces-ea4b15:/workspaces/ersilia# ersilia serve eos3b5e
🚀 Serving model eos3b5e: molecular-weight

URL: http://0.0.0.0:37991
PID: -1
SRV: pulled_docker

👉 To run model:

run

💁 Information:

info (ersilia) root@codespaces-ea4b15:/workspaces/ersilia# ersilia run -i "CCCCC" { "input": { "key": "OFBQJSOFQDEBGM-UHFFFAOYSA-N", "input": "CCCCC", "text": "CCCCC" }, "output": { "mw": 72.1510009765625 } } (ersilia) root@c

eos30gr output


(ersilia) root@codespaces-ea4b15:/workspaces/ersilia# ersilia serve eos30gr
🚀 Serving model eos30gr: deepherg

URL: http://0.0.0.0:59963
PID: -1
SRV: pulled_docker

👉 To run model:

run

💁 Information:

info (ersilia) root@codespaces-ea4b15:/workspaces/ersilia# ersilia run -i "CCCCC" { "input": { "key": "OFBQJSOFQDEBGM-UHFFFAOYSA-N", "input": "CCCCC", "text": "CCCCC" }, "output": { "outcome": [ null ] } }

Tsion-Zeleke commented 3 months ago

@GemmaTuron this and this slack conversation was mentioning that the model works fine for the ones who fetched earlier but other who fetched it later were having null output issue. could it be related to new commits made to the model repo

Or is there a way I could fetch the model rolling back to changes made to the repo a week ago

GemmaTuron commented 3 months ago

Hi @Tsion-Zeleke

The models are being regularly updated for maintenance. I'd suggest making sure you have the latest ersilia code pulled from the repository, and then the models should work fine

Tsion-Zeleke commented 3 months ago

@GemmaTuron thank you so much!!! I tried running the latest model as suggested using ersilia -v fetch eos30gr --from_github and it worked

(ersilia) root@codespaces-ea4b15:/workspaces/ersilia# ersilia run -i "CCCCC"
{
    "input": {
        "key": "OFBQJSOFQDEBGM-UHFFFAOYSA-N",
        "input": "CCCCC",
        "text": "CCCCC"
    },
    "output": {
        "outcome": [
            0.4752475247524752
        ]
    }
}

DhanshreeA commented 3 months ago

Hi @Tsion-Zeleke you can submit the final application.

Tsion-Zeleke commented 3 months ago

@DhanshreeA I encountered multiple issues when running the model and tried different methods to debug I've included the details of issues and debugging I performed here. I will process and submit the final application as your suggestion thanks

Tsion-Zeleke commented 3 months ago

Task 1 can be found here

Model Bias
- Data Processing
- Running Model -Evaluate Prediction -histograph -basic statistics mean,median... -Threshold Prediction using 0.5 as value -Frequency -Top 10 non HERG Blocker molecules -Top 20 non HERG Blocker molecules

Tsion-Zeleke commented 3 months ago

Task 2 can you found [here]()

Model Reproducibility here

Tsion-Zeleke commented 3 months ago

Task 3 can be found here

Model Validation
- In this notebook, I will try to get the performance of the model on an external dataset I have obtained from ChEMBL here
- The dataset contains Small Molecule Inhibitors of the Human hERG Channel Activity
- I'll Clean the dataset
- Add class to represent hERG blockers
- Run prediction for the same smile using eos30gr(the model implemented by ersilia)
- I'll Clean the dataset
- Add class to represent hERG blockers
- Run prediction for the same smile using eos30gr(the model implemented by ersilia)
- Compare results using several ML performance metrics

Tsion-Zeleke commented 3 months ago

@DhanshreeA I have submitted my final application to Outreachy. I thoroughly enjoyed working with the Ersilia Hub and am grateful I chose eos30gr, as it has taught me a lot about how models are implemented and the overall process. Thank you for the support; I have learned a lot over the past few weeks.

ersilia-os / ersilia