search

ersilia-os / ersilia

The Ersilia Model Hub, a repository of AI/ML models for infectious and neglected disease research.
https://ersilia.io
GNU General Public License v3.0
200 stars 128 forks source link

[Internship Project]: Riley Pittman #715

Closed GemmaTuron closed 1 year ago

GemmaTuron commented 1 year ago

Summary

Hello,

This is a public issue for a virtual daily stand-up. We will use this to briefly share the tasks of the day and the challenges and advances made, so that we can ensure smooth support from the Ersilia mentors and alignment between daily tasks and overall internship goals.

Scope

Initiative 🐋

Objective(s)

Internship goals:

Team

Role & Responsibility Username(s)
Intern @pittmanriley
Mentor @miquelduranfrigola
Coordinator @GemmaTuron

Timeline

Before starting your work, line up a few tasks and short description. This should not take long. For example, it could be something like: Wednesday 21st June

Documentation

No response

GemmaTuron commented 1 year ago

In this example, following the tasks of Wednesday 21st June:

I have created all the templates for the Interns, spend a couple of hours revising the GitHub Project and updating the tasks. I have been working on identifying the bug in the GitActions, solved in this issue. I have set a meeting with @miquelduranfrigola to discuss in detail his comments on the Model testing discussion but I haven't been able to start writing yet, i am to do so by the end of the week. Looking forward to the WCAIR next lesson!

Pro tip: adding the links to the issues and discussions you mention will be very helpful!

pittmanriley commented 1 year ago

Wednesday June 21

GemmaTuron commented 1 year ago

Hi @pittmanriley

Great, if you could briefly summarize the issue with BentoMl that you identified and how you solved it with Miquel it would be great!

pittmanriley commented 1 year ago

Of course. We discussed this in our meeting, but the main issue is that during the installation process, the current code only checks to see if we have bentoml downloaded, but it needs to check if we have the correct version of it downloaded. In my case, for some reason I already had bentoml downloaded on my computer, but it was a version incompatible with the Ersilia software. So, it needs to be updated so that it checks the version (or if it's not installed), and if it's not the correct version, then update it to match the one we need (version 0.11)

The code I'm referring to is the is_installed() method in the BentoMLRequirement class in setup.py.

pittmanriley commented 1 year ago

Thursday June 22:

pittmanriley commented 1 year ago

Friday June 23:

GemmaTuron commented 1 year ago

Thanks @pittmanriley Is there a problem in the fetching of eos96ia? is it due to the mac compatibility issues still?

pittmanriley commented 1 year ago

Hi @GemmaTuron

I’m not sure if the issue is related to mac compatibility. I keep getting an error saying that the model couldn’t produce an output. I tried looking at the troubleshooting webpage, but couldn’t make it work. I will try to work on it further and post an issue if I cannot get it.

GemmaTuron commented 1 year ago

Hi @pittmanriley

To debug a model, you need to:

  1. Clone to your local system
  2. Use the repo_path flag to try the fetching from local
  3. Use the verbose option so you see the whole log, collect it and explain in the relevant issue what have you found in the log files that could point you to the error -- the issue is empty now so we can't provide more support: https://github.com/ersilia-os/eos96ia/issues/1

All the above steps are explained in the troubleshooting page in case you need more info. Also remember you can use the slack channel to ask for help if you are unsure about the next steps to take.

pittmanriley commented 1 year ago

Monday June 26:

pittmanriley commented 1 year ago

Tuesday June 27:

I had a really tough time getting things to work properly today. I was unable to figure out why eos96ia is producing an empty output, and eos7ack is still not working with eml_canonical.csv. In addition, I received an error when trying to fetch eos2re5 and eos8d8a, which I will look into deeper tomorrow.

pittmanriley commented 1 year ago

Wednesday June 28:

I'm still having issues with eos2re5 and eos96ia, which I spent the majority of the day looking at. I was able to work on and complete the testing for eos8d8a and eos2hbd, however. I have all of the edits completed with eos96ia, but I don't think I should submit a PR until I fix this issue.

GemmaTuron commented 1 year ago

Hi @pittmanriley !

Thanks for the summary, good progress. Do not submit a PR until the model works, but do add more information on the issue so I can provide further help! Remember to attach the log files of the errors you get so we can have a more active discussion and you don't get lost! Let's aim at closing models eos96ia and eos8d8a before moving on

pittmanriley commented 1 year ago

Thursday June 29:

I made some progress on eos96ia, but I'm still having issues getting the troubleshooting to work. I wrote about those problems in detail in the issues section of the model. As for eos8d8a, I have mostly refactored the model, but I need to find a way to adjust the service.py file to work with the new main.py file, which was made by combining the two .py files in the framework folder.

GemmaTuron commented 1 year ago

@pittmanriley thanks, I've requested Simran's help for eos96ia, see if you could summarise to her the main issues! I've given a few pointers for eos8d8a, which is a complex model, have a look and think how we could structure this repo

pittmanriley commented 1 year ago

Friday June 30:

I was able to fix all my issues on eos96ia today with Miquel using GitHub code spaces. Since the model wouldn't fetch properly on my computer, running and editing in codespaces worked great. I submitted the PR for that model and I believe Miquel accepted it. Additionally, Miquel advised me to only make minimal changes on eos8d8a since they are more complex, and the model runs smoothly without the changes. I talk about this more in the model's issues section. Lastly, I was able to refactor eos2mrz, and I just need to test it a little bit more before I'm ready to submit the PR. Today there were many breakthroughs!

pittmanriley commented 1 year ago

Sunday July 2:

I was able to finish refactoring this model today, and I have finished the assigned tasks from last week (assuming I can submit the PR for eos8d8a without refactoring the whole model).

pittmanriley commented 1 year ago

Monday July 3:

I wasn't able to work very much today, but I spent much of my time trying to understand the issues with eos7ack and how to properly adjust the .gittatributes files. I think I'm ready to re-submit the PR for eos2mrz, but I want to make sure before I do so (I posted a message on Slack and am waiting on that). With eos7ack, the eml_canonical file is the only file that the model has trouble with, and it's still causing an idex error. I talk more about this under the issues. Additionally, after looking at the eos7ack files that I refactored, I think I may have incorrectly adjusted the .gitattributes file, so I'm looking into this as well.

GemmaTuron commented 1 year ago

Hi @pittmanriley

We'll discuss in the meeting today about the indexing error, for the rest of the tasks all looks good, open PR's when ready and confirm the model testing works. Once these open issues are complete I'll assign new models

pittmanriley commented 1 year ago

Tuesday July 4:

GemmaTuron commented 1 year ago

Hi @pittmanriley

Hellen amended model eos7asg so its ready for testing. You already have two open models so won't assign new models for the moment, let me know if you finish them! Also let's try to elucidate what is going on on model eos7ack, remember to add print statements to check

pittmanriley commented 1 year ago

Wednesday July 5:

I had troubles getting the eos2lm8 model to work today. With some of the inputs, I'm getting an index error, which I described more in the issues. I refactored the model, but need to find the cause of the issue before I can test out my adjustments. I will try to troubleshoot this more tomorrow, and hopefully get to refactoring eos78ao and testing eos7qga.

GemmaTuron commented 1 year ago

Hi @pittmanriley

Let's focus on the index errors - we need to see where the models are failing, which can be done with the -v command and adding print statements in the different functions.

pittmanriley commented 1 year ago

Thursday July 6:

I've been trying to figure out the index issue with eos2lm8 with Febie, and we are having issues with getting the print statements to show up. I sent a message to the slack channel about this. I wasn't able to work a full day today, so I'll spend a lot more time on this tomorrow and this weekend if necessary. I've also started on eos78ao but need to finish this as well.

GemmaTuron commented 1 year ago

Thanks for the update @pittmanriley, let's focus on solving the eos2lm8 and eos78ao by the end of the week if possible. I've answered on Slack as well

pittmanriley commented 1 year ago

Friday July 7:

I spent a lot of time troubleshooting eos2lm8 on Friday and over the weekend. I'm sort of stuck on where I should go - I sent a message on Slack about this model, but I'm unable to test the model after making adjustments to the code because I can only run on Codespaces, but when I try to fetch using --repo_path, an error occurs. I tried spending more time on the test project for now. For eos78ao, I'm unable to fetch in CLI and Codespaces for now, so I'm stuck on this as well.

miquelduranfrigola commented 1 year ago

Thanks @pittmanriley

@GemmaTuron - I was looking into eos2lm8. There is an open PR from Ife https://github.com/ersilia-os/eos2lm8/pull/3 (only the README file)

@pittmanriley - let's discuss over Slack and then reflect the relevant points here about eos2lm8. Also, thanks for pushing the "test project" 👌

pittmanriley commented 1 year ago

Monday July 10:

With eos2lm8, I'm still a bit stuck, but I saw that Miquel mentioned there is a pull request pending. For eos78ao, I made some progress because I was able to get it to run by running run.sh in Codespaces, but I'm just not sure if I still need to get it to fetch normally before refactoring (I discuss this in the issues). With eos2mrz, I'm also stuck because the model is working on my system, and it's unclear to me what the error about updating the metadata to Airtable means. Febie and I also made progress on eos7ack today by finding which SMILES seem to cause issues, but we don't find consistent results when we test on this model, so we need Gemma and Miquel's help on this one. Tomorrow, I need to test eos7nno and get started on coding for the test command.

pittmanriley commented 1 year ago

Tuesday July 11:

I was able to get eos78ao to work on Codespaces and I submitted a PR, but the actions are failing due to a 'Nonetype object is not iterable' error. I wrote about this in the issues, but I'm not sure what I can do since it is working in Codespaces. Also, I am still unable to get eos2lm8 to fetch using --repo_path in both CLI and codespaces. I've updated the file the way Gemma recommended, but I can't test that it works because I can't fetch from my local path. I would like to spend most of the day tomorrow working on the test command project so that I can ask questions on Thursday's internship meeting.

GemmaTuron commented 1 year ago

Hi @pittmanriley

Good progress, let's finish the open models before focusing on the test command. Please:

pittmanriley commented 1 year ago

Wednesday July 12:

I tried to get a better understanding for the error happening with eos2lm8, and while I understand the error now, I don't know how to fix it. I talk about this in the issues. For eos7ack, we saw during the meeting on Tuesday that SwissADME blocked some of us, so we need to contact them to get help using their model. I'm unable to go to the SwissADME website and run any SMILES inputs.

pittmanriley commented 1 year ago

Thursday July 13:

GemmaTuron commented 1 year ago

Hi @pittmanriley !

I've merged the PR on eos2lm8, let's see if the tests are passed. I won't assign new models because you are onto the test command work, but make sure to complete model testing if any is assigned to you

pittmanriley commented 1 year ago

Sounds good!

Friday July 14:

pittmanriley commented 1 year ago

Monday July 17:

Today, I tested eos6tg8, and had some issues fetching it in both my terminal and on Codespaces, which I wrote about in the issues. It worked in Colab and Docker, though. I also responded to the message about eos7ack, but it's unclear to me why Gemma is unable to fetch. With the test command, I added functionality that makes it so instead of only testing one smiles input, it generates an example input with 5 smiles and prints the results to those inputs. I also started adding the functionality that allows a user to enter an output file, but I still need to complete this. Now that I have a good baseline, I want to start planning more detailed additions to the code and ask other interns if they'd like to help.

GemmaTuron commented 1 year ago

@pittmanriley Please also summarize the work you are doing on the test module, and I see you also need to work on model eos7ack and eos2lm8. Please prioritize closing off the models before working on the test module, and also run any model testing assigned to you.

pittmanriley commented 1 year ago

@GemmaTuron, of course. I believe the PR for eos2lm8 has already been accepted, and I'll respond to the eos7ack issues.

With the test module, I had some trouble last week understanding how to incorporate the run functionality into the ModelTester class. I needed to spend a day or so to get a better understanding of all the classes and how they work together. Over the weekend I spent some more time on it and got it to test with a single output. It works for now, but I would like to go over the organization of the code with you and Miquel sometime this week. As of now, the command just tests a single smiles string, and has very basic error handling, but it works. I've pushed the code to my repository if you'd like to see. For my next steps, I'd like to:

pittmanriley commented 1 year ago

Tuesday July 18:

I worked on eos96ia for a while, and I'm almost ready to do a PR - I just need to specify the correct version of rdkit. I also worked on eos6tg8, but I saw that Simran was able to test the model so that is all good now. Eos5smc is almost done, but I'm having some difficulties with this model. I wrote it in the issues. I'm almost done with the input/output functionality for the test module.

pittmanriley commented 1 year ago

Wednesday July 19:

Today, I spent a lot of time on the test module because I was unable to make any progress on eos5smc. I'm still having the same issues, which I mentioned in the issues. I'm going to have Simran try to fetch it for me and see if it is a Mac issue.

With the test module, I spent some time cleaning up the code and working with Febie to test the code for inconsistencies. I also added a new exception class that raises an error when someone tries to test a model without fetching it first. I tried doing this by accident once, and it gave me an error that could be difficult for users to understand, so I made it more clear. I need to keep working on the output functionality, since each time I run the model on new inputs, it just overwrites the previous output in the output file instead of including all of them from that test. I also need to figure out how to run the run.sh within the test module.

pittmanriley commented 1 year ago

Thursday July 20:

I'm having a lot of trouble getting eos5smc to fetch, before and after refactoring. As for the test module, I think it's going quite well and I would like to submit a PR soon so that people can use it. I plan to speak with Miquel about how it's been going, and also hopefully get some advice on eos5smc.

pittmanriley commented 1 year ago

Friday July 21:

pittmanriley commented 1 year ago

Monday July 24:

I am not getting anywhere with eos5smc and eos96ia, and hopefully I can get some direction with Miquel tomorrow. I think that the new update to GitHub regarding python 3.11 is affecting the eos96ia PR. For the test module, I got much further today with Febie's help, but we aren't quite finished with the run.sh part yet. We're getting some errors that are taking some time to understand.

GemmaTuron commented 1 year ago

Hi @pittmanriley

I've provided some more info on eos5smc, the error logs are quite clear as to what dependencies are failing. I've also assigned you a new model to work on once you are done with the current tasks.

pittmanriley commented 1 year ago

Tuesday July 25:

While I was able to get the dependency issues solved with eos5smc, I'm not sure what the new error I'm getting means and how to fix it. For the test module, I was able to do a good amount today, but I still need to get some debugging done before the output issues are fixed, and getting it to run the run.sh file is a little bit difficult. Febie is helping with this but this may take a little bit more time.

pittmanriley commented 1 year ago

Wednesday July 26:

pittmanriley commented 1 year ago

Thursday July 27:

pittmanriley commented 1 year ago

Friday July 28:

I spent a good amount of time fixing the output issues for the test module, and it works for each of the parts of the test module that are currently in use (model info check, single input test, and example input test). Now, I need to fix the consistency check, and then I can get the output working for that part as well. I'm also pretty stuck with eos96ia, since I'm not sure what the errors that Simran and I are getting mean. I wrote about that and provided the error logs in the issues.

pittmanriley commented 1 year ago

Monday July 31:

I was able to get all of my models working today. I adjusted the workflows and checked to make sure that they were fetching, and submitted the PRs. eos96ia and eos5smc gave some issues at first, but they ended up passing. Now, I can take on more models if needed or put more time into the test module.

GemmaTuron commented 1 year ago

Hi @pittmanriley

Great work, thanks! I have merged your PR's. There are a few models assigned to you for testing. I'd say focus on the test module to make it complete, but let's discuss in today's meeting about this!

pittmanriley commented 1 year ago

Tuesday August 11: