Kmer association - Githubissues

PNNL-CompBio / Snekmer

Pipeline to apply encoded Kmer analysis to protein sequences

BSD 3-Clause "New" or "Revised" License

12 stars 1 forks source link

Kmer association #115

Closed jjacobson95 closed 9 months ago

jjacobson95 commented 11 months ago

Not ready for pull yet, but just wanted this to be on the radar.

Updates:

Learn output storage space reduced by 40-50%.
Apply output storage space reduced by ~95% when using new "save_apply_associations" parameter .
Small increases in speed and efficiency for both Learn and Apply.
Learn is set up to train on full length and is capable of evaluating confidence based on user defined fragment sizes now.
Code fully restructured to be object oriented. It is now much more readable, maintainable and cleaner for a future publication.
- Doc strings added to all functions. Overall, there should be much greater clarity for the purpose of each section of code.

Before merging pull request several things must still be done:

Confirm that the additive method for kmer count matrix generation is still working for Learn.
Update Documentation with new parameters.
Lint.

jjacobson95 commented 11 months ago

Failing on Model in the test. Is this a known issue? Looks like it may be unrelated to model but maybe just an issue with the test.

jjacobson95 commented 11 months ago

Hi @christinehc, just checking in on this. Do you know if this is a known or previous issue with model?

christinehc commented 11 months ago

Hmm, I took a look and didn't see anything obvious. I would try rerunning the tests and seeing if that works? If it fails again, I'll do some more digging

Edit: started the rerun a short while ago; we'll see how it goes

Edit 2: failed again, hmm. Not getting much clarity from the debug log itself but let me try a few things. Seems to be an issue with the actions workflow itself

christinehc commented 10 months ago

@jjacobson95: Still failing but I found this possibly related issue?

AKA try changing ubuntu-latest to ubuntu-18.04 in the actions.yml and see if that works

jjacobson95 commented 10 months ago

Ubuntu-18.04 failed - look like its no longer supported with github. Looks like only the latest (ubuntu-22.04) and ubuntu-20.04 are available - options are here. Currently testing ubuntu-20.04.

jjacobson95 commented 10 months ago

Note to all (#116) - ubuntu 20.04 works.
Note to me - Learn/Apply unit test needs updating as several new arguments need to be updated in the config file.

jjacobson95 commented 10 months ago

Ready to merge! @biodataganache

biodataganache commented 10 months ago

Couple of items:

Please add a script in the Snekmer/resources/tutorial/learnapp_tutorial_files/ folder that will run the learn and apply examples (see Snekmer/resources/tutorial/demo_example/ for the idea).
Please remove the base file in the Snekmer/resources/tutorial/learnapp_tutorial_files/learn/ folder from the repo (this should be generated by running the example)
Please fix the following error when snekmer learn is run: /Users/d3p620/lib/Snekmer/resources/tutorial/learnapp_tutorial_files/learn

(snekmer) d3p620@WE48427 learn % snekmer learn KeyError in line 914 of /Applications/anaconda3/envs/snekmer/lib/python3.10/site-packages/snekmer/rules/learn.smk: 'conf_weight_modifier' File "/Applications/anaconda3/envs/snekmer/lib/python3.10/site-packages/snekmer/rules/learn.smk", line 914, in

jjacobson95 commented 9 months ago

Changes made. But before merging, I should also update the docs to reflect parameter changes. I'll try to have this done by next Tuesday.

christinehc commented 9 months ago

Please also remember to update the version here before pushing