Pending things until publishing paper

[juanmirocks]

Sorted by member:

@juanmirocks

• [x] Create standard nala annotator, with options multiple_models on/off & we on/off
• [x] Create standard nala trainer, with options multiple_models on/off & elastic_net on/off & we on/off
• [x] Calculate value tables & draw progress performance on iterations by blocks of 5 for 3 classes: ST, SS, and NL --> Will update when the training pipeline is fixed. See here
• [x] Update & consolidate guidelines from IDP4, nala, and IDP4+
• [x] Clean: I have 589 docs on tagtog vs 591 in repo
• [x] Generate model trained on nala_training
• [x] Generate model trained on nala
• [x] Generate model trained on IDP4+_training
• [x] Generate model trained on IDP4+
• [x] Generate model trained on tmVar_training
• [x] Generate model trained on tmVar
• Cross compare performance with SE (Write P, R, F with SE values for 3 classes, ST, SS, NL):
  • Methods: MF, tmVar, SETH, nala, nala trained on tmVar_training
  • Corpora: tmVar_test, SETH, Variome120, nala_test
  • SETH runs:
  • [x] MF_test
  • [x] tmVar_test
  • [x] SETH
  • [x] Variome120
  • [x] nala_test
  • MFmodified runs:
  • [x] MF_test
  • [x] tmVar_test
  • [x] SETH
  • [x] Variome120
  • [x] nala_test
  • tmVar runs:
  • [ ] MF_test
  • [ ] tmVar_test
  • [x] SETH
  • [ ] Variome120
  • [x] nala_test
  • nala runs:
  • [x] MF_test
  • [x] tmVar_test
  • [x] SETH
  • [x] Variome120
  • [x] nala_test
  • nala trained on tmVar runs:
  • [x] tmVar_test (this one trained on _tmVartraining)
• [x] Compute SE in standard code
• [x] Write results.tsv with SE file for each method's predictions
• [ ] replace new mutationfinder corpus in our repository
• [ ] rewrite parser for MutationFinder parser -- See new corpus: https://github.com/abojchevski/nala/wiki/Analysis-of-Corpora
• [ ] Write in table stats & draw in pie for all corpora
• [ ] automatic classifier for annotation. To classify: 1) protein, DNA, or other, and 2) SNP, large substitution, insertion, deletion, frameshift, gene marker, rs number (dbSNP), or other
• [ ] Nothing in 3-Next
• [x] Nothing in 2-ThisWeek
• [ ] At the very end, test significance of word embeddings:
  • [ ] On nala test corpus, test word_embeddings only model
  • [ ] On nala test corpus, test hand_crafted_features only model
• Verify we recognize mentions:
  • [x] delPhe1388
  • [ ] deleted C1 domain This does not work, at least as absolute string, not with contextual info
  • [x] The _mutation found a deleted C1 domain_ in the protein found here
• [x] on/off keep genetic markers
• [x] on/off keep unnumbered mentions

  @carstenuhlig

• [x] @carstenuhlig corpus and parser for OSIRIS
• [ ] @carstenuhlig corpus and parser for SNP Corpus (check if it's the same as SETH's it's different) carsten couldn't do it
• [ ] @carstenuhlig corpus and parser for OMM corpus carsten couldn't do it
• [ ] @carstenuhlig draw Figure: Proportion of NL, SS, ST in considered corpora:
  • [ ] corpora: MF, tmVar, Var, Var120, SETH, OSIRIS, SNP Corpus, OMM
  • [ ] corpora: IDP4+, nala, IDP4, Var, Var120

    @abojchevski

• [x] @abojchevski Get # of NL not translated into ST for Var120 and Var120A ? --> https://onedrive.live.com/view.aspx?resid=858C9DCB1EB1AA7!2895&app=Excel&authkey=!AE1xLyNZCjAaBjs
• Last things to try out for training: https://github.com/abojchevski/nala/issues/47
• [x] Be able to run tmVar
• [x] Verify the output of tmVar in examples of Table ??? (1) in paper done by Juanmi

  Results

Results in OneDrive

[abojchevski]

@jmcejuela What do we mean again by Var120 and Var120A?

[juanmirocks]

Variome120 subcorpus, full and Abstract only. Will explain tmrw On Tue, 8 Mar 2016 at 17:52, Aleksandar Bojchevski notifications@github.com wrote:

@jmcejuela https://github.com/jmcejuela What do we mean again by Var120 and Var120A?

— Reply to this email directly or view it on GitHub https://github.com/abojchevski/nala/issues/46#issuecomment-193862794.

[juanmirocks]

@abojchevski I don't see committed code for how you did elastic net reg. ?

[abojchevski]

trainer.set_params({ 'c1': 1.0, # coefficient for L1 penalty 'c2': 1e-3, # coefficient for L2 penalty })

just add this in pycrfsuite in the train function

[juanmirocks]

@abojchevski awesome thanks

[juanmirocks]

@abojchevski I created in nala taggers.py and script/train.py -- I didn't fully test them because I didn't train so far. But besides possible minor bugs, that should do. Tomorrow I test it to calculate the performance on the various iterations

[juanmirocks]

Not relevant anymore