search

greenelab / snorkeling

Extracting biomedical relationships from literature with Snorkel 🏊
Other
58 stars 17 forks source link

Class Accuracy for Label Function Random Sampling Experiment #79

Closed danich1 closed 5 years ago

danich1 commented 5 years ago

This pull request is a follow up to the previous pr, where I aimed to see if label functions designed for one relation can predict another relation. Inside this pr, I plotted individual class accuracy (how well do these label functions predict the positive and negative classes). Main figures to observe: class_correct_dev_set.png and class_correct_test_set.png.

Feel free to take a look at the code if desired, but it all should relatively familiar in reference towards teh last pr. @ajlee21 I know you have prelims coming up. No worries if you don't have time.

danich1 commented 5 years ago

As discussed in person:

  1. The table here describes the label function matrix in terms of directionality (positive or negative: polarity), amount of sentences a label function covers (coverage), how often does one label function appear with other label functions (overlap) and how often does one label function output labels that conflict with another label function (conflict)
  2. In the curves the right hand side are the relations being predicted and the column titles are the different types of sources being used.
  3. I originally had that depiction; however, I believe the feed back was for me to have the baseline as a point rather than a line. Sticking with the current graph until further notice.
  4. Most of the curves have an inverse relationship. Just some of graphs have a small difference compared to others.