search

snorkel-team / snorkel

A system for quickly generating training data with weak supervision
https://snorkel.org
Apache License 2.0
5.81k stars 857 forks source link

What is output of discriminative model? #1065

Closed wxlwang closed 5 years ago

wxlwang commented 5 years ago

Had hard time to understand the output of discriminative model. The discriminative model is the supervised learning. The input is the probability of label y1. What is output x1_1 and x1_2? Are h1_1, h1_2 and h1_3 hidden nodes? (image)

ajratner commented 5 years ago

Hi @wxlwang - in Snorkel we assume that we are given unlabeled data points $x_i$, and a set of labeling functions (LFs) that take in these unlabeled data points and output a label, or abstain. The generative model takes in these LF labels and returns a single (probabilistic) training label, $y_i$. Then, the discriminative model is just any supervised classifier that we train on the (now labeled) data points (x_i, y_i).

In this particular picture, $x_i$ is assumed to be a vector (in this case with two dimensions- just a toy figure :) ). E.g. for an image model, x_i could be a vector version of the raw pixels, or for text, an indicator representation of the word sequence.

Hope this helps!

wxlwang commented 5 years ago

Thanks for your quick response. But how does this help to label the data? We only know the probability of label still.

ajratner commented 5 years ago

Hi @wxlwang we use the probability as a "probabilistic" label, i.e. confidence weighted. If you want more detail, check out the blog posts and papers at snorkel.stanford.edu!

wxlwang commented 5 years ago

I am reading two papers of snorkel. But don't understand how this discriminative to improve the confidence of probabilistic label. Could you please shine some lights on it?

ajratner commented 5 years ago

Maybe check out #1059 ? I think that might get at your question!

wxlwang commented 5 years ago

@ajratner you mentioned at #1059 "There are three main advantages that you might find from using the predicted labels of the generative model as training labels for another model". Does that mean probability label y1 is the output of discriminate model not the input?