What happens when a spoiler shows up in the youtube comments and takes up a single line? That is a pretty different problem from when an entire article might spoil a show. Yet sometimes, these can be the most common and the most devastating.
Since these comments take up only a single div or span, it may be possible to censor them without censoring the entire page. Plus, a false negative wouldn't be so bad, as you are only censoring one part of a page, which is less annoying than canceling a single page over a single spoiler.
Not only that, but in terms of time complexity, comparing a massive page to the model might take longer than a small string. Especially considering you have to scan through the entire page anyways.
Also, censoring individual spans might be easier to train, considering most big shows on reddit have subreddits where individual episode posts could contain 10,000 posts, 5000 of which are single lines.
Plus, if you really think about when people are actually being spoiled by stuff it isn't in these big articles, its mostly on facebook, twitter and reddit, when a single sentence usually contains the most relevant/spoilery details. An example would be in episode 5 of season 6 of game of thrones, the character Hodor dies. A facebook could contain just the phrase "hodor!!!!" and you might infer the tragic loss.
So the first thing to do would be to create a model which takes in a single span's worth of content, and checks it against the model, making a list of the top 10 episodes it thinks it belongs to, and if any of the 10 belong to the list of episodes, seasons, or shows that the user defined, the span will be censored.
What happens when a spoiler shows up in the youtube comments and takes up a single line? That is a pretty different problem from when an entire article might spoil a show. Yet sometimes, these can be the most common and the most devastating.
Since these comments take up only a single div or span, it may be possible to censor them without censoring the entire page. Plus, a false negative wouldn't be so bad, as you are only censoring one part of a page, which is less annoying than canceling a single page over a single spoiler.
Not only that, but in terms of time complexity, comparing a massive page to the model might take longer than a small string. Especially considering you have to scan through the entire page anyways.
Also, censoring individual spans might be easier to train, considering most big shows on reddit have subreddits where individual episode posts could contain 10,000 posts, 5000 of which are single lines.
Plus, if you really think about when people are actually being spoiled by stuff it isn't in these big articles, its mostly on facebook, twitter and reddit, when a single sentence usually contains the most relevant/spoilery details. An example would be in episode 5 of season 6 of game of thrones, the character Hodor dies. A facebook could contain just the phrase "hodor!!!!" and you might infer the tragic loss.
So the first thing to do would be to create a model which takes in a single span's worth of content, and checks it against the model, making a list of the top 10 episodes it thinks it belongs to, and if any of the 10 belong to the list of episodes, seasons, or shows that the user defined, the span will be censored.