search

fake-news-detector / api

API for saving news flagging by the users
https://fake-news-detector-api.herokuapp.com/
8 stars 1 forks source link

Unshorten urls / Find by title #3

Closed rogeriochaves closed 6 years ago

rogeriochaves commented 6 years ago

Currently we use the news url as the identifier, so if more than one person publishes the same url already marked as fakenews, they will see the others flags.

But, this is very easy to trick this: just use a url shortener, or any kind of redirect, and bam, you have a "new" news.

That's why before checking the news we should first try to unshorten the url. I believe this can be done by opening just the url's header and check if there is a Location: something field, and follow the redirect until finding the real url.

Of course, javascript redirects could be used, but this won't happen so early.

Idk a way of doing this in Rust, I need help.

Or maybe, we could just try to find in our database the same title, this would avoid A LOT of extra work from the server. Of course one could change the title slightly, but we could search for very similars.

What do you think?

rogeriochaves commented 6 years ago

This issue was moved to fake-news-detector/fake-news-detector#19