Handle multiple URLs naming the same page, differing by a suffix after comma.

PrzypisPowszechny / pp-client

7 stars 2 forks source link

Handle multiple URLs naming the same page, differing by a suffix after comma. #259

Open kaglowka opened 5 years ago

kaglowka commented 5 years ago

Both links below lead to the same article:

https://www.polskieradio.pl/13/53/Artykul/2175283,Sygnaly-Dnia-6-sierpnia-2018-roku-rozmowa-z-Andrzejem-Halickim https://www.polskieradio.pl/13/53/Artykul/2175283

(BTW also https://www.polskieradio.pl/13/53/Artykul/2175283 ...) However for only one for them an annotation is returned.

Possible solution:

and use heuristics such as "URL is the same except X" to approximate (mostly rightly) that the URL in question names the same article.

glowka commented 5 years ago

I am afraid it might be hard to solve it technically as there is unlimited number of patterns, so my first reaction is to work on consistency of data inserted by editorial team.