search

robinst / autolink-java

Java library to extract links (URLs, email addresses) from plain text; fast, small and smart
MIT License
207 stars 40 forks source link

Potentially misparsed URL #17

Closed silvanheller closed 6 years ago

silvanheller commented 6 years ago

Using: LinkExtractor.builder().linkTypes(setOf(LinkType.WWW, LinkType.URL)).build() and then extractor.extractLinks()

"man...http://i.imgur.com/rPRnI.jpg" is parsed as an URL, which seems wrong to me (not sure about the specs though)

robinst commented 6 years ago

Hey, thanks for the report. So, RFC 3986 allows "." in the scheme:

scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )

Having said that, the only use of "." I can find in IANA's registry is "z39.50". So maybe we could restrict it to only allow a single "." instead of multiple dots.

That would mean something like foo.http://i.imgur.com would still be recognized as a URL, but not your example.

What do you think?

silvanheller commented 6 years ago

Thanks for the reply! I see the problem with RFC 3986. In that case, I think in that case it's maybe better to leave this as documented behavior which programmers have to catch since I feel like Protocol adherence > convenience. Especially since your proposed solution does not fully fix the problem. What would solve it is an extractor which only looks for HTTP[s] schemes and strips everything else from the scheme, but i feel like that's outside the scope of this library. Funnily enough, twitter doesn't have that problem on their Conformance List

robinst commented 6 years ago

Sounds good! I've added a note to the README now, thanks.

(And yeah, Twitter's list is missing some interesting edge cases :).)