InvalidLinkBear: pip VCS URL parser

[jayvdb]

InvalidLinkBear should skip or understand git+https://github.com/jayvdb/coala-quickstart@gh_39, which is pip vcs support.

https://travis-ci.org/mamat-rahmat/GitGutter/builds/176631745#L241

.travis.yml
|   4| ••-•docker•run•--volume=$(pwd):/app•--workdir=/app•coala/base:pre•/bin/bash•-c•"pip3•install•'git+https://github.com/jayvdb/coala-quickstart@gh_39';•coala-quickstart•--ci;•coala-ci"
|    | [NORMAL] InvalidLinkBear:
|    | Broken link - unable to connect to https://github.com/jayvdb/coala-quickstart@gh_39 (HTTP Error: 404)

cc @mamat-rahmat

[sils]

it should at least cut off everything from the @ right?

[jayvdb]

ya, there are a few ways to handle this.

The simplest is to require that @ and other special chars used by pip vcs URLs are percent encoded; if they are found raw, they are assumed to be not part of the URL. This is likely to be in conflict with how URLs normally appear in files, as typically coders write percent_encode('https://.../...@...') and cutting at the @ will be an incorrect URL.

One valid approach is to detect that x+https://means that the tail end of the URL needs to be trimmed at @ and other pip vcs syntax characters, therefore allowing the link checker to verify only that the repo exists.

A better approach is to register a more advanced resolver of these schemes of URLs, so instead of using requests directly we use a different link checker, which will also allow ssh:// and other link types to be checked, and ensures that the branch/tag/sha component of the URL is also validated. (This will also give us the infrastructure to add a ftp link checker, which are not supported by requests; see https://github.com/coala/coala-bears/pull/948)

[jayvdb]

It would be good if https://github.com/coala/coala-bears/issues/1018 and https://github.com/coala/coala-bears/issues/1011 are merged first.