Support relative URLs in GitHub README/LICENSE files

[imphil]

GH recommends using relative URLs inside markdown files, i.e. for embedded images or links to other markdown files. We should make these relative URLs absolute when importing markdown files from GH.

1. detect if we import content from GH
2. Rewrite URLs to github static URLs

[recrsn]

We would need to add more repo crawlers, each specific to the service we are importing from. An added benefit is we could use service specific APIs and retrieve data more efficiently and easily without the need of cloning. Github API has nice methods for fetching the LICENSE and README files. Same is true for Gitlab and Bitbucket.

We might need to modify the new project page and add a source field. In the source, the user would specify the service which hosts the Git repo - Github, Bitbucket, Gitlab or generic git. The source may also be auto detected from the URL that the user enters. According to the source parameter, the appropriate repo crawler would run.

[wallento]

I would tend to think we should always be able to detect the hosting provider by the URL, or not?

An exception is probably Gitlab, where people may have their own server, we should maybe check if there is an efficient way to check for the API.

[recrsn]

I would tend to think we should always be able to detect the hosting provider by the URL, or not?

Probably for 95% of cases.

An exception is probably Gitlab, where people may have their own server, we should maybe check if there is an efficient way to check for the API.

There is Github enterprise too. (But I expect that would be the case in < 0.001% of projects)

What I would propose is let the user select his Git provider. We can help him with auto-detecting the choice from the URL he enters, if we can. And we would then validate the choice when the user submits the form. Github and Gitlab have APIs that allow for querying the existence of projects:

• https://docs.gitlab.com/ee/api/projects.html#get-single-project
• https://developer.github.com/v3/repos/#get

We can use this to validate the following:

• the user has entered the correct repository provider
• the entered repository has public access