holmes-app / holmes-api

API for holmes website validation.
MIT License
17 stars 7 forks source link

Link Crawler when URL starts with "/" and sub-domain page #8

Closed heynemann closed 10 years ago

heynemann commented 10 years ago

If the crawler starts crawling a page and it's not in the root of a domain:

http://www.page.com/something/else.html

And it finds links like:

/other/test.html

It translates to http://www.page.com/something/other/test.html, when it should be http://www.page.com/other/test.html.