search

tarides / river

A planet (feed aggregator) for OCaml.
ISC License
26 stars 5 forks source link

ocl-reporter issues: resolution failed #2

Closed avsm closed 9 years ago

avsm commented 9 years ago

Looks like it still occasionally fails to resolve your website's scheme:

`` Downloading http://kcsrk.info/atom-ocaml.xml ... make wFailed: resolution failed: no scheme Downloading http://amirchaudhry.com/tags/ocamllabs-atom.xml ... wwdone (cached). Downloading http://www.skjegstad.com/feeds/all.atom.xml ... done (cached). Downloading http://www.lpw25.net/atom.xml ... ^Cmake: *\ [run] Interrupt: 2

$ make run ocamlbuild -j 4 -use-ocamlfind -package omd,cohttp.lwt,core,cow,cow.syntax,lwt,netstring,river,sexplib.syntax,syndic,uri -tag thread,annot -syntax camlp4o lib/www.native Finished, 38 targets (38 cached) in 00:00:00. ./_build/lib/www.native writing : pages/people/index-b.html writing : pages/tasks/index-b.html writing : pages/tasks/platform-b.html writing : pages/tasks/compiler-b.html skipping content/mirage/nigori.md writing : pages/tasks/mirage-b.html writing : pages/tasks/outreach-b.html writing : pages/papers/index-b.html writing : pages/news/index-b.html writing : pages/news/atom.xml writing : pages/blogs/index-b.html writing : pages/blogs/atom.xml Downloading http://kcsrk.info/atom-ocaml.xml ... done (cached). Downloading http://amirchaudhry.com/tags/ocamllabs-atom.xml ... done.


The second time is cached.
kayceesrk commented 9 years ago

I've narrowed down the issue to HTTP redirection. Occasionally, fetching "http://kcsrk.info/atom-ocaml.xml" returns 302 and the new location points to "/atom-ocaml.xml". Resolution fails when this new location is fetched.

Not sure what the fix is; redirected URL is the same as the original one. URL fetching always seems to work the second time. Should I just fetch again and terminate after a fixed number of redirects? This happens with my blog and Amir's; both are hosted on github.

amirmc commented 9 years ago

Is this something we should report to GitHub? If this is related to fetching feeds, I wonder if it happens when ocaml.org builds its planet feed.

On 2 Apr 2015, at 11:34, KC Sivaramakrishnan notifications@github.com wrote:

I've narrowed down the issue to HTTP redirection. Occasionally, fetching "http://kcsrk.info/atom-ocaml.xml" returns 302 and the new location points to "/atom-ocaml.xml". Resolution fails when this new location is fetched.

Not sure what the fix is; redirected URL is the same as the original one. URL fetching always seems to work the second time. Should I just fetch again and terminate after a fixed number of redirects? This happens with my blog and Amir's; both are hosted on github.

� Reply to this email directly or view it on GitHub.

kayceesrk commented 9 years ago

Good idea. I'll find out if ocaml.org runs into the same issue.

kayceesrk commented 9 years ago

After some digging about here is what i found: http://www.rovrov.com/blog/2014/11/11/github-pages-302-redirect/.

Github does this on purpose for DDoS mitigation, and recommends repeating the request verbatim to the URI specified in the Location header field. This isn't quite right as the location header field returned by github is relative and not absolute:

$ curl -I kcsrk.info/atom-ocaml.xml
HTTP/1.1 302 Found
Connection: close
Pragma: no-cache
cache-control: no-cache
Location: /atom-ocaml.xml

Anyway, the desired behavior is to repeat the request. I've pushed a fix for this: https://github.com/kayceesrk/river/commit/80222cf8a8ba0beceeb2fbf0b1d52b0f72dff258