Optimised content fetch by removing redundant requests (closes #37)

[wilsonpage]

@arcturus r?

[arcturus]

Still believe the HEAD request will be a cleaner way of getting the final url. Could you give it a try @wilsonpage ? Implementing our own mechanism is a bit scary.

[wilsonpage]

HEAD will require us to make several requests per URL, no?

1. HEAD to check if the URL is a redirect
2. GET to fetch content when we find it's not a redirect URL

But perhaps I'm not following.

On Wed, 18 May 2016 19:18 Francisco Jordano, notifications@github.com wrote:

Still believe the HEAD request will be a cleaner way of getting the final url. Could you give it a try @wilsonpage https://github.com/wilsonpage ? Implementing our own mechanism is a bit scary.

— You are receiving this because you were mentioned.

Reply to this email directly or view it on GitHub https://github.com/mozilla-magnet/magnet-metadata-service/pull/38#issuecomment-220113576

[arcturus]

It will require just two. And we will get ride of the checkings, not saying will be faster, but easy to maintain.

[wilsonpage]

I don't understand why that would be easier to maintain. We'd be doubling the number of requests for each URL found.

On Wed, 18 May 2016 19:39 Francisco Jordano, notifications@github.com wrote:

It will require just two. And we will get ride of the checkings, not saying will be faster, but easy to maintain.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/mozilla-magnet/magnet-metadata-service/pull/38#issuecomment-220119566

[wilsonpage]

@arcturus comments addressed

[arcturus]

r+ SHIIIIIP IT!

Just left a couple of nits, but looking really good. Also tried from the command line and working fine.

Nice work @wilsonpage