Some responses to URLs has incorrect Content-Length

[graymalkin]

e.g.

16:27 <+Elfy_Sheep> http://php.net/images/logo.php
16:27 < Gwyn> [image/png 0 B]

It should be 1406 B

This appears to be related to the type of request. Doing a HEAD request doesn't always yield a Content-Length, whereas doing a GET seems to always yield the header.

GET

curl --dump-header hdr http://php.net/images/logo.php > /dev/null && cat hdr

Does a full GET and correctly gives

...
Content-Length: 1406
...

HEAD

curl -I http://php.net/images/logo.php

Does a HEAD request, and doesn't have the Content-Length header:

HTTP/1.1 200 OK
Server: nginx/1.6.2
Date: Fri, 06 Mar 2015 16:32:04 GMT
Content-Type: image/png
Connection: keep-alive
X-Powered-By: PHP/5.6.5-1
Last-Modified: Fri, 06 Mar 2015 16:32:04 GMT
Expires: Sat, 07 Mar 2015 16:32:04 +0000

Proposed Change

I'll need to modify the URL module to do a GET request, and stop at the end of the header or at a given size (to prevent people getting the server to download a bunch of stuff)

wr.Timeout = 5000; // 5s timeout
wr.Method = "GET";
// This may need to become a stream reader.
using (WebResponse wresp = wr.GetResponse())
{
    // ...
}