antoniojrod commented 9 years ago

I created and deployed docker cloud application which works fine, except when I tried to add a facebook share link. Apparently the Facebook scraper uses an IPV6 address which is resulting in a 502 response from the dart server. Here's the log:

2a03:2880:2050:1ff7:face:b00c:0:1 - - [11/Nov/2014:09:50:09 -0800] "GET / HTTP/1.1" 502 63 - "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)" "www.*****.com" ms=48 cpu_ms=0 cpm_usd=0.000007 instance=1 app_engine_release=1.9.16

Is there any way to allow the appengine serve to handle these requests?

mkustermann commented 9 years ago

@antoniojrod Could you help us a bit and provide some more information:

You mention you use a "docker cloud application". Just to be sure: you are using the "gcloud preview app" commands with the this appengine package?
Could you SSH to the machine and look at the stdout of the dart process? [By running "docker ps" or "docker ps -a" and "docker logs CONTAINER_HASH" where CONTAINER_HASH is the hash of you application container]

There is an HTTP load balancer in front of the Compute Engine VM (SSL termination is also managed by AppEngine). This means that a tcp connection from the Facebook scrapper does not end at the Dart application directly, but only indirectly.

So I'm a bit puzzled where it could go wrong.

antoniojrod commented 9 years ago

Yes, I did in fact mean the 'gcloud preview app'
I did this and found an exception in the log that appears to correlate to a request by the facebook scraper.

2014-11-11 20:40:37.328: Got request: / Unhandled exception: Uncaught Error: HttpException: Content size below specified contentLength. 5622 bytes written but expected 524288., uri = /

0 _rootHandleUncaughtError. (dart:async/zone.dart:883)

dart-lang/labs#99 _asyncRunCallbackLoop (dart:async/schedule_microtask.dart:41) dart-lang/labs#100 _asyncRunCallback (dart:async/schedule_microtask.dart:48) dart-lang/labs#101 _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:84) dart-lang/labs#102 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:131)

mkustermann commented 9 years ago

Okay, this does not seem to be correlated with IPv6 at all.

This error indicates that a request handler is setting the 'content-length' to 524288 but only writes 5622 bytes into the response.

This narrows down the possibilities a lot. How do you serve the '/' URI? Do you use "assets.serve()" or do you generate content yourself?

antoniojrod commented 9 years ago

Interesting.

My server code is

VirtualDirectory virDir;

main() { runAppEngine((HttpRequest request) {

String rootPath = '../build/web';

virDir = new VirtualDirectory(rootPath)
    // The following are needed in dev mode to be able to access
    // Dart packages in the cache.
    //..directoryHandler = directoryHandler
    //..followLinks = true
    ..jailRoot = false;

String root = Platform.script.resolve(rootPath).toFilePath();

if (request.uri.path == '/') {
  virDir.serveFile(new File(root + "/index.html"), request);
}

}); }

So it seems pretty straight forward and works otherwise, but only fails with the Facebook scraper. So that IPV6 thing was the best I could come up with!

The live app is here (in case that's useful): http://lou-or-boo.appspot.com/

mkustermann commented 9 years ago

Okay, that limits the bug to: VirtualDirectory, package:appengine's HttpRequest wrapper or dart:io.

I just looked a bit into the VirtualDirectory implementation and it has some special casing for the "range" http header.

Would you do me another favor ? A dump of the HTTP headers the facebook scraper is setting would be interesting. Try printing something like: request.headers.forEach((String name, List values) { client.services.logging.info('Header: $name => ${values.join(', ')}'); // or print('Header: $name => ${values.join(', ')}'); });

That's the only thing I can think of right now.

antoniojrod commented 9 years ago

OK so here's what I got back:

13:52:39.219 502 63 B 35ms / 2a03:2880:2050:1ff6:face:b00c:0:1 - - [12/Nov/2014:10:52:39 -0800] "GET / HTTP/1.1" 502 63 - "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)" "www.louorboo.com" ms=35 cpu_ms=0 cpm_usd=0.000007 instance=1 app_engine_release=1.9.16 I 13:52:39.206 Header: user-agent => facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php) I 13:52:39.207 Header: x-appengine-user-id => I 13:52:39.207 Header: x-appengine-request-log-id => 5463ac7700ff02ce08edb2789b0001737e6c6f752d6f722d626f6f00016c6f752d6f722d626f6f000100 I 13:52:39.207 Header: x-appengine-city => ? I 13:52:39.207 Header: x-appengine-request-id-hash => EDB2789B I 13:52:39.207 Header: range => bytes=0-524287 I 13:52:39.207 Header: x-appengine-user-nickname => I 13:52:39.207 Header: x-appengine-user-ip => 2a03:2880:2050:1ff6:face:b00c:0:1 I 13:52:39.207 Header: x-appengine-https => off I 13:52:39.207 Header: x-appengine-country => US

sgjesse commented 9 years ago

Thanks for logging the headers. It is a bug in the http_server package. When there is a range header it will always set the length header in the response to the size of the range, no matter the size of the actual resource.

Opened https://code.google.com/p/dart/issues/detail?id=21587 on this.

Regards, Søren

On Wed, Nov 12, 2014 at 7:54 PM, antoniojrod notifications@github.com wrote:

OK so here's what I got back:

13:52:39.219 502 63 B 35ms / 2a03:2880:2050:1ff6:face:b00c:0:1 - - [12/Nov/2014:10:52:39 -0800] "GET / HTTP/1.1" 502 63 - "facebookexternalhit/1.1 (+ http://www.facebook.com/externalhit_uatext.php)" "www.louorboo.com" ms=35 cpu_ms=0 cpm_usd=0.000007 instance=1 app_engine_release=1.9.16 I 13:52:39.206 Header: user-agent => facebookexternalhit/1.1 (+ http://www.facebook.com/externalhit_uatext.php) I 13:52:39.207 Header: x-appengine-user-id => I 13:52:39.207 Header: x-appengine-request-log-id => 5463ac7700ff02ce08edb2789b0001737e6c6f752d6f722d626f6f00016c6f752d6f722d626f6f000100 I 13:52:39.207 Header: x-appengine-city => ? I 13:52:39.207 Header: x-appengine-request-id-hash => EDB2789B I 13:52:39.207 Header: range => bytes=0-524287 I 13:52:39.207 Header: x-appengine-user-nickname => I 13:52:39.207 Header: x-appengine-user-ip => 2a03:2880:2050:1ff6:face:b00c:0:1 I 13:52:39.207 Header: x-appengine-https => off I 13:52:39.207 Header: x-appengine-country => US

— Reply to this email directly or view it on GitHub https://github.com/dart-lang/labs/issues/112.

sgjesse commented 9 years ago

Proposed change https://codereview.chromium.org/721213002.

sgjesse commented 9 years ago

I have now uploaded a new version (0.9.4) of the http_server package which includes a fix for this range header bug.

Run 'pub update' to update the pubspec.lock to point to this new version.

dart-lang / labs

App engine dart server not handling IPV6 requests #112

0 _rootHandleUncaughtError. (dart:async/zone.dart:883)