natebosch
commented
4 years ago Thank you for the detailed report! Your analysis is spot on.
Closed kristoffer-zliide closed 4 years ago
natebosch
commented
4 years ago Thank you for the detailed report! Your analysis is spot on.
natebosch
commented
4 years ago Note that this doesn't in itself cause a problem to happen, what it does is mask the real problem and potentially thwart workarounds that would otherwise catch the ClientException.
I think the fix here is easy, it's a bit tougher to test. What I need is to be able to come up with a way to guarantee that the HttpResponse is a stream that emits an HttpException. @zichangg - do you know of a way to do that? I could probably mock things out entirely, but it would be nice if there was a specific http request that we could make, or response that the server could send, that would result in this exception on the HttpResponse...
zichangg
commented
4 years ago What I need is to be able to come up with a way to guarantee that the HttpResponse is a stream that emits an HttpException.
I'm not entirely sure I understand the problem. I guess what you mean is HttpClientResponse(which is simply a stream of List
I think current implementation does make sure only HttpException will be added to the stream.
natebosch
commented
4 years ago What I'm hoping for is some situation that I can hopefully set up with a server (in Dart) such that there would be an error on that Stream. Something like, perhaps, closing the stream early on the server side?
If it's not easy to write up a server in Dart that would cause an error on the response stream in a Dart client I can instead try the mocking approach.
zichangg
commented
4 years ago What I'm hoping for is some situation that I can hopefully set up with a server (in Dart) such that there would be an error on that Stream. Something like, perhaps, closing the stream early on the server side?
With the current setup, all HttpExceptions are generated from client side parser. I don't think server can send an "error" explicitly.
One way to do it is detachSocket(). The headers will be written and you can send some malformed data to trigger the error on the client side.
LeandroMoura3
commented
4 years ago I have this error when I put have a response very larger.
I need to pass 10 images from server to my app. If I don't put then in response, the request works. Else, I see this error message.
zichangg
commented
4 years ago @LeandroMoura3 This is suspicious. I don't think a large payload will lead to an error. Do you have a sample program to repro?
@natebosch I think the original problem should be fixed. What I mean is we need a better way to surface the underlying exception. With the current setup, all invalid exceptions will trigger this subtype error. But what users want is the real root cause. It should be changed to be more informative.
Is it a good idea to optional adding a type to ClientException if error is not a HttpException?
LeandroMoura3
commented
4 years ago @zichangg I think that you can reproduce that error very easy. In my case, works in the follow way: I have a db with registers of establishments. When I request this this querys with the base64 images, the error happens. When I request this querys without base64 images, then works fine.
You can convert a image to base64 image in the internet, is a big string. I convert it to a image with base64decode. I make this strategy because I wanna a faster developer for begin, but I think that not is the more performatic way to deal with images in web.
Edit: when I send a json array with 10 itens, don't works. When I send only 1 item, this works. Here is the error when fails:
#0 _invokeErrorHandler (dart:async/async_error.dart:18:23)
#1 _HandleErrorStream._handleError (dart:async/stream_pipe.dart:288:9)
#2 _ForwardingStreamSubscription._handleError (dart:async/stream_pipe.dart:170:13)
#3 _rootRunBinary (dart:async/zone.dart:1204:38)
#4 _CustomZone.runBinary (dart:async/zone.dart:1093:19)
#5 _CustomZone.runBinaryGuarded (dart:async/zone.dart:995:7)
#6 _BufferingStreamSubscription._sendError.sendError (dart:async/stream_impl.dart:358:15)
#7 _BufferingStreamSubscription._sendError (dart:async/stream_impl.dart:376:16)
#8 _BufferingStreamSubscription._addError (dart:async/stream_impl.dart:275:7)
#9 _ForwardingStreamSubscription._addError (dart:async/stream_pipe.dart:139:11)
#10 _addErrorWithReplacement (dart:async/stream_pipe.dart:190:8)
#11 _HandleErrorStream._handleError (dart:async/stream_pipe.dart:293:11)
#12 _ForwardingStreamSubscription._handleError (dart:async/stream_pipe.dart:170:13)
#13 _rootRunBinary (dart:async/zone.dart:1204:38)
#14 _CustomZone.runBinary (dart:async/zone.dart:1093:19)
#15 _CustomZone.runBinaryGuarded (dart:async/zone.dart:995:7)
#16 _BufferingStreamSubscription._sendError.sendError (dart:async/stream_impl.dart:358:15)
#17 _BufferingStreamSubscription._sendError (dart:async/stream_impl.dart:376:16)
#18 _BufferingStreamSubscription._addError (dart:async/stream_impl.dart:275:7)
#19 _SyncStreamControllerDispatch._sendError (dart:async/stream_controller.dart:783:19)
#20 _StreamController._addError (dart:async/stream_controller.dart:663:7)
#21 _StreamController.addError (dart:async/stream_controller.dart:615:5)
#22 _HttpParser._reportBodyError (dart:_http/http_parser.dart:1150:21)
#23 _HttpParser._onDone (dart:_http/http_parser.dart:862:9)
#24 _rootRun (dart:async/zone.dart:1180:38)
#25 _CustomZone.run (dart:async/zone.dart:1077:19)
#26 _CustomZone.runGuarded (dart:async/zone.dart:979:7)
#27 _BufferingStreamSubscription._sendDone.sendDone (dart:async/stream_impl.dart:392:13)
#28 _BufferingStreamSubscription._sendDone (dart:async/stream_impl.dart:402:15)
#29 _BufferingStreamSubscription._close (dart:async/stream_impl.dart:286:7)
#30 _SyncStreamControllerDispatch._sendDone (dart:async/stream_controller.dart:787:19)
#31 _StreamController._closeUnchecked (dart:async/stream_controller.dart:644:7)
#32 _StreamController.close (dart:async/stream_controller.dart:637:5)
#33 _Socket._onData (dart:io-patch/socket_patch.dart:1989:21)
#34 _rootRunUnary (dart:async/zone.dart:1196:13)
#35 _CustomZone.runUnary (dart:async/zone.dart:1085:19)
#36 _CustomZone.runUnaryGuarded (dart:async/zone.dart:987:7)
#37 _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
#38 _BufferingStreamSubscription._add (dart:async/stream_impl.dart:266:7)
#39 _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:779:19)
#40 _StreamController._add (dart:async/stream_controller.dart:655:7)
#41 _StreamController.add (dart:async/stream_controller.dart:597:5)
#42 new _RawSocket.<anonymous closure> (dart:io-patch/socket_patch.dart:1534:35)
#43 _NativeSocket.issueReadEvent.issue (dart:io-patch/socket_patch.dart:1012:18)
#44 _microtaskLoop (dart:async/schedule_microtask.dart:43:21)
#45 _startMicrotaskLoop (dart:async/schedule_microtask.dart:52:5)I think the original problem should be fixed. What I mean is we need a better way to surface the underlying exception.
This will be fixed by https://github.com/dart-lang/http/pull/429 - blocked by validation with a test.
zichangg
commented
4 years ago This is probably a duplicate with https://github.com/flutter/flutter/issues/58676
zichangg
commented
4 years ago @LeandroMoura3 do you know how large your json is? Our buffer size limit is 8K. I'm not super familiar with our secure socket implementation. But I guess your json exceeds the size limit.
LeandroMoura3
commented
4 years ago There is 149889 bytes.
This pass a lot of 8K, right? But sometimes works, sometime don't work. I put a looping for when the error ocorres, try again, until all images are loaded. There implies in more call to the server.
I use a local server. I think that the fast speed of answer and big content are the key for this bug.
I have a little hope that when I use a remote server, this bug don't happens.
PS: with one image have this size. Ten images are approximately ten times this.
zichangg
commented
4 years ago @LeandroMoura3 I'm spending time to understand that part of code. Are you able to get a sample program to repro? I'll assign myself first and come back to this once I have better understanding!
zichangg
commented
4 years ago I think the reason is because, ssl operations in dart:io is in non-blocking mode. Once the lowest level socket received "close" signal. It will trigger this onDone callback of the stream and the http listener stream will close. The data has been pushed to ssl filter before the "close" signal and has not been translated to application data.
This exception is thrown by http parser, which utilizes some internal states to find the http response has not fully received but onDone callback has fired.
I can't verify without a sample program.
I've seen openssl having a "ssl_has_pending()" to check whether there is unprocessed data. This will be helpful. We can add a check to close the stream until all incoming data has been processed. But looks like boringssl we are using doesn't have this API yet.
LeandroMoura3
commented
4 years ago Hello @zichangg
As I talk latter, it's a very simples bug to reproduce, in my opinion. But I will put here a example of a code that I use to connect to my API server.
Here is:
@override
Future<Uint8List> updateEstablishmentImage(int index) async {
String _apiUrl = _url + "image_establishment";
Map<String, String> _headers = {"Content-type": "application/json"};
String _body = """{
\"index\": $index
}""";
return await post(_apiUrl, headers: _headers, body: _body).then((response) {
if (response.statusCode == 200) {
Map<String, dynamic> jsonResponse = jsonDecode(response.body);
return Future<Uint8List>.value(base64Decode(jsonResponse['image']));
} else {
throw InternetException();
}
}).catchError((e, st) {
log(
'API error 8:',
error: e,
stackTrace: st,
);
throw PostException();
}).timeout(Duration(seconds: _timeOut), onTimeout: () => throw TimeoutException());
}Here is the log: `[log] API error 8: type '(HttpException) => Null' is not a subtype of type '(dynamic) => dynamic'
#1 _HandleErrorStream._handleError (dart:async/stream_pipe.dart:288:9)
#2 _ForwardingStreamSubscription._handleError (dart:async/stream_pipe.dart:170:13)
#3 _rootRunBinary (dart:async/zone.dart:1204:38)
#4 _CustomZone.runBinary (dart:async/zone.dart:1093:19)
#5 _CustomZone.runBinaryGuarded (dart:async/zone.dart:995:7)
#6 _BufferingStreamSubscription._sendError.sendError (dart:async/stream_impl.dart:358:15)
#7 _BufferingStreamSubscription._sendError (dart:async/stream_impl.dart:376:16)
#8 _BufferingStreamSubscription._addError (dart:async/stream_impl.dart:275:7)
#9 _ForwardingStreamSubscription._addError (dart:async/stream_pipe.dart:139:11)
#10 _addErrorWithReplacement (dart:async/stream_pipe.dart:190:8)
#11 _HandleErrorStream._handleError (dart:async/stream_pipe.dart:293:11)
#12 _ForwardingStreamSubscription._handleError (dart:async/stream_pipe.dart:170:13)
#13 _rootRunBinary (dart:async/zone.dart:1204:38)
#14 _CustomZone.runBinary (dart:async/zone.dart:1093:19)
#15 _CustomZone.runBinaryGuarded (dart:async/zone.dart:995:7)
#16 _BufferingStreamSubscription._sendError.sendError (dart:async/stream_impl.dart:358:15)
#17 _BufferingStreamSubscription._sendError (dart:async/stream_impl.dart:376:16)
#18 _BufferingStreamSubscription._addError (dart:async/stream_impl.dart:275:7)
#19 _SyncStreamControllerDispatch._sendError (dart:async/stream_controller.dart:783:19)
#20 _StreamController._addError (dart:async/stream_controller.dart:663:7)
#21 _StreamController.addError (dart:async/stream_controller.dart:615:5)
#22 _HttpParser._reportBodyError (dart:_http/http_parser.dart:1150:21)
#23 _HttpParser._onDone (dart:_http/http_parser.dart:862:9)
#24 _rootRun (dart:async/zone.dart:1180:38)
#25 _CustomZone.run (dart:async/zone.dart:1077:19)
#26 _CustomZone.runGuarded (dart:async/zone.dart:979:7)
#27 _BufferingStreamSubscription._sendDone.sendDone (dart:async/stream_impl.dart:392:13)
#28 _BufferingStreamSubscription._sendDone (dart:async/stream_impl.dart:402:15)
#29 _BufferingStreamSubscription._close (dart:async/stream_impl.dart:286:7)
#30 _SyncStreamControllerDispatch._sendDone (dart:async/stream_controller.dart:787:19)
#31 _StreamController._closeUnchecked (dart:async/stream_controller.dart:644:7)
#32 _StreamController.close (dart:async/stream_controller.dart:637:5)
#33 _Socket._onData (dart:io-patch/socket_patch.dart:1989:21)
#34 _rootRunUnary (dart:async/zone.dart:1196:13)
#35 _CustomZone.runUnary (dart:async/zone.dart:1085:19)
#36 _CustomZone.runUnaryGuarded (dart:async/zone.dart:987:7)
#37 _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
#38 _BufferingStreamSubscription._add (dart:async/stream_impl.dart:266:7)
#39 _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:779:19)
#40 _StreamController._add (dart:async/stream_controller.dart:655:7)
#41 _StreamController.add (dart:async/stream_controller.dart:597:5)
#42 new _RawSocket.<anonymous closure> (dart:io-patch/socket_patch.dart:1534:35)
#43 _NativeSocket.issueReadEvent.issue (dart:io-patch/socket_patch.dart:1012:18)
#44 _microtaskLoop (dart:async/schedule_microtask.dart:43:21)
#45 _startMicrotaskLoop (dart:async/schedule_microtask.dart:52:5)`
The server is local flask with mongoengine.
LeandroMoura3
commented
4 years ago LeandroMoura3
commented
4 years ago I put this call in a looping for a workaround.
Uint8List image;
bool error = true;
while (error) {
try {
image = await state.api.updateEstablishmentImage(index);
error = false;
} on Exception {}
}In my server, for ten imagens, a make 15~20 requests, and it works. All teen images are loaded. Anyway, this bug happens with a minor frequency in shorts call too, like Login Call API. The structure of call is the same that showed.
But, again, it is in local server.
kristoffer-zliide
commented
4 years ago I appreciate the TDD stance on this; however, we’re approaching two months for a simple one-line fix of a fairly serious bug that’s been there for eight months. How about just reverting https://github.com/dart-lang/http/commit/afcbd57e27676b42749e30446d5901fc97ef9e01 (which did not come with tests), and then reintroducing that commit along with a fix and a test that proves the bug is not there?
We occasionally see an error with the message "type '(HttpException) => Null' is not a subtype of type '(dynamic) => dynamic'" with the top of the stack
I think it's caused by https://github.com/dart-lang/http/blob/master/lib/src/io_client.dart#L49, which passes an error handler of wrong type.
The fact that the function handler is wrong can be seen by running
Changing it to
resolves the issue.
Possibly related: https://github.com/flutter/flutter/issues/50042.