WorkerException: TypeError: Instance of 'JsLinkedHashMap<dynamic, dynamic>': type 'JsLinkedHashMap<dynamic, dynamic>' is not a subtype of type 'Map<String, dynamic>'

d-markey / squadron

Multithreading and worker thread pool for Dart / Flutter, to offload CPU-bound and heavy I/O tasks to Isolate or Web Worker threads.

https://pub.dev/packages/squadron

MIT License

79 stars 0 forks source link

WorkerException: TypeError: Instance of 'JsLinkedHashMap<dynamic, dynamic>': type 'JsLinkedHashMap<dynamic, dynamic>' is not a subtype of type 'Map<String, dynamic>' #5

Closed SaadArdati closed 2 years ago

SaadArdati commented 2 years ago

The issues just don't stop do they?

So how do we fix this one?

WorkerException: TypeError: Instance of 'JsLinkedHashMap<dynamic, dynamic>': type 'JsLinkedHashMap<dynamic, dynamic>' is not a subtype of type 'Map<String, dynamic>'
TypeError: Instance of 'JsLinkedHashMap<dynamic, dynamic>': type 'JsLinkedHashMap<dynamic, dynamic>' is not a subtype of type 'Map<String, dynamic>'
    at Object.wrapException (http://localhost:5000/solver_worker.dart.js:762:17)
    at Object._failedAsCheck (http://localhost:5000/solver_worker.dart.js:1931:15)
    at Rti._generalAsCheckImplementation [as _as] (http://localhost:5000/solver_worker.dart.js:1920:9)
    at Object.LinkedHashSet_LinkedHashSet$from (http://localhost:5000/solver_worker.dart.js:3775:28)
    at PositionManagerThread.performLayout$6 (http://localhost:5000/solver_worker.dart.js:16322:28)
    at SolverServiceImpl_operations_closure.call$1 (http://localhost:5000/solver_worker.dart.js:16141:64)
    at http://localhost:5000/solver_worker.dart.js:8415:27
    at _wrapJsFunctionForAsync_closure.$protected (http://localhost:5000/solver_worker.dart.js:3111:15)
    at _wrapJsFunctionForAsync_closure.call$2 (http://localhost:5000/solver_worker.dart.js:10874:12)
    at Object._asyncStartSync (http://localhost:5000/solver_worker.dart.js:3075:20)
solver_worker.dart.js 762:17                                                  wrapException
solver_worker.dart.js 1931:15                                                 _failedAsCheck
solver_worker.dart.js 1920:9                                                  _generalAsCheckImplementation
solver_worker.dart.js 3775:28                                                 LinkedHashSet_LinkedHashSet$from
solver_worker.dart.js 16322:28                                                performLayout$6
solver_worker.dart.js 16141:64                                                call$1
solver_worker.dart.js 8415:27                                                 <fn>
solver_worker.dart.js 3111:15                                                 $protected
solver_worker.dart.js 10874:12                                                call$2
solver_worker.dart.js 3075:20                                                 _asyncStartSync
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 251:49  throw_
packages/squadron/src/worker.dart 134:7                                       send
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 60:31            <fn>
dart-sdk/lib/async/zone.dart 1450:47                                          _rootRunBinary
dart-sdk/lib/async/zone.dart 1342:19                                          runBinary
dart-sdk/lib/async/future_impl.dart 174:22                                    handleError
dart-sdk/lib/async/future_impl.dart 778:46                                    handleError
dart-sdk/lib/async/future_impl.dart 799:13                                    _propagateToListeners
dart-sdk/lib/async/future_impl.dart 609:5                                     [_completeError]
dart-sdk/lib/async/future_impl.dart 665:7                                     <fn>
dart-sdk/lib/async/zone.dart 1426:13                                          _rootRun
dart-sdk/lib/async/zone.dart 1328:19                                          run
dart-sdk/lib/async/zone.dart 1236:7                                           runGuarded
dart-sdk/lib/async/zone.dart 1276:23                                          callback
dart-sdk/lib/async/schedule_microtask.dart 40:11                              _microtaskLoop
dart-sdk/lib/async/schedule_microtask.dart 49:5                               _startMicrotaskLoop
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 166:15           <fn>

For reference, this is what the performLayout function does:

and the solver impl is unchanged

One theory is that .toJson() is creating a Map<String, dynamic> and js/dart simply don't know how to strip its type to a Map, so I tried to convert map using your function:

Unfortunately, this does not resolve the issue. It's the same error

WorkerException: TypeError: Instance of 'JsLinkedHashMap<dynamic, dynamic>': type 'JsLinkedHashMap<dynamic, dynamic>' is not a subtype of type 'Map<String, dynamic>'
TypeError: Instance of 'JsLinkedHashMap<dynamic, dynamic>': type 'JsLinkedHashMap<dynamic, dynamic>' is not a subtype of type 'Map<String, dynamic>'

And looking at the js stacktrace and the generated js file, It seems it is throwing at this line:

Which is the equivalent of this:

The desired type is

    Set<Map<String, dynamic>> nodeReferences,

SaadArdati commented 2 years ago

Also getting this at the end of the stack trace, now that I added the new Squadron logger:

d-markey commented 2 years ago

So the code throws on:

final Set<Map<String, dynamic>> nodeReferences = Set<Map<String, dynamic>>.from(dynNodeReferences);

It cannot work because per Set<E>.from documentation, dynNodeReferences must be an iterable "where all elements should be instances of E", so instances of Map<String, dynamic> in your case.

But statically speaking dynNodeReferences is now just a Set = Set<dynamic> even if you know that items in the set are Map<String, dynamic> since they come from your main app.

So you have to rebuild proper type for the map entries first (because your keys are String), and then rebuid the Set. Something like:

final nodeReferences = Set<Map<String, dynamic>>.from(dynNodeReferences.map(
      (refs) => refs.map(
            (key, value) => MapEntry<String, dynamic>(key, value))));

Unfortunately there's not much I can do about this :-( Types in Isolates may be preserved (because Squadron relies on Isolate.spawn() in which case objects are copied and -- I suppose -- types are preserved) but types in Web Workers (JavaScript) are mostly lost or somehow "diminished" during communication and a simple cast is not enough to restore strong types.

SaadArdati commented 2 years ago

I would have never had guessed I needed to be this verbose with my types but it makes total sense! Unfortunately, it's still throwing an error :(

WorkerException: TypeError: Instance of 'JsLinkedHashMap<dynamic, dynamic>': type 'JsLinkedHashMap<dynamic, dynamic>' is not a subtype of type 'Map<String, dynamic>'
TypeError: Instance of 'JsLinkedHashMap<dynamic, dynamic>': type 'JsLinkedHashMap<dynamic, dynamic>' is not a subtype of type 'Map<String, dynamic>'
    at Object.wrapException (http://localhost:5000/solver_worker.dart.js:762:17)
    at Object._failedAsCheck (http://localhost:5000/solver_worker.dart.js:1931:15)
    at Rti._generalAsCheckImplementation [as _as] (http://localhost:5000/solver_worker.dart.js:1920:9)
    at Object.LinkedHashSet_LinkedHashSet$from (http://localhost:5000/solver_worker.dart.js:3837:28)
    at PositionManagerThread.performLayout$6 (http://localhost:5000/solver_worker.dart.js:16899:28)

Same error.

SaadArdati commented 2 years ago

Tried to replace <String, dynamic> with <dynamic, dynamic> and moved the other Set.from functions before to make sure those are at least passing.

No dice.

WorkerException: TypeError: Instance of 'JsLinkedHashMap<dynamic, dynamic>': type 'JsLinkedHashMap<dynamic, dynamic>' is not a subtype of type 'Map<String, dynamic>'
TypeError: Instance of 'JsLinkedHashMap<dynamic, dynamic>': type 'JsLinkedHashMap<dynamic, dynamic>' is not a subtype of type 'Map<String, dynamic>'
    at Object.wrapException (http://localhost:5000/solver_worker.dart.js:762:17)
    at Object._failedAsCheck (http://localhost:5000/solver_worker.dart.js:1931:15)
    at Rti._generalAsCheckImplementation [as _as] (http://localhost:5000/solver_worker.dart.js:1920:9)
    at Object._$CanvasNodeFromJson (http://localhost:5000/solver_worker.dart.js:6204:37)
    at NodeJsonConverter.fromJson$1 (http://localhost:5000/solver_worker.dart.js:18384:20)
    at PositionManagerThread.performLayout$6 (http://localhost:5000/solver_worker.dart.js:16911:19)
    at http://localhost:5000/solver_worker.dart.js:16678:93
    at _wrapJsFunctionForAsync_closure.$protected (http://localhost:5000/solver_worker.dart.js:3111:15)
    at _wrapJsFunctionForAsync_closure.call$2 (http://localhost:5000/solver_worker.dart.js:10955:12)
    at Object._asyncStartSync (http://localhost:5000/solver_worker.dart.js:3075:20)
solver_worker.dart.js 762:17                                                  wrapException
solver_worker.dart.js 1931:15                                                 _failedAsCheck
solver_worker.dart.js 1920:9                                                  _generalAsCheckImplementation
solver_worker.dart.js 6204:37                                                 _$CanvasNodeFromJson
solver_worker.dart.js 18384:20                                                fromJson$1
solver_worker.dart.js 16911:19                                                performLayout$6
solver_worker.dart.js 16678:93                                                <fn>
solver_worker.dart.js 3111:15                                                 $protected
solver_worker.dart.js 10955:12                                                call$2
solver_worker.dart.js 3075:20                                                 _asyncStartSync
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 251:49  throw_
packages/squadron/src/worker.dart 134:7                                       send
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 60:31            <fn>
dart-sdk/lib/async/zone.dart 1450:47                                          _rootRunBinary
dart-sdk/lib/async/zone.dart 1342:19                                          runBinary
dart-sdk/lib/async/future_impl.dart 174:22                                    handleError
dart-sdk/lib/async/future_impl.dart 778:46                                    handleError
dart-sdk/lib/async/future_impl.dart 799:13                                    _propagateToListeners
dart-sdk/lib/async/future_impl.dart 609:5                                     [_completeError]
dart-sdk/lib/async/future_impl.dart 665:7                                     <fn>
dart-sdk/lib/async/zone.dart 1426:13                                          _rootRun
dart-sdk/lib/async/zone.dart 1328:19                                          run
dart-sdk/lib/async/zone.dart 1236:7                                           runGuarded
dart-sdk/lib/async/zone.dart 1276:23                                          callback
dart-sdk/lib/async/schedule_microtask.dart 40:11                              _microtaskLoop
dart-sdk/lib/async/schedule_microtask.dart 49:5                               _startMicrotaskLoop
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 166:15           <fn>

SaadArdati commented 2 years ago

Even a simple

    final nodeReferences = Set.from(dynNodeReferences
        .map((refs) => refs.map((key, value) => MapEntry(key, value))));

is failing. We don't need types on maps, so It's easy to remove in dart, but it's the same error regardless!

SaadArdati commented 2 years ago

One clever thing we can do is pass a json String instead of a Set object... I will give that a shot if you don't think this can be resolved any other way.

SaadArdati commented 2 years ago

    final Set<Map<String, dynamic>> nodeReferences = {};

    dynNodeReferences.forEach((mapElement) {
      Map map = mapElement as Map;
      nodeReferences.add(map as Map<String, dynamic>);
    });

This also failed with the same error.

  A.PositionManagerThread_performLayout_closure.prototype = {
    call$1(mapElement) {
      this.nodeReferences.add$1(0, type$.Map_String_dynamic._as(type$.Map_dynamic_dynamic._as(mapElement)));
    },
    $signature: 11
  };

SaadArdati commented 2 years ago

This one is interesting:

Different error

    final Set<Map<String, dynamic>> nodeReferences = {};

    dynNodeReferences.forEach((mapElement) {
      final Map<String, dynamic> castedMap = {};
      for (final entry in mapElement.entries) {
        castedMap[entry.key] = entry.value;
      }
    });

a null check on the key does not fix it.

        if (entry.key == null) continue;

WorkerException: NoSuchMethodError: method not found: 'toString' on null
TypeError: Cannot read properties of null (reading 'toString')
    at PositionManagerThread_performLayout_closure2.call$1 (http://localhost:5000/solver_worker.dart.js:16998:10)
    at MappedIterator.moveNext$0 (http://localhost:5000/solver_worker.dart.js:9914:48)
    at _LinkedHashSet.addAll$1 (http://localhost:5000/solver_worker.dart.js:12535:99)
    at PositionManagerThread.performLayout$6 (http://localhost:5000/solver_worker.dart.js:16524:13)
    at http://localhost:5000/solver_worker.dart.js:16234:93
    at _wrapJsFunctionForAsync_closure.$protected (http://localhost:5000/solver_worker.dart.js:3111:15)
    at _wrapJsFunctionForAsync_closure.call$2 (http://localhost:5000/solver_worker.dart.js:10888:12)
    at Object._asyncStartSync (http://localhost:5000/solver_worker.dart.js:3075:20)
    at SolverServiceImpl_operations_closure.$call$body$SolverServiceImpl_operations_closure0 (http://localhost:5000/solver_worker.dart.js:16246:16)
    at SolverServiceImpl_operations_closure.call$1 (http://localhost:5000/solver_worker.dart.js:16204:19)
solver_worker.dart.js 16998:10                                                call$1
solver_worker.dart.js 9914:48                                                 moveNext$0
solver_worker.dart.js 12535:99                                                addAll$1
solver_worker.dart.js 16524:13                                                performLayout$6
solver_worker.dart.js 16234:93                                                <fn>
solver_worker.dart.js 3111:15                                                 $protected
solver_worker.dart.js 10888:12                                                call$2
solver_worker.dart.js 3075:20                                                 _asyncStartSync
solver_worker.dart.js 16246:16                                                $call$body$SolverServiceImpl_operations_closure0
solver_worker.dart.js 16204:19                                                call$1
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 251:49  throw_
packages/squadron/src/worker.dart 134:7                                       send
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 60:31            <fn>
dart-sdk/lib/async/zone.dart 1450:47                                          _rootRunBinary
dart-sdk/lib/async/zone.dart 1342:19                                          runBinary
dart-sdk/lib/async/future_impl.dart 174:22                                    handleError
dart-sdk/lib/async/future_impl.dart 778:46                                    handleError
dart-sdk/lib/async/future_impl.dart 799:13                                    _propagateToListeners
dart-sdk/lib/async/future_impl.dart 609:5                                     [_completeError]
dart-sdk/lib/async/future_impl.dart 665:7                                     <fn>
dart-sdk/lib/async/zone.dart 1426:13                                          _rootRun
dart-sdk/lib/async/zone.dart 1328:19                                          run
dart-sdk/lib/async/zone.dart 1236:7                                           runGuarded
dart-sdk/lib/async/zone.dart 1276:23                                          callback
dart-sdk/lib/async/schedule_microtask.dart 40:11                              _microtaskLoop
dart-sdk/lib/async/schedule_microtask.dart 49:5                               _startMicrotaskLoop
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 166:15           <fn>

d-markey commented 2 years ago

Hello Saad I've been working on a Flutter sample heavily based on your code last night, I'll try to finish that tonight and post it so you can try. I think you're looking at the wrong place and the problem is not in that code, but it's difficult to locate it.

Eg can you try this:

    final Set<Map<String, dynamic>> nodeReferences = {};

    dynNodeReferences.forEach((mapElement) {
      final Map<String, dynamic> castedMap = {};
      for (final entry in mapElement.entries) {
        castedMap['constant'] = entry.value;
      }
    });

I believe it will throw anyway! So the exception is probably not coming from that piece of code.

d-markey commented 2 years ago

BTW I suppose this is addAll(), not forEach()? Maybe it's missing a return castedMap;?

Or rather, it is forEach() and you're missing a nodesReferences.add(castedMap)?

SaadArdati commented 2 years ago

You're right. trying.

SaadArdati commented 2 years ago

ChromeProxyService: Failed to evaluate expression 'nodeReferences'. 
worker start
worker started. result: {Instance of 'ModelFingerPrint'}
WorkerException (workerId=738884832, command=1): NoSuchMethodError: method not found: 'BaseNodeMixin___BaseNodeMixin_id' on null
TypeError: Cannot read properties of null (reading 'BaseNodeMixin___BaseNodeMixin_id')
    at PositionManagerThread.performLayout$6 (http://localhost:5000/solver_worker.dart.js:16443:63)
    at http://localhost:5000/solver_worker.dart.js:16210:93
    at _wrapJsFunctionForAsync_closure.$protected (http://localhost:5000/solver_worker.dart.js:3111:15)
    at _wrapJsFunctionForAsync_closure.call$2 (http://localhost:5000/solver_worker.dart.js:10878:12)
    at Object._asyncStartSync (http://localhost:5000/solver_worker.dart.js:3075:20)
    at SolverServiceImpl_operations_closure.$call$body$SolverServiceImpl_operations_closure0 (http://localhost:5000/solver_worker.dart.js:16222:16)
    at SolverServiceImpl_operations_closure.call$1 (http://localhost:5000/solver_worker.dart.js:16180:19)
    at http://localhost:5000/solver_worker.dart.js:8415:27
    at _wrapJsFunctionForAsync_closure.$protected (http://localhost:5000/solver_worker.dart.js:3111:15)
    at _wrapJsFunctionForAsync_closure.call$2 (http://localhost:5000/solver_worker.dart.js:10878:12)
solver_worker.dart.js 16443:63                                                    performLayout$6
solver_worker.dart.js 16210:93                                                    <fn>
solver_worker.dart.js 3111:15                                                     $protected
solver_worker.dart.js 10878:12                                                    call$2
solver_worker.dart.js 3075:20                                                     _asyncStartSync
solver_worker.dart.js 16222:16                                                    $call$body$SolverServiceImpl_operations_closure0
solver_worker.dart.js 16180:19                                                    call$1
solver_worker.dart.js 8415:27                                                     <fn>
solver_worker.dart.js 3111:15                                                     $protected
solver_worker.dart.js 10878:12                                                    call$2
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 251:49      throw_
packages/squadron/src/worker.dart 134:7                                           send
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 60:31                <fn>
dart-sdk/lib/async/zone.dart 1450:47                                              _rootRunBinary
dart-sdk/lib/async/zone.dart 1342:19                                              runBinary
dart-sdk/lib/async/future_impl.dart 174:22                                        handleError
dart-sdk/lib/async/future_impl.dart 778:46                                        handleError
dart-sdk/lib/async/future_impl.dart 799:13                                        _propagateToListeners
dart-sdk/lib/async/future_impl.dart 592:7                                         [_complete]
dart-sdk/lib/async/stream_pipe.dart 61:11                                         _cancelAndValue
dart-sdk/lib/async/stream.dart 1288:7                                             <fn>
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/operations.dart 334:14  _checkAndCall
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/operations.dart 339:39  dcall
dart-sdk/lib/html/dart2js/html_dart2js.dart 37301:58                              <fn>
dart-sdk/lib/async/zone.dart 1442:13                                              _rootRunUnary
dart-sdk/lib/async/zone.dart 1335:19                                              runUnary
dart-sdk/lib/async/zone.dart 1244:7                                               runUnaryGuarded
dart-sdk/lib/async/zone.dart 1281:26                                              <fn>

Interestingly, we're getting a new error now??

SaadArdati commented 2 years ago

BaseNodeMixin is nothing fancy:

SaadArdati commented 2 years ago

The error is consistent, not random.

method not found: 'BaseNodeMixin___BaseNodeMixin_id' on null

Must mean that we're calling .id on a null value. Very odd...

This issue is not from our code, there's just no way. We're a whole team and this is a core class that's battle-tested and has been running flawlesly for a year now... You're correct the errors are not true, there's definitely something else going wrong.

SaadArdati commented 2 years ago

Let's add more context to the problem. Since we don't care about the type of the map at all, I did the following:

    final nodeReferences = dynNodeReferences;

    // final nodeReferences = Set.from(dynNodeReferences
    //     .map((refs) => refs.map((key, value) => MapEntry(key, value))));

    final Map<String, SceneNode> freshlyBakedNodes = {};
    print('LAYOUT START: ----------------------------------------->');
    for (final dynamic nodeJson in nodeReferences) {
      final SceneNode node =
          nodeJsonConverter.fromJson(Map<String, dynamic>.from(nodeJson))!;

I directly passed dynNodeREferences and surprise surprise, still got a similar error.


WorkerException (workerId=287635479, command=1): TypeError: Instance of 'JsLinkedHashMap<dynamic, dynamic>': type 'JsLinkedHashMap<dynamic, dynamic>' is not a subtype of type 'Map<String, dynamic>'
TypeError: Instance of 'JsLinkedHashMap<dynamic, dynamic>': type 'JsLinkedHashMap<dynamic, dynamic>' is not a subtype of type 'Map<String, dynamic>'
    at Object.wrapException (http://localhost:5000/solver_worker.dart.js:762:17)
    at Object._failedAsCheck (http://localhost:5000/solver_worker.dart.js:1931:15)
    at Rti._generalAsCheckImplementation [as _as] (http://localhost:5000/solver_worker.dart.js:1920:9)
    at Object._$CanvasNodeFromJson (http://localhost:5000/solver_worker.dart.js:6138:37)
    at NodeJsonConverter.fromJson$1 (http://localhost:5000/solver_worker.dart.js:17844:20)
    at PositionManagerThread.performLayout$6 (http://localhost:5000/solver_worker.dart.js:16383:19)
    at http://localhost:5000/solver_worker.dart.js:16153:93
    at _wrapJsFunctionForAsync_closure.$protected (http://localhost:5000/solver_worker.dart.js:3111:15)
    at _wrapJsFunctionForAsync_closure.call$2 (http://localhost:5000/solver_worker.dart.js:10872:12)
    at Object._asyncStartSync (http://localhost:5000/solver_worker.dart.js:3075:20)
solver_worker.dart.js 762:17                                                      wrapException
solver_worker.dart.js 1931:15                                                     _failedAsCheck
solver_worker.dart.js 1920:9                                                      _generalAsCheckImplementation
solver_worker.dart.js 6138:37                                                     _$CanvasNodeFromJson
solver_worker.dart.js 17844:20                                                    fromJson$1
solver_worker.dart.js 16383:19                                                    performLayout$6
solver_worker.dart.js 16153:93                                                    <fn>
solver_worker.dart.js 3111:15                                                     $protected
solver_worker.dart.js 10872:12                                                    call$2
solver_worker.dart.js 3075:20                                                     _asyncStartSync
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 251:49      throw_
packages/squadron/src/worker.dart 134:7                                           send
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 60:31                <fn>
dart-sdk/lib/async/zone.dart 1450:47                                              _rootRunBinary
dart-sdk/lib/async/zone.dart 1342:19                                              runBinary
dart-sdk/lib/async/future_impl.dart 174:22                                        handleError
dart-sdk/lib/async/future_impl.dart 778:46                                        handleError
dart-sdk/lib/async/future_impl.dart 799:13                                        _propagateToListeners
dart-sdk/lib/async/future_impl.dart 592:7                                         [_complete]
dart-sdk/lib/async/stream_pipe.dart 61:11                                         _cancelAndValue
dart-sdk/lib/async/stream.dart 1288:7                                             <fn>
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/operations.dart 334:14  _checkAndCall
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/operations.dart 339:39  dcall
dart-sdk/lib/html/dart2js/html_dart2js.dart 37301:58                              <fn>
dart-sdk/lib/async/zone.dart 1442:13                                              _rootRunUnary
dart-sdk/lib/async/zone.dart 1335:19                                              runUnary
dart-sdk/lib/async/zone.dart 1244:7                                               runUnaryGuarded
dart-sdk/lib/async/zone.dart 1281:26                                              <fn>

ChromeProxyService: Failed to evaluate expression 'nodeJsonConverter'.

SaadArdati commented 2 years ago

NodeJsonConverter:

I'm starting to sense a pattern...

d-markey commented 2 years ago

Hello, here's an example based on your code samples.

The bottom line is to try to handle all type issues related to platform workers as close as possible to the communication layer. You have to take into account the most constrained platform, and here that would be JavaScript.

The best places to handle types aspect are in:

the service worker implementation, when calling "send()": make sure you "serialize" your service arguments (use only base types and List/Map) and "deserialize" the worker response
the service implementation "operations" map: make sure you "deserialize" the service arguments, and make sure you "serialize" the result.

To sum things up:

architecture

and the sample code:

solver_example.zip

d-markey commented 2 years ago

In fact, for production code, the best way to go is maybe to define classes to communicate with your worker methods through Squadron, with built-in serialization / deserialization.

This standardized approach will make it easier to understand and maintain the code. It also makes serialization/deserialization for one class (the request or the response) a responsibility of the class, so you don't have to bother with reinterpreting types in your service methods anymore.

Something like:

class ServiceResponse {
   static ServiceResponse deserialize(dynamic result) {
      // deserialize "result", knowing it was produced by serialize()
   }

   dynamic serialize() {
      // serialize using only base types or simple List/Map
   }
}

class ServiceRequest {
   static ServiceRequest deserialize(dynamic result) {
      // deserialize "result", knowing it was produced by serialize()
   }

   dynamic serialize() {
      // serialize using only base types or simple List/Map
   }
}

abstract class ServiceDefinition {
   FutureOr<ServiceResponse> serviceMethod(ServiceRequest request);

   static const cmdServiceMethod = 1;
}

class ServiceImplementation implements ServiceDefinition {
   @override
   ServiceResponse serviceMethod(ServiceRequest request) {
      // your code to process the request and send the response
      // because all the rest is quite "automatic", this is were you want to concentrate
      // all the surroundings (request, response, worker, operations map...) becomes simple, standard plumbing
   }

   // in the operations map, deserialize arguments and serialize result

   @override
   late final Map<int, CommandHandler> operations = {
      ServiceDefinition.cmdServiceMethod: (WorkerRequest r) => serviceMethod(ServiceRequest.deserialize(r.args[0])).serialize();
   };
}

class ServiceWorker implements ServiceDefinition, WorkerService {

   // in the worker overrides, serialize arguments and deserialize result

   @override
   ServiceResponse serviceMethod(ServiceRequest request)
      => ServiceResponse.deserialize(await send(ServiceDefinition.cmdServiceMethod, [ request.serialize() ]));
}

SaadArdati commented 2 years ago

@d-markey I'm at a loss to be honest with you. Your sample runs perfectly, yet my code does not, and I've tried to mimic it as much as possible since the two code-bases are nearly identical.

I'm still getting the error

WorkerException (workerId=66711533, command=1): TypeError: Instance of 'JsLinkedHashMap<dynamic, dynamic>': type 'JsLinkedHashMap<dynamic, dynamic>' is not a subtype of type 'Map<String, dynamic>'

multithreading.zip

This is a chunk of the classes I'm dealing with, it's near identical to your sample.

I noticed that you're instantiating a new instance of PositionManagerThread in more than one place in your sample instead of relying on createWorker and SolverWorkerImpl. I don't think it matters, but you could've used it, but you have PositionManagerThread acting like its own Worker and it seems redundant since the operations are now duplicated in your sample.

Are you able to spot what I'm doing wrong? This is driving my a bit crazy at this point. If you need to see some of the other classes, I will provide them.

I think the biggest difference is that I'm not using a WorkerPool because I need a single instance of this thing, but that shouldn't matter since it's created in the exact same fashion?

d-markey commented 2 years ago

Actually I don't instantiate a SolverWorkerImpl anymore in my sample because from the code snippets you provided, SolverWorkerImpl is just a wrapper around PositionManagerThread. It does nothing else but proxy the calls to PositionManagerThread. So I made PositionManagerThread a SolverService and a WorkerService. You can delete SolverWorkerImpl from the sample, it will work the same :-)

Regarding pool/single worker, I've already tested all possibilities. To see for yourself, you can activate single worker instead of pool by changing the main() program in my sample:

void main() {
  Squadron.setId('main');
  Squadron.logLevel = SquadronLogLevel.ALL;
  Squadron.logger = ConsoleSquadronLogger();

  // runApp(MyApp(SolverWorkerPool())); // if you want a pool
  runApp(MyApp(createWorker())); // if you want only one worker
  // runApp(MyApp(PositionManagerThread(notifier: (pos) => Squadron.info(pos)))); // you can even test the service itself!
}

Now, my implementation of PositionManagerThread is a pure invention and the only difference I see is that your version does probably not return Map<String, dynamic> values, and that would be the purpose of toJson() in SolverWorkerImpl to convert the return values from PositionManagerThread to a Map<String, dynamic>.

So I wonder, what does toJson() look like?

SaadArdati commented 2 years ago

performLayout returns a LayoutResult which is a @JsonSerializable object:

@JsonSerializable(explicitToJson: true, anyMap: true)
class ModelFingerPrint {
  final String id;
  final String? parentId;
  final List<String> children;
  final double x, y, width, height;
  final double summarizedChildrenWidth, summarizedChildrenHeight;

  ModelFingerPrint({
    required this.id,
    required this.parentId,
    required this.children,
    required this.x,
    required this.y,
    required this.width,
    required this.height,
    required this.summarizedChildrenWidth,
    required this.summarizedChildrenHeight,
  });

  factory ModelFingerPrint.fromJson(Map json) =>
      _$ModelFingerPrintFromJson(json);

  Map<String, dynamic> toJson() => _$ModelFingerPrintToJson(this);
}

@JsonSerializable(explicitToJson: true, anyMap: true)
class LayoutResult {
  final Set<ModelFingerPrint> models;
  final Set<String> changedIDs;

  LayoutResult({
    required this.models,
    required this.changedIDs,
  });

  LayoutResult.empty()
      : models = {},
        changedIDs = {};

  factory LayoutResult.fromJson(Map json) => _$LayoutResultFromJson(json);

  Map<String, dynamic> toJson() => _$LayoutResultToJson(this);
}

It works fine on native, so it's not like its not able to serialize/deserialize it, but perhaps its too complex of an object for js?

You're right in thinking its not the contents of performLayout that are the issue because its unable to print the first line inside the function, it instantly throws once it runs.

Here's a sample toJson output produced by the native implementation of this (since browser refuses to even print the first line in performLayout)

{models: [{id: rootNode, children: [0QHxMdQRKSuFJYw1t1ei], x: -0.0, y: -0.0, width: 20000.0, height: 20000.0, summarizedChildrenWidth: 375.0, summarizedChildrenHeight: 812.0}, {id: 0QHxMdQS5gdBIDIvgPZT, parentId: 0QHxMdQS5gdBIDIvgPZR, children: [], x: 10203.74, y: 10123.91, width: 39.96549999937415, height: 22.0, summarizedChildrenWidth: 0.0, summarizedChildrenHeight: 0.0}, {id: 0QHxMdQRKSuFJYw1t1ei, parentId: rootNode, children: [0QHxMdQS5gdBIDIvgPZD, 0QHxMdQS5gdBIDIvgPZR, 0QJ7Yt8mlmIwvbVzdlCm], x: 9815.74, y: 9801.91, width: 375.0, height: 812.0, summarizedChildrenWidth: 528.0, summarizedChildrenHeight: 818.0}, {id: 0QHxMdQS5gdBIDIvgPZD, parentId: 0QHxMdQRKSuFJYw1t1ei, children: [0QHxMdQS5gdBIDIvgPZE], x: 10282.74, y: 10074.91, width: 76.0, height: 80.0, summarizedChildrenWidth: 76.0, summarizedChildrenHeight: 80.0}, {id: 0QHxMdQS5gdBIDIvgPZE, parentId: 0QHxMdQS5gdBIDIvgPZD, children: [], x: 10282.74, y: 10074.91, width: 76.0, height: 80.0, summarizedChildrenWidth: 0.0, summarizedChildrenHeight: 0.0}, {id: 0QHxMdQS5gdBIDIvgPZR, parentId: 0QHxMdQRKSuFJYw1t1ei, children: [0QHxMdQS5gdBIDIvgPZS, 0QHxMdQS5gdBIDIvgPZT], x: 10194.74, y: 10074.91, width: 76.0, height: 80.0, summarizedChildrenWidth: 76.0, summarizedChildrenHeight: 80.0}, {id: 0QHxMdQS5gdBIDIvgPZS, parentId: 0QHxMdQS5gdBIDIvgPZR, children: [], x: 10194.74, y: 10074.91, width: 76.0, height: 80.0, summarizedChildrenWidth: 0.0, summarizedChildrenHeight: 0.0}, {id: 0QJ7Yt8mlmIwvbVzdlCm, parentId: 0QHxMdQRKSuFJYw1t1ei, children: [], x: 9754.74, y: 9836.91, width: 528.0, height: 818.0, summarizedChildrenWidth: 0.0, summarizedChildrenHeight: 0.0}], changedIDs: [0QHxMdQS5gdBIDIvgPZR, rootNode, 0QHxMdQRKSuFJYw1t1ei, 0QHxMdQS5gdBIDIvgPZD, 0QHxMdQS5gdBIDIvgPZT, 0QHxMdQS5gdBIDIvgPZE, 0QHxMdQS5gdBIDIvgPZS, 0QJ7Yt8mlmIwvbVzdlCm]}

Converted to actual json:

{
  "models": [
    {
      "id": "rootNode",
      "children": [
        "0QHxMdQRKSuFJYw1t1ei",
        "0QJ8cQrjMtNjctbJ2hpZ"
      ],
      "x": 0,
      "y": 0,
      "width": 20000,
      "height": 20000,
      "summarizedChildrenWidth": 375,
      "summarizedChildrenHeight": 812
    },
    {
      "id": "0QJ8cQrjMtNjctbJ2hpZ",
      "parentId": "rootNode",
      "children": [],
      "x": 8993.87890625,
      "y": 9838.6328125,
      "width": 109.64453125,
      "height": 310.4140625,
      "summarizedChildrenWidth": 0,
      "summarizedChildrenHeight": 0
    },
    {
      "id": "0QHxMdQS5gdBIDIvgPZT",
      "parentId": "0QHxMdQS5gdBIDIvgPZR",
      "children": [],
      "x": 10203.74,
      "y": 10123.91,
      "width": 39.96549999937415,
      "height": 22,
      "summarizedChildrenWidth": 0,
      "summarizedChildrenHeight": 0
    },
    {
      "id": "0QHxMdQRKSuFJYw1t1ei",
      "parentId": "rootNode",
      "children": [
        "0QHxMdQS5gdBIDIvgPZD",
        "0QHxMdQS5gdBIDIvgPZR",
        "0QJ7Yt8mlmIwvbVzdlCm"
      ],
      "x": 9815.74,
      "y": 9801.91,
      "width": 375,
      "height": 812,
      "summarizedChildrenWidth": 528,
      "summarizedChildrenHeight": 818
    },
    {
      "id": "0QHxMdQS5gdBIDIvgPZD",
      "parentId": "0QHxMdQRKSuFJYw1t1ei",
      "children": [
        "0QHxMdQS5gdBIDIvgPZE"
      ],
      "x": 10282.74,
      "y": 10074.91,
      "width": 76,
      "height": 80,
      "summarizedChildrenWidth": 76,
      "summarizedChildrenHeight": 80
    },
    {
      "id": "0QHxMdQS5gdBIDIvgPZE",
      "parentId": "0QHxMdQS5gdBIDIvgPZD",
      "children": [],
      "x": 10282.74,
      "y": 10074.91,
      "width": 76,
      "height": 80,
      "summarizedChildrenWidth": 0,
      "summarizedChildrenHeight": 0
    },
    {
      "id": "0QHxMdQS5gdBIDIvgPZR",
      "parentId": "0QHxMdQRKSuFJYw1t1ei",
      "children": [
        "0QHxMdQS5gdBIDIvgPZS",
        "0QHxMdQS5gdBIDIvgPZT"
      ],
      "x": 10194.74,
      "y": 10074.91,
      "width": 76,
      "height": 80,
      "summarizedChildrenWidth": 76,
      "summarizedChildrenHeight": 80
    },
    {
      "id": "0QHxMdQS5gdBIDIvgPZS",
      "parentId": "0QHxMdQS5gdBIDIvgPZR",
      "children": [],
      "x": 10194.74,
      "y": 10074.91,
      "width": 76,
      "height": 80,
      "summarizedChildrenWidth": 0,
      "summarizedChildrenHeight": 0
    },
    {
      "id": "0QJ7Yt8mlmIwvbVzdlCm",
      "parentId": "0QHxMdQRKSuFJYw1t1ei",
      "children": [],
      "x": 9754.74,
      "y": 9836.91,
      "width": 528,
      "height": 818,
      "summarizedChildrenWidth": 0,
      "summarizedChildrenHeight": 0
    }
  ],
  "changedIDs": [
    "rootNode",
    "0QHxMdQS5gdBIDIvgPZR",
    "0QHxMdQRKSuFJYw1t1ei",
    "0QHxMdQS5gdBIDIvgPZD",
    "0QJ8cQrjMtNjctbJ2hpZ",
    "0QHxMdQS5gdBIDIvgPZT",
    "0QHxMdQS5gdBIDIvgPZE",
    "0QHxMdQS5gdBIDIvgPZS",
    "0QJ7Yt8mlmIwvbVzdlCm"
  ]
}

SaadArdati commented 2 years ago

// GENERATED CODE - DO NOT MODIFY BY HAND

part of 'position_manager_thread.dart';

// **************************************************************************
// JsonSerializableGenerator
// **************************************************************************

ModelFingerPrint _$ModelFingerPrintFromJson(Map json) => ModelFingerPrint(
      id: json['id'] as String,
      parentId: json['parentId'] as String?,
      children:
          (json['children'] as List<dynamic>).map((e) => e as String).toList(),
      x: (json['x'] as num).toDouble(),
      y: (json['y'] as num).toDouble(),
      width: (json['width'] as num).toDouble(),
      height: (json['height'] as num).toDouble(),
      summarizedChildrenWidth:
          (json['summarizedChildrenWidth'] as num).toDouble(),
      summarizedChildrenHeight:
          (json['summarizedChildrenHeight'] as num).toDouble(),
    );

Map<String, dynamic> _$ModelFingerPrintToJson(ModelFingerPrint instance) {
  final val = <String, dynamic>{
    'id': instance.id,
  };

  void writeNotNull(String key, dynamic value) {
    if (value != null) {
      val[key] = value;
    }
  }

  writeNotNull('parentId', instance.parentId);
  val['children'] = instance.children;
  val['x'] = instance.x;
  val['y'] = instance.y;
  val['width'] = instance.width;
  val['height'] = instance.height;
  val['summarizedChildrenWidth'] = instance.summarizedChildrenWidth;
  val['summarizedChildrenHeight'] = instance.summarizedChildrenHeight;
  return val;
}

LayoutResult _$LayoutResultFromJson(Map json) => LayoutResult(
      models: (json['models'] as List<dynamic>)
          .map((e) => ModelFingerPrint.fromJson(e as Map))
          .toSet(),
      changedIDs:
          (json['changedIDs'] as List<dynamic>).map((e) => e as String).toSet(),
    );

Map<String, dynamic> _$LayoutResultToJson(LayoutResult instance) =>
    <String, dynamic>{
      'models': instance.models.map((e) => e.toJson()).toList(),
      'changedIDs': instance.changedIDs.toList(),
    };

SaadArdati commented 2 years ago

Moving toJson into the inside of performLayout and making the function return the Map<String, dynamic> directly does not fix it :/

d-markey commented 2 years ago

what browser are you using? have you tried several?

SaadArdati commented 2 years ago

I noticed in the Stack trace the following:

The NodeJsonConverter looks like this:

Could it be the cause? It's expecting a Map<String, dynamic> but we're giving it a <dynamic, dynamic> from javascript. This is triggered here:

I've been testing in a chrome browser on my macbook pro M1. Chrome is up to date.

SaadArdati commented 2 years ago

Could always try this on all our serializable data? But this assumes that javascript simply can't handle any toJson conversion which is absurd because flutter runs in the web just fine...

d-markey commented 2 years ago

Yes, you need to "deep rebuild" the data after communication, not just the first level.

for (final dynamic nodeJson in nodeReferences) {

=> nodeReferences is a Map<String, dynamic> and I guess nodeJson has lost its strong type too. It's probably now just a Map. So I don't think it's valid to call Map<String, dynamic>.from(nodeJson). Maybe try RebuildMap<String>(nodeJson)?

SaadArdati commented 2 years ago

Same error

I want to remark that not even the print in the screenshot is running.

d-markey commented 2 years ago

Or in NodeJsonConverter, change to fromJson(Map json) which should allow you to call fromJson(nodeJson) directly in performLayout. You might have to check the other fromJson() methods from RootNode, CanvasNode, etc.

SaadArdati commented 2 years ago

Yes. We have a lot of node classes I will need to convert to use anyMap = true... I will need some time to do this.

By the way, once we figure this out, I'm writing a big technical article about squadron :)

d-markey commented 2 years ago

Cross platform multithreading comes with a cost! I hope you'll get there! 🤞

d-markey commented 2 years ago

Regarding print(), I've also seen strange behaviour in the browser. it seems it does not always log to the console...

SaadArdati commented 2 years ago

It seems to have finally worked! It's a massive caveat though :( I had to make an extension to cast Map to Map<String, dynamic> and used it where all the errors where because huge parts of the project assume the usage of Map<String, dynamic>

This is... quite the cost... This should 100% be documented very loudly.

SaadArdati commented 2 years ago

Thank you so much for the continued support and being patient with me <3

d-markey commented 2 years ago

You're welcome, glad you did it! I will document those type aspects to warn people about dealing with complex data, serializing all the way down on one side then deserializing all the way up on the other side.

I recognize json_serializable from your annotations, could you share the configuration required to achieve this serialization?

SaadArdati commented 2 years ago

Yes, sure.

That's all that's required for jsonSerializable. You need to regenerate the generated files afterwards.

d-markey commented 2 years ago

I was thinking of the build.yaml configuration, did you just switch any_map to true and that made it work?

SaadArdati commented 2 years ago

You CAN use build.yaml and generate it that way, its the same thing. But you'll still need to strip away the types from toJson and fromJson.

I set the any map manually in this case