search

NativeScript / ios-jsc

NativeScript for iOS using JavaScriptCore
http://docs.nativescript.org/runtimes/ios
Apache License 2.0
299 stars 59 forks source link

[Proposal]: Support for Workers #620

Open ivanbuhov opened 8 years ago

ivanbuhov commented 8 years ago

Support for Workers

For Android solution please check - https://github.com/NativeScript/android-runtime/issues/532

Description

General guidelines for the Workers implementation effort and specification in the context of NativeScript. We have an existing issue here, but the purpose here is to show the road map that the team intends to follow in developing the Workers functionality. The general scenarios we want to support are listed at the bottom.

Limitations

In NativeScript we don’t need to implement all details of the web workers specification, because some of these details are only related to the browser, and won’t have any meaning in the context of a NativeScript application.

List of features we decided not to support:

The features and syntax that will follow has the purpose of describing the adoption of the web workers specification in NativeScript. In this document we will describe and list what we intend to support.

Guidelines from the specification to follow

context

Notice that onmessage and postMessage() need to be hung off the Worker object when used in the main script thread, but not when used in the worker. This is because, inside the worker, the worker is effectively the global scope.

messaging specifics

Data passed between the main thread and workers is copied, not shared. Objects are serialized as they're handed to the worker, and subsequently, deserialized on the other end. The page and worker do not share the same instance, so the end result is that a duplicate is created on each end. Most browsers implement this feature as structured cloning. The structured cloning algorithm can accept JSON and a few things that JSON can't — like circular references.

messaging transfer ownership

Passing data by transferring ownership (transferable objects): in nativescript we can try to pass native objects by transferring ownership on them.

onerror spec

The error event has the following three fields that are of interest: message: A human-readable error message. filename: The name of the script file in which the error occurred. lineno: The line number of the script file on which the error occurred.

Syntax

The following example will be the general syntax to use when working with Workers. Syntax is based on web worker specification.

_/app/main.js_

// create new worker
var myWorker = new Worker("worker.js");

// on worker error handler
myWorker.onerror = function (e) {
 //do on worker error
}

// receive messages from worker
myWorker.onmessage = function (e) {
  result.textContent = e.data;
}
....
// send messages to worker
myWorker.postMessage("message will be sent to worker");

//The worker thread is killed immediately without an opportunity to complete its operations or clean up after itself.
myWorker.terminate();

myWorker.onclose = function () {
//on close (clean up)
}

_/app/worker.js_

onmessage = function(e) {
  postMessage(e.data); // worker -> main
}

//worker closes it self
close();

Supported APIs:

Worker Object:

speigg commented 8 years ago

Will the Worker be able to leverage FTL JIT compilation ? If not, it loses a lot of value for me. It should be possible to leverage FTL JIT compilation if the worker is implemented via a WKWebView, rather than a JSContext.

ivanbuhov commented 8 years ago

Hi @speigg.

Worker threads will not take advantage of the FTL JIT but keep in mind that the same is valid for the main JS thread too. The lack of FTL JIT compilation is due to iOS restriction which forbids allocating writable and executable memory.

Implementing Workers via WKWebView means that we have to use the built-in JavaScriptCore which comes with too many limitations. In the early days of NativeScript we evaluated both options - built-in JSC vs custom JSC build and we strongly believe that using a custom build of JSC is the right approach. Many of the existing iOS runtime features require using a custom JSC build.

If not, it loses a lot of value for me

Can you be more specific on this? It would be great if you provide some use cases in which a lot of value is lost due to the lack of FTL JIT.

speigg commented 8 years ago

I'm decrypting messages using openpgpjs. In the main JS thread, this can pause the UI for up to 8 seconds. I have it now working in a hacked together web worker with some basic structured cloning support (just serializing/deserializing typed arrays and Date objects, not dealing with cyclic references or any other web worker features) using a WkWebView. This cuts down the decryption time to about a second. A second I can deal with, but for my app 8 seconds is way too long, as the decryption happens in response to a user action and the user would be forced to wait while things "load" for 8 seconds or more. The other nice thing about WKWebView as a worker (for me) is that openpgp can automatically leverage the built in web crypto APIs.

speigg commented 8 years ago

Perhaps the limitations you mention with the built-in JSC were unsuitable for the main JS Thread (in order to support performant access to native libraries, I'm guessing?), and these limitations may not be as much of a concern for web worker use cases. I think it would be a great balance to have the main thread be "slower" but with quick interop between native and JS, while a web worker runs JS in a highly optimized manner but with slow interop with native libraries. Or perhaps even allowing a developer to choose how the worker should run: optimizing for interop, or for raw JS performance.

ivanbuhov commented 8 years ago

Hi @speigg

Having a custom JSC build allow us to use the C++ API of JSC which brings many of the runtime features. Some of them are:

Other benefits of using a custom JSC build are:

Possibly, we can add more bullets to the list if we ever start working on alternative solution (e.g. WKWebView based bridge).

To be honest, I like the idea of allowing a developer to choose how the worker should run but even if we are still fine to give up some of the runtime features in order to support WKWebView based workers, this would still be a very huge effort. Most of our codebase can't be reused as is. The task would be near to implementing and supporting a second bridge. We need very good reasons (e.g. much more use cases and currently unsolvable scenarios even by using a native libs) to take this path.

On your particular case: What about using a native library for the decryption part?

speigg commented 8 years ago

Yes, a native library would work too (though I would have to create a plugin that wraps an appropriate library for each platform).

NathanaelA commented 8 years ago

@speigg - My NS-WebWorker plugin actually wraps the WKWebView... We don't need the Team to do this; it already exists. However, the issue with it is that you have no access to the native api's which is the tradeoff...

roblav96 commented 8 years ago

How does one go about experimenting with the preview channel of the WebWorkers?

jasssonpet commented 8 years ago

If you want to try them out now, you'll have to checkout this branch and create an Xcode project as instructed in the readme. After that open the HelloWorld target and create some JS files in the app folder.

roblav96 commented 8 years ago

@jasssonpet Sounds good. I'm gonna play around with this 😄

farfromrefug commented 5 years ago

Is there any plan to move forward on this? I would love to be able to pass native object.

mbektchiev commented 5 years ago

Hi @farfromrefug, we haven't planned on new worker features for the immediate future.

But we've managed to devise a workaround for passing native instances to workers by passing their pointers instead. Taking the unit test from the above pull request for example you can achieve it like this:

var worker = new Worker("./EvalWorker.js");
var nativeDict = NSDictionary.dictionaryWithObjectForKey("value", "key");
var message = {
    value: { dictionaryPtr: interop.handleof(nativeDict).toNumber() }
};
// increase reference count to account for `dictionaryPtr`
nativeDict.retain(); 
worker.postMessage(message);

In the worker you can retrieve the native object like this:

function pointerTo(type, value) {
    var outerPtr = interop.alloc(interop.sizeof(interop.Pointer));
    var outerRef = new interop.Reference(type, outerPtr);
    outerRef.value = value;
    return outerPtr;
}

function valueFromPointerNumber(type, value) {
    const ptr = new interop.Pointer(value);
    const ptrToPtr = pointerTo(interop.Pointer, ptr);
    const ref = new interop.Reference(type, ptrToPtr);

    return ref.value;
}

onmessage = function(msg) {
    var dict = valueFromPointerNumber(NSDictionary, msg.data.value.dictionaryPtr);
    // decrease reference count
    dict.release();
}

While trying to make it work I discovered a bug in the runtime which is currently preventing the above code to work on 64-bit devices and Simulators. But you can test it for example on an iPhone 4s Simulator ор device.

After we merge the fix it will shortly be available as a @next version of tns-ios. To use it you'll have to install it with tns platform remove ios; tns platform add ios@next.

farfromrefug commented 5 years ago

@mbektchiev thanks a lot! the same technique was proposed on android. Will try it soon to see if it work.

mbektchiev commented 5 years ago

No problem. Tell us if it works out for you after you try it.

I forgot to mention something else which is very important -- to correctly keep the native object alive you'll have to call retain and release manually. I've added it to to example above.

farfromrefug commented 5 years ago

@mbektchiev is the fix already released for your code sample to work? Finally had the time to test this. and in the worker code the dict object is null.

EDIT: it works! will share an example here really soon

farfromrefug commented 5 years ago

@mbektchiev as explaned before i got it to work. Now i am facing another issue. What i am trying to achieve is call a worker method from an Objective-C delegate call made on a background Thread:

Right now this is an issue because of memory management, so i have to "clone" the image in didOutputSampleBuffer before sending it to the worker, which is done on UI thread and thus slow. I have to go through the main thread in between the capture background thread and the worker thread.

No real question here, just kind of brainstorming. Does anyone see a way of directly going to the worker?

EDIT: sample code is the demo of the plugin i am working on.

mbektchiev commented 5 years ago

Why do you need to clone it on the main thread. Can't you follow the advice in https://developer.apple.com/documentation/avfoundation/avcapturevideodataoutputsamplebufferdelegate/1385775-captureoutput?language=objc:

If you need to reference the CMSampleBuffer object outside of the scope of this method, you must CFRetain it and then CFRelease it when you are finished with it.

farfromrefug commented 5 years ago

@mbektchiev right now i use OpenCV CvVideoCamera for easy testing/dev

The mat that is passed in processImage is released by CvVideoCamera after that method is done. But in our case the mat is sent asynchronously to the worker. By the time the mat is used by the worker, the mat is released. That's why i need to clone the mat.

farfromrefug commented 5 years ago

@mbektchiev i think there is an issue with the retain, release If i retain in main and release in worker, the objects contained in the NSDictionary are never released If i dont use retain, the dict that i read from the worker is null.

// main

function postMessageToWorker(data) {
            if (isIOS) {
                // the clone makes the UI slow! No solution right now
                const nativeDict = NSMutableDictionary.dictionaryWithObjectForKey(data, 'data');
                const message = {
                    value: { dictionaryPtr: interop.handleof(nativeDict).toNumber() }
                };
                // increase reference count to account for `dictionaryPtr`
                (nativeDict as any).retain();
                worker.postMessage(message);
            }
        }

function postImageToWorker() {
            const folder = knownFolders.currentApp();
            const imageFromLocalFile = fromFile(path.join(folder.path, 'images/test.jpg'));
            const imageSource = new ImageSource();
            imageSource.loadFromBase64(imageFromLocalFile.toBase64String('jpg'));
            const image = imageSource.ios;
            this.postMessageToWorker(image);
}
//worker
equire('globals');
import {  isIOS } from 'tns-core-modules/ui/page/page';

function pointerTo(type, value) {
    const outerPtr = interop.alloc(interop.sizeof(interop.Pointer));
    const outerRef = new interop.Reference(type, outerPtr);
    outerRef.value = value;
    return outerPtr;
}

function valueFromPointerNumber(type, value) {
    const ptr = new interop.Pointer(value);
    const ptrToPtr = pointerTo(interop.Pointer, ptr);
    const ref = new interop.Reference(type, ptrToPtr);

    return ref.value;
}
(global as any).onmessage = function(msg) {
    if (isIOS) {
        const dict = valueFromPointerNumber(NSDictionary, msg.data.value.dictionaryPtr) as NSDictionary<string, any>;
        const data = dict.objectForKey('data');
        // decrease reference count
        (dict as any).release();

    }
};