predict in an AWS Lambda or GCP Function?

[bittlingmayer]

Model size makes serverless a bit odd, but in theory it should work fine once the function is called often enough to keep the re-usable container loaded.

Have you tried it and do you plan to support it or do you think it is not doable or not sensical?

https://cloud.google.com/functions/ http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html

[loretoparisi]

@bittlingmayer thanks I will provide a simple Amazon Lambda example.

[bittlingmayer]

@loretoparisi The equivalent for a standard service would be a Docker image or example Docker file, that could also be useful.

[loretoparisi]

@bittlingmayer thanks for the suggestion! In the meanwhile take a look at the current Dockerfile here https://github.com/loretoparisi/fasttext.js/blob/master/Dockerfile I will add the AWS sdk and a simple lambda processing function as well!

[bittlingmayer]

I am trying this now on a Cloud function and have a question:

It seems I must call .load() and then call .predict() directly after that and then call .unload, because without .unload it hangs.

But ideally I would call .load() in the init - I assume it's heavy - and then on each request just call .predict().

Am I doing something wrong?

[loretoparisi]

@bittlingmayer basically the fasttext need a first call to load the binary model into memory once, that is what the load api is doing. The consecutive calls to predict, train, etc. will then run commands like inference, training, etc. against that model loaded already. The unload will simply clean up resources from memory, in terms of the sub process loaded from node.js, to be sure that the binary model is unloaded from memory, when the child process has been properly killed i.e. gracefully exited.

It is correct therefore to

• Call load once
• Call predict while running your process i.e. at each request.
• Call unload before exiting or when you want to restart your application or this process caller.

If you look at the server example here, this is what I'm doing:

1. Calling the load and start the server - https://github.com/loretoparisi/fasttext.js/blob/master/examples/server.js#L56
2. Calling predict at every api call - https://github.com/loretoparisi/fasttext.js/blob/master/examples/server.js#L37

In this example I'm not calling the unload api, since it is supposed to not be stopped or restarted. Hope this helps!

[bittlingmayer]

Made it work now.

The issue was that I had to include this repo directly in the package.json, the latest version up on npm does not work.

[bittlingmayer]

Work in progress but here it is: https://github.com/SignalN/ftfn

[bittlingmayer]

Now that the fixes to #15 are pushed to npm, I suppose we can close this.