Closed bittlingmayer closed 6 years ago
@bittlingmayer thanks I will provide a simple Amazon Lambda example.
@loretoparisi The equivalent for a standard service would be a Docker image or example Docker file, that could also be useful.
@bittlingmayer thanks for the suggestion! In the meanwhile take a look at the current Dockerfile here https://github.com/loretoparisi/fasttext.js/blob/master/Dockerfile I will add the AWS sdk and a simple lambda processing function as well!
I am trying this now on a Cloud function and have a question:
It seems I must call .load() and then call .predict() directly after that and then call .unload, because without .unload it hangs.
But ideally I would call .load() in the init - I assume it's heavy - and then on each request just call .predict().
Am I doing something wrong?
@bittlingmayer basically the fasttext
need a first call to load the binary model into memory once, that is what the load
api is doing. The consecutive calls to predict
, train
, etc. will then run commands like inference, training, etc. against that model loaded already. The unload
will simply clean up resources from memory, in terms of the sub process loaded from node.js, to be sure that the binary model is unloaded from memory, when the child process has been properly killed i.e. gracefully exited.
It is correct therefore to
load
oncepredict
while running your process i.e. at each request.unload
before exiting or when you want to restart your application or this process caller.If you look at the server example here, this is what I'm doing:
load
and start the server - https://github.com/loretoparisi/fasttext.js/blob/master/examples/server.js#L56predict
at every api call - https://github.com/loretoparisi/fasttext.js/blob/master/examples/server.js#L37In this example I'm not calling the unload
api, since it is supposed to not be stopped or restarted.
Hope this helps!
Made it work now.
The issue was that I had to include this repo directly in the package.json, the latest version up on npm does not work.
Work in progress but here it is: https://github.com/SignalN/ftfn
Now that the fixes to #15 are pushed to npm, I suppose we can close this.
Model size makes serverless a bit odd, but in theory it should work fine once the function is called often enough to keep the re-usable container loaded.
Have you tried it and do you plan to support it or do you think it is not doable or not sensical?
https://cloud.google.com/functions/ http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html