ROADMAP

[psolymos]

This is the roadmap for revising the R/rstats templates for OpenFaaS.

Base images

We need the following base image options (BASEIMAGE below):

• Debian based rocker/r-base: consistency for most adopted base images, call this base;
• Ubuntu based rocker/r-ubuntu: long term support can be important in corporate context and it jives with RSPM, call this ubuntu;
• Alpine based r-hub/r-minimal: small size is always a good thing, call this minimal.

Watchdog version

• Classic watchdog with stdin/stdout: call these images rstats-BASEIMAGE;
• of-watchdog with http support: call these images rstats-BASEIMAGE-FRAMEWORK.

Existing server frameworks

The following are server frameworks capable of running R based APIs:

• Rserve: very bare-bones implementation (RestRserve wraps this), rstats-BASEIMAGE-rserve;
  • RestRserve: a full-fledged API framework based on Rserve, rstats-BASEIMAGE-restserve;
• httpuv: the most popular server option behind other frameworks, rstats-BASEIMAGE-httpuv;
  • plumber: rstats-BASEIMAGE-plumber;
  • opencpu: rstats-BASEIMAGE-opencpu;
  • fiery: rstats-BASEIMAGE-fiery;
  • beakr: rstats-BASEIMAGE-beakr.

Template structure and workflows

• template.yml: yaml file for OpenFaaS, docker user, function name etc might have to be edited, see OpenFaaS config.
• Dockerfile: usually does not need editing.
• index.R: this file abstracts the server framework. Templates are set up for JSON in JSON out re/res types. This file only needs to be edited if the mime type depends on different midlewear, headers need to be changed etc.
• function/PACKAGES: file listing required packages and remotes to be installed for the function/handler.R file, plus additional sysreqs.
• function/handler.R: R script implementing the handler:
  • load libraries;
  • load data;
  • define functions/methods as needed;
  • it should include the handle = function(req, res) { ... } function called by index.R.

I really like how the r-minimal image implements remotes::install_deps() with the _R_SHLIB_STRIP_=true envvar. This behavior should be implemented for other templates through the Dockerfile as added by @mrchypark for the r-minimal template.

[psolymos]

✅ Need to have

Cassic watchdog:

• [x] rstats-base
• [x] rstats-ubuntu
• [x] rstats-minimal

of-watchdog:

• [x] rstats-base-plumber
• [x] rstats-ubuntu-plumber
• [x] rstats-minimal-plumber

Good to have

Rserve and alikes

❗ Rserve branch

Would be nice to have a bare bones solution with Rserve.

❗ RestRserve: issue | branch

• [ ] rstats-base-restrserve
• [ ] rstats-ubuntu-restrserve
• [ ] rstats-minimal-restrserve

Having JSON POST issues.

httpuv and alikes

❌ Rook

Rook serves web apps using R's help server and forces the routes to prepend /custom. There was a hack to overwrite tools:::httpd but we get locked binding of ‘httpd’ cannot be changed. The following starts, but R help says 'Only NEWS and URLs under /doc and /library are allowed':

library(Rook)
s <- Rhttpd$new()
s$add(RhttpdApp$new(system.file('exampleApps/RookTestApp.R',package='Rook'), name='httpd'))
s$start(listen='127.0.0.1', port=5000)

✅ httpuv: issue | branch

• [x] rstats-base-httpuv
• [x] rstats-ubuntu-httpuv
• [x] rstats-minimal-httpuv

✅ beakr: issue | branch

A nice minimalist framework, somewhere between httpuv and fiery in terms of complexity.

• [x] rstats-base-beakr
• [x] rstats-ubuntu-beakr
• [x] rstats-minimal-beakr

✅ fiery: issue | branch

• [x] rstats-base-fiery
• [x] rstats-ubuntu-fiery
• [x] rstats-minimal-fiery

❌ OpenCPU: issue | branch

Use local server, as a microservice. GET works, POST has similar JSON issues as RestRserve.

[psolymos]

Lack of websocket connection can be behind the problems identified (https://github.com/openfaas/of-watchdog/issues/20). Closing this for now.

[psolymos]

As it turns out, Rserve, RestRserve and opencpu works fine. of-watchdog needs http_buffer_req_body="true". See https://docs.openfaas.com/deployment/troubleshooting/#my-function-gets-a-nil-or-empty-body. This causes the HTTP request to be buffered in memory, then sent in one shot to the upstream function.