timeout or updater function

[daroczig]

Hey @nfultz,

I'm thinking about a way to schedule some updates within the server function, so that eg I can keep up-to-date the cached data loaded at startup time. Image eg I load the iris dataset and build a model on server startup and using the model for scoring in a method, but I'd like to retrain the model from time to time.

Currently, I can achieve that by running the app behind a load balancer and killing one of the apps from time to time. Another way would be to add an Updater method and calling that from a scheduler -- but none of these are optimal.

In the AWR.Kinesis package, I do this by providing updater functions, which takes a list of frequencies and functions to run, and the server evaluates the functions at the provided frequencies: https://github.com/daroczig/AWR.Kinesis#writing-a-record-processor-application

I'd love to see such functionality in grpc as well :) Eg if we could run an R function inside of start_server every 60 mins or so.

But another, a probably much simpler solution would be to provide a timeout argument to start_server, which would automatically shut down after the provided number of minutes or similar -- that would resolve the auto-update feature behind an app load balancer pretty easily.

What do you think?

[daroczig]

Workaround until then:

system(paste0('(sleep 60;kill ', Sys.getpid(), ') &'))

[nfultz]

Part of me feels like that is mixing application code in to the server - there is already a mechanism for the server to run callbacks, so what about adding an update method to the server?

eg for iris, psuedocode:

service Classifier {
  rpc Classify (Features) returns (Class) {}
  rpc Update (UpdateOptions) returns (UpdateStatus) {}
} 

then on the server (or anywhere), you could instantiate a update client -

update_client <- grpc_client(impl, 'localhost:50051')
update_options <- client$Update$build('foo')

update_proc <- mcparallel(repeat{
  Sys.sleep(60)
  client$Update(update_options)
})

##...

globalState <- c()

impl$Update$f <- function(update_request) {
  globalState <<- foo()
  newResponse(status = "bar")
}

### run the server here

Currently everything runs in the same thread, so I believe Update$f can write to the global environment and have the changes picked up in later calls to Classify$f.

There's some existing protobufs for this kind of thing to use as reference, but GRPC is a bit harder to find - https://github.com/google/protobuf/blob/master/src/google/protobuf/empty.proto https://github.com/dsoprea/RestPipe/blob/master/rpipe/resources/protobuf/heartbeat.proto etc

Your update interval is very high so it doesn't matter as much, but faster updates (eg heartbeats every second) would also probably benefit from stream support to minimize connection churn.

See also https://github.com/nfultz/suicide for my version of self termination ;) - also not sure what would need to change to support setTimeLimit() but I would probably go that way rather than adding a timeout= option to the server directly.

From the manual -

     Time limits are checked whenever a user interrupt could occur.
     This will happen frequently in R code and during 'Sys.sleep', but
     only at points in compiled C and Fortran code identified by the
     code author.

[daroczig]

Oh, thanks for the pointer on suicide -- doing exactly the same as the above workaround, but in a much more elegant way :+1: Any chance it might end up on CRAN? :)

Re Updater method: yeah, I was thinking about it, but the update part would be blocking the whole process, so eg if you use the other methods for live scoring, then there would be a significant delay when the update happens .. Also, if the grpc service runs behind a load balancer to avoid downtimes (eg when update happens), it would be difficult to update the thread with the outdated cache -- so it's just much more easier to kill a service from time to time behind a load balancer and let the killed proceesed respawn with fresh data => resulting in no downtime or slowdown at all at the overall service.

Anyway, I understand you are not happy with this solution :) I think I will stick with the suicide option -- it worked really well for me on Kinesis consumers.

[daroczig]

As per the above, I'm totally OK closing this.

[nfultz]

Also FYI,

setTimeLimit(elapsed=10)

will stop the server gracefully after 10s.

[daroczig]

Yeah, or even more setSessionTimeLimit before start_server