Open dereckmezquita opened 11 months ago
For some more context here is what I'm trying to do. This is an example of a toy REST API using RestRserve
. Note that RestRserve
has to be run with Rscript
and so should plumber
another REST API framework.
#!/usr/bin/env Rscript
box::use(RestRserve)
box::use(later)
box::use(jsonlite)
# Function to fetch and prepare data
fetch_data <- function() {
print("New data...")
# Fetch data from external API
data <- rnorm(100, 0, 1)
# Save data to a file or a database
saveRDS(data, "data.rds")
# Schedule the next data fetch
later::later(fetch_data, delay = 2)
}
# --------------------------------------------
# private event loop so as not to block RestRserve
private_loop <- later$create_loop()
later$with_loop(private_loop, {
later$later(fetch_data, delay = 2)
later$run_now()
})
# --------------------------------------------
# Function to serve data
serve_data <- function(.req, .res) {
# Load data from a file or a database
data <- readRDS("data.rds")
# Serve data
.res$set_body(data)
.res$set_content_type("application/json")
.res$set_status_code(200L)
}
# Create an application
app <- RestRserve$Application$new()
# Add a route to serve data
app$add_route("/", "GET", serve_data)
# Start the server
server <- RestRserve$BackendRserve$new()
server$start(app) # This will block the script
I discovered that if I ran the server in interactive mode (which I'm not supposed to do) and I can use the server and go to the end point. However the later loops are never ran. If I quit the process by doing CMD + C then the server shuts down and I can see my later process loop starting running.
Is RestRserve
monopolising the global event loop somehow? How can I get this working? I do note that this seems to be a different yet related issue to my original post.
server <- RestRserve$BackendRserve$new()
server$start(app) # This will block the script
{"timestamp":"2023-10-29 16:47:56.632314","level":"INFO","name":"Application","pid":64074,"msg":"","context":{"http_port":8080,"endpoints":{"GET":"/"}}}
-- running Rserve in this R session (pid=64074), 2 server(s) --
(This session will block until Rserve is shut down)
^CCaught break signal, shutting down Rserve.
[1] TRUE
Warning message:
In server$start(app) :
Starting RestRserve app from interactive session might cause unstable work of the service.
Please consider to start the application from a shell in non-interactive mode:
> Rscript your_app.R
r$> [1] "New data..."
r$> [1] "New data..."
r$> [1] "New data..."
[1] "New data..."
[1] "New data..."
[1] "New data..."
After experimenting, I was able to determine that it was indeed the RestRserve
server blocking the main event loop (I don't fully understand what's happening so I would appreciate someone to review my experiment/deductions @dselivanov).
I was able to put this together where I run the RestRserve
on a separate process with callr::r_bg
. This is not ideal and could cause problems with the multithreaded nature of RestRserve
. Moreover, in my mind that is exactly what later
is meant to solve (not blocking the event loop) @wch.
Does anyone know another way of doing this?
Note this script can also be written the reverse way where my "Hello world" loop is in the background process.
#!/usr/bin/env Rscript
# This works, the way this works is that the server is started in a separate R process
# and the main R process is used to run the event loop
box::use(later)
box::use(callr)
# Create a separate event loop
private_loop <- later$create_loop()
# Define the function to be called for the 2-second loop
print_hello_world <- function() {
print("Hello world")
later$later(print_hello_world, 2, loop = private_loop)
}
# Schedule the function to be called in the separate loop
later$later(print_hello_world, 0, loop = private_loop)
# --------------------------------------------
# Run the RestRserve backend in a separate R process using callr
server_process <- callr::r_bg(\() {
box::use(RestRserve)
# Create a new RestRserve application
app <- RestRserve$Application$new()
# Add the "echo" endpoint to the application
app$add_get("/echo", function(.req, .res) {
.res$set_body("hello")
.res$set_content_type("text/plain")
})
# Create a RestRserve backend
backend <- RestRserve$BackendRserve$new()
# Start the RestRserve backend
backend$start(app, http_port = 8080)
}, stdout = "server.log", stderr = "server.log")
# --------------------------------------------
# Run the separate event loop to print "Hello world" every 2 seconds
while (TRUE) {
later$run_now(loop = private_loop)
Sys.sleep(1)
}
Finally, I want to add that I thought about creating two separate scripts, one for my REST API and another for my interval logic. However, I need these to run together as I want to be able to receive commands from the API and pass them to my logic being run in the interval.
In short, I'm building a trading bot for academic purposes in R
/shiny
and Cpp
/Rcpp
, will publish open source :)
Edit: I am finally circling back to the original title of my post. If I use later
with Rscript
it doesn't run. It just opens and closes. I find I have to use a while
loop to keep it open and use later::run_now
. If I just use a while loop, then later
never executes. This defeats the purpose of later
, I could just call my logic/function in the while
loop.
Edit: I believe the issue with RestRserve
and running with Rscript
not working are two separate issues, should I create a separate issue for the Rscript
problem @wch?
I've done some experiments trying to keep the process open when using Rscript
by using an infinite while
. In my mind this should work - the while loop should run in the global event loop and then my later loops should run separately. Here's what I put together. I tried to mimic JavaScript
's behaviour.
I noticed any time I used an infinite while loop even in interactive mode later does not execute. Does later only work in interactive mode?
In summary:
If I run in interactive mode later
works nicely. If run in Rscript
then I need a way to keep the process open artifically, however, if I use an infinite while
loop then later
never executes.
library("later")
setInterval <- function(func, interval, loop) {
func <- rlang::as_function(func)
loop <- later::create_loop()
repeat_func <- function() {
func()
later::later(repeat_func, interval, loop = loop)
}
later::later(repeat_func, interval, loop = loop)
return(loop)
}
clearInterval <- function(id) {
later::destroy_loop(id)
}
setTimeout <- function(func, delay, loop, ...) {
func <- rlang::as_function(func)
loop <- later::create_loop()
later::later(\() {
func(...)
}, delay, loop = loop)
}
keep_alive <- function() {
while(TRUE) {
Sys.sleep(Inf)
}
}
# ----------------------------------------------
hello <- function() {
print("Hello world.")
}
loop <- setInterval(hello, 2, later::global_loop())
# if run interactively without `while` loop then this works
# we get "Hello world." and it stops after 5 seconds
setTimeout(\(hello) {
clearInterval(loop)
}, 5, later::global_loop(), hello = "Killing loop")
# but if run with `while` loop then we don't see "Hello world."
keep_alive()
Edit: I updated the title of my issue. Now I'm looking for a way to run later and RestRserve together and communicate between the two. Please skip to this comment:
https://github.com/r-lib/later/issues/179#issuecomment-1784275198
I will leave my previous comments for context.
Hello, I'm still learning about
later
and event loops. I would appreciate any help.I wrote this script to test
later
and create a "private event loop". I'm still learning about these. Is there any documentation somewhere with examples of how to use them? All I found was the help pages but with no examples - toy use case demos would be very helpful - I want to use it to get data from an API and run a REST API without blocking the global even loop (the REST API).I'm more familiar with just using
later::later
by itself but not with private event loops.Anywho, I wrote this script that runs well in interactive mode but only opens and closes with no printing if run with
Rscript
. In the end this script I want to write with the loop is meant to be server side code so I have to be able to run it usingRscript
.I plan on using this in a
RestRserve
server, so I need this to run without blocking the global event loop - hence, I started looking into private event loops.I tried to keep the global loop open by sleeping for 10 seconds; the initial print of "Hello world!" worked, but not my loop.