Open rmc3 opened 8 years ago
hi @rmc3 - sorry for not getting back to you sooner. you're correct, for the most part. "blocking" IO is supposed to "pause" and allow other tasks to run, but lately I don't have a good pulse on the Reel/Celluloid community. Compute tasks that should be in another thread (another Actor object) are blocking responses. There have been many changes to celluloid world, and also some initial misunderstandings by me in the design of Angelo.
Can you tell me which ruby engine you're using? If not JRuby, you may have better luck with that; though I will warn you the websocket tests do not seem to work on JRuby.
I have also written some node.js lately although with Hapi, not Express. I understand what you are trying to do. I'm not sure that it could be accomplished in Angelo using the task/async/future features, since those define tasks on the reactor itself, which is also running the service. I experimented with adding an actor pool to the server by default, for these kinds of things, but obviously not done.
@kenichi, I've been demoing Angelo with MRI. I'm happy to try it out with JRuby, but I'm not certain what this would change. Can you elaborate what differences I should see on the with blocking operations on JRuby, or anything I'd need to do differently with Angelo with JRuby to see these differences?
@rmc3 Here is an example. Some things to note are:
task
builder)#!/usr/bin/env ruby
require 'angelo'
# require 'httpclient'
class Worker
include Celluloid
def fib n = 40
n <= 1 ? n : fib(n-1) + fib(n-2)
end
end
class Futures < Angelo::Base
class << self
def worker
@worker ||= Worker.new
end
end
get '/a' do
"hi\n"
end
get '/b' do
Futures.worker.fib.to_s + "\n"
end
get '/c' do
future(:io).value.to_s + "\n"
end
task :io do
body = ''
sock = Celluloid::IO::TCPSocket.new 'example.com', 80
begin
sock.write "GET /big_file.tgz HTTP/1.1\n" +
"Host: example.com\n" +
"Connection: close\n\n"
while data = sock.readpartial(4096)
body << data
end
rescue EOFError
ensure
sock.close
end
# hc = ::HTTPClient.new
# body = hc.get('http://example.com/big_file.tgz').body
body.length
end
end
Futures.run!
GETing /a will still respond with 'hi', even while the another request is GETing /b, or another is GETing /c. /b requests will block serially due to there being only one worker thread.
The commented httpclient lines show how not using celluloid-io sockets will block. I had to write a GET request by hand using Celluloid::IO:TCPSocket
because apparently the http gem is not using them anymore... ?
I had to write a GET request by hand using Celluloid::IO:TCPSocket because apparently the http gem is not using them anymore... ?
That sounds like the behavior I observed too, if it resulted in HTTP requests blocking. Thanks for the example! It may be worth adding something similar to this to the README or wiki for people who are also looking into Angelo for a similar use case.
Having gotten used to writing REST APIs in Node.js with Express, I'm accustomed to being able to kick off asynchronous tasks when an endpoint is hit and not send the HTTP response until a callback is called. I've failed to find any documentation that gives insight into how to do something similar with Angelo (or the underlying Reel/Celluloid).
With the documented
tasks
andasync
/futures
, it seems that either the caller can't get the result of the task (async) or the caller can get the result of the task at the cost of blocking all other execution until the task is complete (futures). As a result, neither method seems to suit my goal of doing some non-blocking calculation or IO when an endpoint is hit and not returning a response to the client until the calculation is complete. It seems that my goal is not possible with Angelo. Is this correct?Please feel free to close this issue if you feel it's irrelevant, I wasn't sure how else to get in contact with the Angelo community.