Open wjordan opened 3 years ago
Hey, I wondered if I should write in here or create another issue, because I have failed to understand something about how this scheduler works and if I'm observing something wrong with this implementation or if I'm just not using it well
I run this process that I built on fibers and it has some shell-outs
run sync
Setting scheduler to Libev
calling Fiber.schedule for do_update loop
MyCLI::sync scheduled Fibers at 2022-06-20 14:21:50 -0400
finished
/Users/kingdonb/.rvm/rubies/ruby-3.0.4/lib/ruby/3.0.0/net/protocol.rb:219:in `wait_readable': no implicit conversion of Libev::Scheduler into Integer (TypeError)
If I do RestClient.get
, or Open3.popen3
, or Net::HTTP.get
, or pretty much anything that forks a process (surprisingly though, exec via backticks
is OK) I get this error message
@wjordan What other fibers scheduler implementations did you try that worked better for you? This one is working great as long as I am careful and don't do any of those things inside of the fibers. They are blocking operations, I suppose, so have to be undertaken in the main thread, but that doesn't work for me, and I didn't exactly get that from reading all of the thoughts / etc notes in this repo.
It is a different error message but there is only this one issue open in the whole repo so, forgive me for intrusion if this is not the right place to ask, but I saw this one had no response, (and I figured that noise may be better corralled than distributed!)
(The code in context: https://github.com/kingdonb/simplest-commitbee/pull/16)
If I try to fetch the URL from within a fiber, I always get this error, I guess Ruby only knows how to fetch HTTP via a fork (or whatever makes this fail? IDK, not understanding fully though it's pretty clear) – if I make this change to call a script, which runs curl, and curl writes to a file, and the Ruby code just reads a file, then it works just fine with this fibers implementation! Since I'm not blocking on a network read, near as I can tell, but blocking on a script execution instead, which is OK I guess.
Edit: I tried the FiberScheduler
implementation at https://github.com/bruno-/fiber_scheduler and it seems to not have this issue at least as far as I can tell. Requires an upgrade to Ruby 3.1 but at first glance it looks like it just works in these cases where I had no idea why libev scheduler does not work.
I suspect my issue has something to do with this paragraph: https://github.com/digital-fabric/libev_scheduler/blob/main/thoughts.md#waiting-for-processes not sure if that's the same issue as yours or not.
I don't exactly understand the relationship between polyphony and the libev_scheduler, but it looks like we're intended to use polyphony instead of accessing this gem directly? (Does polyphony use this codebase as a testbed, or is it related at all to the polyphony scheduler?)
If I had started with the tutorial for polyphony I think I might have had a better time with this. In the end I switched for bruno's FiberScheduler and upgraded to Ruby 3.1, this covered the 20% of cases I had which were not working with libev_scheduler and made them work without any other changes. I looked at the polyphony doc and found it clear, but hard to tie back to the fiber scheduler interface as it seems to lean heavily on the spin
kernel method, rather than Fiber.schedule
I really gained a lot by reading the examples here in the libev_scheduler repo. They were just simple enough to get me started and just complex enough to get me over the line, to where I could isolate that 20% of things which weren't working.
Sorry for contributing noise on this repo, it seems now that I understand my own issue better, it is almost definitely not the same issue. I can open a different issue if someone is interested in that.
I looked at this issue a bit yesterday but could not find the culprit. FWIW if you change from sleep 0
to sleep 0.001
the problem seems to disappear.
The following code consistently produces an error,
in 'close': wrong argument type Array (expected fiber) (TypeError)
:This particular error doesn't come up with other fiber-scheduler implementations so it doesn't seem to be a ruby-core issue. My first guess is that the fiber pointer is getting clobbered by Ruby GC and that some
RB_GC_GUARD
statements might need to be added somewhere, but I haven't been able to figure it out on my own.