digital-fabric / libev_scheduler

A libev-based fiber scheduler for Ruby 3.0
MIT License
34 stars 3 forks source link

Wrong argument type Array (expected fiber) #4

Open wjordan opened 3 years ago

wjordan commented 3 years ago

The following code consistently produces an error, in 'close': wrong argument type Array (expected fiber) (TypeError):

require 'libev_scheduler'
Fiber.set_scheduler Libev::Scheduler.new

Fiber.schedule do
  finish = false
  Fiber.schedule do
    sleep 0 until finish
  end
  sleep 0.1
  finish = true
end

This particular error doesn't come up with other fiber-scheduler implementations so it doesn't seem to be a ruby-core issue. My first guess is that the fiber pointer is getting clobbered by Ruby GC and that some RB_GC_GUARD statements might need to be added somewhere, but I haven't been able to figure it out on my own.

kingdonb commented 2 years ago

Hey, I wondered if I should write in here or create another issue, because I have failed to understand something about how this scheduler works and if I'm observing something wrong with this implementation or if I'm just not using it well

I run this process that I built on fibers and it has some shell-outs

run sync
Setting scheduler to Libev
calling Fiber.schedule for do_update loop
MyCLI::sync scheduled Fibers at 2022-06-20 14:21:50 -0400
finished
/Users/kingdonb/.rvm/rubies/ruby-3.0.4/lib/ruby/3.0.0/net/protocol.rb:219:in `wait_readable': no implicit conversion of Libev::Scheduler into Integer (TypeError)

If I do RestClient.get, or Open3.popen3, or Net::HTTP.get, or pretty much anything that forks a process (surprisingly though, exec via backticks is OK) I get this error message

@wjordan What other fibers scheduler implementations did you try that worked better for you? This one is working great as long as I am careful and don't do any of those things inside of the fibers. They are blocking operations, I suppose, so have to be undertaken in the main thread, but that doesn't work for me, and I didn't exactly get that from reading all of the thoughts / etc notes in this repo.

It is a different error message but there is only this one issue open in the whole repo so, forgive me for intrusion if this is not the right place to ask, but I saw this one had no response, (and I figured that noise may be better corralled than distributed!)

(The code in context: https://github.com/kingdonb/simplest-commitbee/pull/16)

If I try to fetch the URL from within a fiber, I always get this error, I guess Ruby only knows how to fetch HTTP via a fork (or whatever makes this fail? IDK, not understanding fully though it's pretty clear) – if I make this change to call a script, which runs curl, and curl writes to a file, and the Ruby code just reads a file, then it works just fine with this fibers implementation! Since I'm not blocking on a network read, near as I can tell, but blocking on a script execution instead, which is OK I guess.

Edit: I tried the FiberScheduler implementation at https://github.com/bruno-/fiber_scheduler and it seems to not have this issue at least as far as I can tell. Requires an upgrade to Ruby 3.1 but at first glance it looks like it just works in these cases where I had no idea why libev scheduler does not work.

kingdonb commented 2 years ago

I suspect my issue has something to do with this paragraph: https://github.com/digital-fabric/libev_scheduler/blob/main/thoughts.md#waiting-for-processes not sure if that's the same issue as yours or not.

I don't exactly understand the relationship between polyphony and the libev_scheduler, but it looks like we're intended to use polyphony instead of accessing this gem directly? (Does polyphony use this codebase as a testbed, or is it related at all to the polyphony scheduler?)

If I had started with the tutorial for polyphony I think I might have had a better time with this. In the end I switched for bruno's FiberScheduler and upgraded to Ruby 3.1, this covered the 20% of cases I had which were not working with libev_scheduler and made them work without any other changes. I looked at the polyphony doc and found it clear, but hard to tie back to the fiber scheduler interface as it seems to lean heavily on the spin kernel method, rather than Fiber.schedule

I really gained a lot by reading the examples here in the libev_scheduler repo. They were just simple enough to get me started and just complex enough to get me over the line, to where I could isolate that 20% of things which weren't working.

Sorry for contributing noise on this repo, it seems now that I understand my own issue better, it is almost definitely not the same issue. I can open a different issue if someone is interested in that.

noteflakes commented 2 years ago

I looked at this issue a bit yesterday but could not find the culprit. FWIW if you change from sleep 0 to sleep 0.001 the problem seems to disappear.