socketry / async

An awesome asynchronous event-driven reactor for Ruby.
MIT License
2.04k stars 85 forks source link

Allow queue to be used for task `finished`. #276

Open ioquatix opened 10 months ago

ioquatix commented 10 months ago
#!/usr/bin/env ruby

require_relative 'lib/async'
require_relative 'lib/async/queue'

Async do
  queue = Async::Queue.new

  10.times do |i|
    Async(finished: queue) do
      # raise "Boom" if rand > 0.5
      sleep(i)
    end
  end

  queue.each do |task|
    puts task.wait
  end
end

However, it's not clear how to close the queue to indicate no more jobs will be added.

emiltin commented 10 months ago

I'm sure I understand how this would be used to make a barrier fail fast in case a task fail, as discussed in #272.

Is the idea that all tasks in the barrier would use the same condition to signal when they are done, so that the barrier will be notified when a task finish, and potentially fail fast? I think that would have the side effect that if some other code wait for a particular task in the barrier, then that wait would end whenever any task in the barrier finishes?

emiltin commented 10 months ago

However, it's not clear how to close the queue to indicate no more jobs will be added.

What about a drain method, which works like each, except it returns as soon as the queue is empty?

emiltin commented 10 months ago

What about a drain method, which works like each, except it returns as soon as the queue is empty?

To clarify - this would imply that tasks are added to the queue when created, and are removed when they finish. drain would yield each task as they complete, returning when there is no more items.

The current PR works differently: it adds task to the queue only when finished. each waits for new items to be added, which are then removed and yielded. It never returns.

ioquatix commented 9 months ago

I have not had time to revisit this yet, but I will in the coming week.