grosser / parallel

Ruby: parallel processing made simple and fast
MIT License
4.15k stars 255 forks source link

Incorrect exit code #232

Open nlastovich opened 5 years ago

nlastovich commented 5 years ago

I am trying to execute two rake tasks in parallel in next way: task :run_parallel_regression do Parallel.map(%i[thread1 thread2]) do |task| Rake::Task[task].invoke raise Parallel::Break end end

If thread1 will be finished first and the result will fail but thread2 will succeed Parallel.map exit code will be 0. What I should change in my code to make Parallel.map exit with code 1 if one of the sub-processes failed?

grosser commented 5 years ago

Can you simplify this further, for example:

Does this also happen when using a plain Ruby script without rake ? Does it also happen when not using the 'raise brake' ?

On Thu, Dec 20, 2018, 13:47 Nikita Lastovych <notifications@github.com wrote:

I am trying to execute two rake tasks in parallel in next way: task :run_parallel_regression do Parallel.map(%i[thread1 thread2]) do |task| Rake::Task[task].invoke raise Parallel::Break end end

If thread1 will be finished first and the result will fail but thread2 will succeed Parallel.map exit code will be 0. What I should change in my code to make Parallel.map exit with code 1 if one of the sub-processes failed?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/grosser/parallel/issues/232, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAsZ33GYkimj3YLmuUWzOmUCU6mLSV0ks5u64bNgaJpZM4ZcIOu .

nlastovich commented 5 years ago

If I will not use raise break I will get this error even if both sub-processes succeed `rake aborted! Parallel::DeadWorker: Parallel::DeadWorker

Caused by: EOFError: end of file reached`

Not really sure how to implement it without rake tasks. The interesting thing that if last finished sub-process will be failed exit code will be correct. Like it is ignoring all previously finished sub-processes

grosser commented 5 years ago

so this works:

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'parallel'
end

Parallel.map([1,2]) { raise }
test.rb:8:in `block in <main>': unhandled exception

... the same with a Rakefile:

require 'parallel'

task :foo do
  Parallel.map([1,2]) { raise "hi" }
end

rake foo
rake aborted!
hi

then nested rake task ... getting a little strange, but still kinda as expected:

task :bar do
  raise "hi"
end

task :foo do
  Parallel.map([1,2]) do
    Rake::Task["bar"].invoke
  end
end

rake foo
rake aborted!
Parallel::UndumpableException: RuntimeError: hi

and ordering:

task :foo do
  Parallel.map([1,2]) do |i|
    raise "hi" if i == 1
  end
end

task :bar do
  Parallel.map([1,2]) do |i|
    raise "hi" if i == 2
  end
end

both work as expected ... so need a better example, afaik everything works fine ...