digital-fabric / polyphony

Fine-grained concurrency for Ruby
https://www.rubydoc.info/gems/polyphony
MIT License
655 stars 17 forks source link

Zlib::GzipReader raises "No thread set for fiber" #93

Closed floriandejonckheere closed 1 year ago

floriandejonckheere commented 1 year ago

When using Polyphony's I/O extensions, the following code raises an error after a seemingly arbitrary number of lines/bytes decompressed. It seems to vary depending on line length/number of lines and doesn't happen when using GzipReader#readlines. Perhaps a buffer being overrun somewhere?

require "zlib"
require "polyphony"

# Write a file with 100 lines of 100 000 characters each
File.open("test.gz", "w+") do |file|
  gz = Zlib::GzipWriter.new(file)
  gz.write("#{'a' * 10_000}\n" * 1000)
  gz.close
end

# Read the file
gz = Zlib::GzipReader.open("test.gz")

it = gz.each_line

loop do
  it.next
rescue StopIteration
  break
rescue
  puts "Crashed at line #{gz.lineno}"

  raise
end
$ ruby test.rb
Crashed at line 166
test.rb:in `each_line': No thread set for fiber (RuntimeError)
    from test.rb:in `each'
    from test.rb:in `each'
noteflakes commented 1 year ago

The following works fine:

gz = Zlib::GzipReader.open('/tmp/test.gz')
gz.each_line do |l|
rescue
  trace "Crashed at line #{gz.lineno}"
  raise
end

There's some weird interaction between the Enumerator and the Polyphony. I'm investigating...

noteflakes commented 1 year ago

Of course, Enumerator uses fibers to do external iteration. A fix is forthcoming.

floriandejonckheere commented 1 year ago

Thanks for the quick fix!