ruby / spec

The Ruby Spec Suite aka ruby/spec
MIT License
599 stars 388 forks source link

Better testing for finalization #935

Open headius opened 2 years ago

headius commented 2 years ago

While attempting to write a spec for jruby/jruby#7267 I ran into various issues and questions...

  1. There are no specs testing that GC eventually finalizes objects. This is obviously difficult to predict, but it is behavior I believe we should be testing one way or another.
  2. I was testing that exceptions in one finalizer are not seen by the next finalizer, but could not figure out a way to eliminate the warning output from Ruby indicating that a finalizer raised an exception.

The spec I attempted is below, but only makes a best attempt at forcing GC-oriented finalization and still does not suppress the error output.

  it "hides raised exceptions from one finalizer to the next" do
    def scoped(result)
        Proc.new { result << "ok" if $!.nil?; raise }
    end
    def test(result)
      obj = "Test"
      # finalizer order may vary so both handlers check $! and raise an error
      ObjectSpace.define_finalizer(obj, scoped(result))
      ObjectSpace.define_finalizer(obj, scoped(result))
    end

    result = []
    begin
      old_verbose, $VERBOSE = $VERBOSE, false
      test(result)
    ensure
      $VERBOSE = old_verbose
    end

    100.times { GC.start; break if result.size == 2 }

    result.should == ["ok", "ok"]
  end

I don't want to leave the fix for jruby/jruby#7267 untested, but I'm unsure how we should move forward to improve the GC-triggered finalization specs.

eregon commented 2 years ago

We've tried in the past and it seems near-impossible to have a reliable test for this unfortunately. PR welcome of course if you find one way.

GC in a loop is also something I tried but it's 1) very slow 2) still doesn't finalize sometimes. Also on CRuby because of the conservative GC it's completely possible some object never gets finalized even though it is semantically unreachable.

My take is: no one should rely on finalizers on Ruby, they run unreliably (on all Ruby impls) and always late. Releasing resources should always be done promptly with an ensure or block pattern .

See https://github.com/ruby/spec/blob/8059e5955fc92fc28cd12910c9f371361e7170e8/core/objectspace/define_finalizer_spec.rb#L4-L10

TruffleRuby also tried using TruffleRuby-specific methods but even that spuriously fails: https://github.com/oracle/truffleruby/blob/4054469bbd985db7dc5c86299004ff86a2835baf/spec/truffle/objspace/define_finalizer_spec.rb#L17-L27

eregon commented 2 years ago

One possibility might be to rely on the fact CRuby runs finalizers on exit, and use that as a way of testing finalizers aspects other than "they happen when the object is GC'd". However, IIRC that's actually tricky to implement as there might be ordering rules between objects (based on what they reference) and which object should finalize first. I'm not sure how CRuby deals with that.