search

pact-foundation / pact-ruby

Enables consumer driven contract testing, providing a mock service and DSL for the consumer project, and interaction playback and verification for the service provider project.
https://pact.io
MIT License
2.17k stars 216 forks source link

Recent versions of Oj (3.11.4 and 3.11.5) are causing issues with the Pact library #240

Closed iamvery closed 3 months ago

iamvery commented 3 years ago

👋 I'll say up front that this was difficult to narrow down, and I'm still not certain of the root cause. It's unclear to me whether it's ultimately on the Oj or Pact side.

Problem

After updating to Oj 3.11.4 and 3.11.5, we are seeing intermittent test failures for Pact examples. The failures appear to be a result of how objects are serialized. On oj 3.11.3, responses present like:

{ "some_field" => #<Pact::SomethingLike...> }

But after upgrading certain responses later in test runs present like:

{ "some_field" => { "json_class" => "Pact::SomethingLike", ... }

Note: the issue is currently presenting itself for certain RSpec "seeds" and often happens after some successfully and correctly serialized response bodies. For example, the first Pact test will run correctly and successfully and a subsequent one is serialized incorrectly as shown above which results in failure.

Thank you for your work on this library, and please let me know if I can provide more information. Unfortunately at this time, I'm not certain how to reproduce the issue outside of our code and test suite, but I'm hopefully the reporting it may trigger some thought on recent changes.

» ruby -v
ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-darwin19]
» rails -v
Rails 6.1.3.1

See also https://github.com/ohler55/oj/issues/656

zorab47 commented 3 years ago

Possibly related to this previous issue: https://github.com/pact-foundation/pact-ruby/issues/166

bethesque commented 3 years ago

Thanks for the details. Until you can reproduce it reliably in a codebase you can share with me, I don't think there's much I can do to fix it unfortunately. My gut feeling based on the previous issues people have had, that you've already found, is that there will be a specific OJ setting that needs to be set to fix this.

iamvery commented 3 years ago

I tried removing our lock on Oj and running things again, and it appears that something in a recent release has fixed the issue.

iamvery commented 3 years ago

I spoke too soon. The failures are order dependent so a passing build gave me false confidence 😞

iamvery commented 3 years ago

For reference, a script to reproduce this is available here: https://gist.github.com/zorab47/f00e4ad8d5b02b583c2ba97db6011f67

bethesque commented 3 years ago

I'll have another look at it on my next OSS day.

YOU54F commented 3 months ago

Thanks everyone for all the investigations and the excellent reproducer!

This is now resolved as per comments here

https://github.com/ohler55/oj/issues/656#issuecomment-1696028716

the original example has been updated.

tl;dr - if you use OJ, set Oj.default_options = { create_additions: true }