search

parroty / exvcr

HTTP request/response recording library for elixir, inspired by VCR.
MIT License
722 stars 132 forks source link

ExVCR seems to play back responses just recorded during a recording session #78

Open myronmarston opened 7 years ago

myronmarston commented 7 years ago

I'm working on a test for an API that is stateful and does not always return the same response for the same request. The code I am testing makes the same request multiple times expecting different responses because the API behaves that way, but ExVCR is getting in the way. When the 2nd request happens, it appears to playback the response for the 1st request, even though the cassette does not even exist when I run my test--at least, my test is failing due to getting an identical response as was received on the first request, and the recorded cassette contains only one interaction.

To provide a more concrete example, think about an banking API that provides /current-balance and /send-payment endpoints. One can imagine writing a test that does this sequence:

  1. Calls /current-balance and gets the current balance.
  2. Calls /send-payment with n dollars.
  3. Calls /current-balance and asserts the new balance is original_balance - n.

Can ExVCR be fixed so that no playback happens when the cassette did not exist at the point the test began, no playback occurs? Given that APIs can be stateful it is often necessary to make the same request multiple times as part of a test, expecting different responses.

coop commented 6 years ago

@myronmarston I have the same issue. In https://github.com/parroty/exvcr/issues/104 @parroty suggests using query parameters to distinguish requests... Unfortunately that isn't a solution for me. Did you come up with a workaround?

myronmarston commented 6 years ago

@coop I've just used multiple cassettes within a single test so that when duplicate requests are made, it's made to a new cassette that will record it.

coop commented 6 years ago

Clever. Thanks.

jcypret commented 6 years ago

I ran into a similar issue when working with the Amazon MWS reports API. A report is requested, then a different endpoint polled using the request ID until the response has a status of "done" along with the report id for download. In this case, the same request is made but returns different results over time. The query param hack doesn't work in this case because with Amazon you have to sign the entire request and cannot add extra params.

I like @myronmarston 's idea of using multiple cassettes. I'll give that a try.

lmarlow commented 6 years ago

We've gone down the multiple cassette path, too. I'm wondering if it would be better to not use the cache at all if the cassette file didn't exist when the test started.

joevandyk commented 5 years ago

I'm running into this as well - I agree with @lmarlow that the cache shouldn't be used if it's recording.