kiwicom / pytest-recording

A pytest plugin that allows recording network interactions via VCR.py
MIT License
445 stars 35 forks source link

new_episodes doc ? #36

Closed asmodehn closed 4 years ago

asmodehn commented 4 years ago

I wonder what is the intended logic behind the 'new_episodes' option ? I couldn't find it in the README, and I had some "strange" behavior with it...

I ran multiple requests to the same endpoint, in one vcr with --record-mode=new_episodes. But only the first attempt of each request was registered.

So something like :

import time

import pytest
import aiohttp

@pytest.mark.asyncio
@pytest.mark.vcr()
async def test_multi_call():
    async with aiohttp.ClientSession() as session:
        async with session.get('http://httpbin.org/get') as resp:
            print(resp.status)
            print(await resp.text())

        async with session.get('http://httpbin.org/ip') as resp:
            print(resp.status)
            print(await resp.text())

        time.sleep(1)
        async with session.get('http://httpbin.org/get') as resp:
            print(resp.status)
            print(await resp.text())

        async with session.get('http://httpbin.org/ip') as resp:
            print(resp.status)
            print(await resp.text())

if __name__ == '__main__':
    #pytest.main(['-s', __file__, '--block-network'])
    # record run
    pytest.main(['-s', __file__,'--record-mode=new_episodes'])

Will store only 2 requests/responses, one for http://httpbin.org/get and one for http://httpbin.org/ip.

I was naively expecting to record all requests/responses in one run, and then record new cassettes if a difference between the registered cassettes and the current "run" was found (calling to one more url, or not calling an expected url, etc.)...

So does "episode" means "url" and not "run" ? Thanks for any info.

jraregris commented 4 years ago

I wondered about this, too. I found some docs on vcr.rb, https://relishapp.com/vcr/vcr/v/5-1-0/docs/record-modes/new-episodes and assume this is the same behaviour.

Stranger6667 commented 4 years ago

Sorry for not responding here for way too long.

Yes, pytest-recording relies on the VCR-py behavior, a port of Ruby's VCR gem. Unfortunately, there is not much about this mode in the VCR-py docs, and I think that it would be better to improve it there and avoid duplication on the pytest-recording side. However, we can add a reference to the VCR-py docs in our README.

So does "episode" means "url" and not "run" ?

It depends on how request matchers are configured. For example, there could be a custom matcher, and VCR will decide if it is the same "episode" depending on the current set of matcher for a particular test. But yes, it is the default behavior according to their docs:

The default behavior is ['method', 'scheme', 'host', 'port', 'path', 'query'] which means that requests with both the same URL and method (ie POST or GET) are considered identical.

I will add this reference note to the #49 TODO list and close this issue. Feel free to reopen it in case of any further questions