Different results from Betamax vs Requests

[facetoe]

Hi, I'm having an issue with a failing test and I'm not sure how to debug it further. When I execute the following code in my IDE:

ticket_generator = zenpy_client.tickets()
tickets = ticket_generator[99:101]
print(tickets)

These requests are generated:

DEBUG - https://d3v-zenpydev.zendesk.com:443 "GET /api/v2/tickets.json?&include=users,groups,organizations,last_audits,metric_sets,dates,sharing_agreements,comment_count,incident_counts,ticket_forms,metric_events,slas HTTP/1.1" 200 None
DEBUG - https://d3v-zenpydev.zendesk.com:443 "GET /api/v2/tickets.json?include=users%2Cgroups%2Corganizations%2Clast_audits%2Cmetric_sets%2Cdates%2Csharing_agreements%2Ccomment_count%2Cincident_counts%2Cticket_forms%2Cmetric_events%2Cslas&page=2&per_page=100 HTTP/1.1" 200 None

Note the page=2 parameter in the last request.

These requests result in the expected output: [Ticket(id=100), Ticket(id=101)]

However, when I execute the following test using Betamax:

    def test_ticket_slice_cross_page_size_boundry(self):
        with self.recorder.use_cassette(self.generate_cassette_name(), serialize_with='prettyjson'):
            ticket_generator = self.zenpy_client.tickets()
            values = ticket_generator[99:101]
            self.assertTrue(len(values) == 2)
            for i, n in enumerate(range(100, 102)):
                self.assertIsInstance(values[i], Ticket)
                self.assertTrue(values[i].id == n, msg="expected Ticket id: {}, found: {}, values: {}".format(n, values[i], values))

It fails with the error: AssertionError: expected Ticket id: 101, found: Ticket(id=1), values: [Ticket(id=100), Ticket(id=1)]

Looking at the requests saved by Betamax, I notice something funny:

⇒ grep uri tests/test_api/betamax/TestTicketGeneratorSlice.test_ticket_slice_cross_page_size_boundry.json
        "uri": "https://d3v-zenpydev.zendesk.com/api/v2/tickets.json?&include=users,groups,organizations,last_audits,metric_sets,dates,sharing_agreements,comment_count,incident_counts,ticket_forms,metric_events,slas"
          "Strict-Transport-Security": "max-age=31536000;",
        "uri": "https://d3v-zenpydev.zendesk.com/api/v2/tickets.json?include=users%2Cgroups%2Corganizations%2Clast_audits%2Cmetric_sets%2Cdates%2Csharing_agreements%2Ccomment_count%2Cincident_counts%2Cticket_forms%2Cmetric_events%2Cslas&per_page=100&page=1"
          "Strict-Transport-Security": "max-age=31536000;",

Note the page=1 parameter in the last request. I'm confused as to why the generated requests differ?

The page and per_page parameters are passed to requests via the params dict, if that helps at all.

Any idea what's going on here? My Betamax instance is configured as follows:

def configure():
    config = Betamax.configure()
    config.cassette_library_dir = "tests/test_api/betamax/"
    config.default_cassette_options['record_mode'] = 'once'
    config.default_cassette_options['match_requests_on'] = ['method', 'path_matcher']
    if credentials:
        auth_key = 'token' if 'token' in credentials else 'password'
        config.define_cassette_placeholder(
            '<ZENPY-CREDENTIALS>',
            str(base64.b64encode(
                "{}/token:{}".format(credentials['email'], credentials[auth_key]).encode('utf-8')
            ))
        )
    session = requests.Session()
    credentials['session'] = session
    zenpy_client = Zenpy(**credentials)
    recorder = Betamax(session=session)

    class PathMatcher(URIMatcher):
        """
        I use trial accounts for testing Zenpy and as such the subdomain is always changing.
        This matcher ignores the netloc section of the parsed URL which prevents the tests
        failing when the subdomain is changed.
        """
        name = 'path_matcher'

        def parse(self, uri):
            parse_result = super(PathMatcher, self).parse(uri)
            parse_result.pop('netloc')
            return parse_result

    Betamax.register_request_matcher(PathMatcher)
    recorder.register_serializer(PrettyJSONSerializer)
    return zenpy_client, recorder

Version information:

⇒ pip freeze | grep 'betamax\|requests'
betamax==0.8.0
betamax-matchers==0.4.0
betamax-serializers==0.2.0
requests==2.18.4
requests-toolbelt==0.8.0

⇒ python --version
Python 3.6.3

Any help would be greatly appreciated!

[hroncok]

Does this fail when recording the cassette, when playing it, or in both cases?

[facetoe]

Yep, it fails in both cases.

[hroncok]

I assume this is from https://github.com/facetoe/zenpy could you share the code entirely? I.e. push to a wip branch etc.

[facetoe]

Sure, no worries, I've pushed to this branch: https://github.com/facetoe/zenpy/tree/betamax_test

[hroncok]

From the failing tests in on Travis, it seems betamax is trying to handle the correct request:

A request was made to https://d3v-zenpydev.zendesk.com/api/v2/tickets.json?include=users%2Cgroups%2Corganizations%2Clast_audits%2Cmetric_sets%2Cdates%2Csharing_agreements%2Ccomment_count%2Cincident_counts%2Cticket_forms%2Cmetric_events%2Cslas&&page=2&per_page=100 that could not be found in TestTicketGeneratorSlice.test_ticket_slice_cross_page_size_boundry.

[facetoe]

I'm pretty sure this is a bug in my code somehow (surprise surprise!) and not Betamax. When it executes under test it generates the parameter page=1, yet when it executes in my IDE or on Travis it generates page=2... Going to need to do some digging.

Thanks for your help!

[facetoe]

Aha! Looks like nosetests executes python2, and I can recreate that same bug by running my code under python2, I'm not going mad after all! So nothing to do with Betamax after all :)

[sigmavirus24]

Glad you figured it out @facetoe! For what it's worth, I think you might also want to match on the query string (you're currently matching on the path and method) but you know your code better so feel free to ignore that suggestion :)

[hroncok]

Also, I see some recorded authorization headers in the cassettes, not sure if real or fake.

[facetoe]

@sigmavirus24, yes good point, I should be matching on query also. Thanks!

@hroncok thanks for the heads up, I'll fix that.