Intermittent issues accessing release.data items

[rexovas]

Hi, me again.

A while back I had this handy bit of code which was working great

        query_dicts = [
            {'release_id': release.id, 'format': release_format}
            for release_format in release.data["format"]
            if release_format in VALID_FORMATS
        ]

Given a release, this code is supposed to construct a list of dictionaries that look like this

[{'release_id': 44872, 'format': '12"'}, {'release_id': 44872, 'format': 'Vinyl'}, {'release_id': 44872, 'format': '45 RPM'}]

Suddenly - this stopped working. I'm not sure if it's because I switched from using release objects returned via an api search, to simply obtaining them via release_id lookup

but now I receive a persistent keyerror on release.data["format"]. That said, release.data["formats"] works, although instead of returning a list similar to this ['12"', 'Vinyl', '45 RPM'] it now returns a list of dicts that look like this [{'name': 'Vinyl', 'qty': '1', 'descriptions': ['12"', '45 RPM']}].

Annoying to say the least because now 'Vinyl' is separated from the rest of the formats in descriptions and they have to be combined.

No harm no foul - I simply rewrote the code to look something like this

            query_dicts = []
            combined_formats = set()
            for release in releases:
                for format_entry in release.data["formats"]:
                    combined_formats.add(format_entry["name"])
                    combined_formats.update(format_entry["descriptions"])

                for release_format in combined_formats:
                    if release_format in VALID_FORMATS:
                        query_dicts.append({'release_id': release.id, 'format': release_format})
            return query_dicts

which works - except now I am running into an extremely frustrating issue.

I am receiving a persistent key error on release.data["formats"]. I had seen a similar issue in the past where it seemed that randomly "format" would be available when formats was not - but in this case, format is never available, and formats is intermittently available.

So I added a breakpoint within the exception clause of the below code:

        try:
            query_dicts = []
            combined_formats = set()
            for release in releases:
                for format_entry in release.data["formats"]:
                    combined_formats.add(format_entry["name"])
                    combined_formats.update(format_entry["descriptions"])

                for release_format in combined_formats:
                    if release_format in VALID_FORMATS:
                        query_dicts.append({'release_id': release.id, 'format': release_format})
            return query_dicts
        except KeyError as error:
            return 'failed' <-------- breakpoint here

the error shows

KeyError('formats')

and yet, in the console, calling release.data["formats"] returns without any issue whatsoever!

>>> release.data["formats"]
[{'name': 'Vinyl', 'qty': '1', 'descriptions': ['12"', '33 ⅓ RPM']}]

this leads me to believe there is a slight time delay before this data is available. I tried implementing a recursion approach to simply retry after a keyerror is encountered, but the timing issue appears to be significant enough that I am encountering maximum recursion depth errors.

I was only able to fix it by adding a release.refresh() in the exception clause before retrying

Why is this happening?

[rexovas]

Wow, you might have already shared a solution to this problem in the previous issue I submitted here https://github.com/joalla/discogs_client/issues/129

Somehow I thought that this was a separate issue, will try .fetch and see what happens

[AnssiAhola]

@rexovas I assume this is solved? If not, feel free to re-open / let us know.