Cannot serialise GitHub issues to JSON

[lewtun]

Description of the problem

Hello Jeremy and Hamel,

I'm trying to use ghapi to fetch issues from a GitHub repo and then dump them to disk as a jsonl file, where each line is an issue.

The problem I'm running into is that the result from GhAPI.issues.list_for_repo contains fastcore's special L type which appears to not be JSON serialisable. As a result I'm getting the following error: TypeError: Object of type L is not JSON serializable

Is there a way to make L play nice with JSON?

Steps to reproduce

import json
from ghapi.all import GhApi

api = GhApi()
issues = api.issues.list_for_repo("huggingface", "transformers", page=1, per_page=5, state="all")

# try to write the first issue
with open("issues.jsonl", "w") as f:
    json.dump(issues[0], f)

Expected result

I can write GitHub issues to disk in JSON format.

Actual result

Here's the stack trace

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-48-04139a7641e4> in <module>
      6 
      7 with open("issues.json", "w") as f:
----> 8     json.dump(issues[0], f)

~/miniconda3/envs/transformersbook/lib/python3.8/json/__init__.py in dump(obj, fp, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
    177     # could accelerate with writelines in some versions of Python, at
    178     # a debuggability cost
--> 179     for chunk in iterable:
    180         fp.write(chunk)
    181 

~/miniconda3/envs/transformersbook/lib/python3.8/json/encoder.py in _iterencode(o, _current_indent_level)
    429             yield from _iterencode_list(o, _current_indent_level)
    430         elif isinstance(o, dict):
--> 431             yield from _iterencode_dict(o, _current_indent_level)
    432         else:
    433             if markers is not None:

~/miniconda3/envs/transformersbook/lib/python3.8/json/encoder.py in _iterencode_dict(dct, _current_indent_level)
    403                 else:
    404                     chunks = _iterencode(value, _current_indent_level)
--> 405                 yield from chunks
    406         if newline_indent is not None:
    407             _current_indent_level -= 1

~/miniconda3/envs/transformersbook/lib/python3.8/json/encoder.py in _iterencode(o, _current_indent_level)
    436                     raise ValueError("Circular reference detected")
    437                 markers[markerid] = o
--> 438             o = _default(o)
    439             yield from _iterencode(o, _current_indent_level)
    440             if markers is not None:

~/miniconda3/envs/transformersbook/lib/python3.8/json/encoder.py in default(self, o)
    177 
    178         """
--> 179         raise TypeError(f'Object of type {o.__class__.__name__} '
    180                         f'is not JSON serializable')
    181 

TypeError: Object of type L is not JSON serializable

Versions

• ghapi: v0.1.16

[jph00]

Does it work if you apply fastcore's obj2dict to issues before you serialize it?

[lewtun]

thanks @jph00 - this indeed works!

for reference this is what i used:

import json
from ghapi.all import GhApi
from fastcore.xtras import obj2dict

api = GhApi()
issues = api.issues.list_for_repo("huggingface", "transformers", page=1, per_page=5, state="all")
issues = [obj2dict(issue) for issue in issues]

# try to write the first issue
with open("issues.jsonl", "w") as f:
    json.dump(issues[0], f)

[kapilt]

this should be in the readme, its pretty basic core functionality.