Stub file generation

[bengsparks]

Generate stub files from the data generated from the tracing. Find a library to assist this.

[bengsparks]

Starting this now. stubfile and inline type hint generation will be researched by me

[bengsparks]

Not particularly important for our project, but maybe a nice to have feature

[RecurvedBow]

An implementation will be made using mypy stubgen. However, as the API does not fit our requirements, the current implementation requires two temp files to generate a stub: One to store the type hinted file, and the other to store the stub.

If anyone wishes to try to implement without using these files, feel free to try. However, only if the time allows it. - Here is an attempt using the type hinted file content directly:

    def _store_hinted_ast(self, source_file: pathlib.Path, hinting: cst.Module) -> None:
        # Stub means generating a stub without overwriting the original
        contents = str.encode(hinting.code)

        filename = source_file.stem
        output_path = source_file.with_suffix('.pyi')
        print(output_path)
        path_to_stub = str(output_path.relative_to(pathlib.Path.cwd()))
        options = stubgen.parse_options([str(source_file)])
        mypy_opts = stubgen.mypy_options(options)
        module = stubgen.StubSource(filename, str(source_file))

        source = mypy.util.decode_python_encoding(contents, mypy_opts.python_version)

        errors = stubgen.Errors()
        module.ast = mypy.parse.parse(source, fnam=module.path, module=module.module,
                                      errors=errors, options=mypy_opts)
        assert not errors.is_blockers()
        module.ast._fullname = module.module

        with stubgen.generate_guarded(module.module, path_to_stub):
            stubgen.generate_stub_from_ast(module, path_to_stub,
                                           False, mypy_opts.python_version)

[bengsparks]

Stubbed files do not require a file with type hints. e.g. a stub for a/b/c.py would be stored in a/b/c.pyi. The annotations live ONLY in the .pyi, and there are none in the .py code. Variable annotations do not seem to be present in such files, as callables do not have bodies.

See https://github.com/PyGithub/PyGithub/tree/master/github for an example.

[RecurvedBow]

The StubFileGenerator does not override the original, only generates the stubs with hints. Because it overrides from InlineGenerator, the quality of the stub depends on the inline generator. Example vom test:

def function(c):
    return c.outer(5)

class AnotherC:
    pass

def another_function(c):
    return "4"

With the sample test trace data, it generates:

from tests.resource.typegen.callable import C as C

def function(c: C) -> int: ...

class AnotherC: ...

def another_function(c: AnotherC) -> str: ...

Original stays the same.

[RecurvedBow]

Need to implement #119 first to prevent the original type hints from being removed. Is required for a test.