[GSoC] Add Python 3 typing annotations to TFDS

tensorflow / datasets

TFDS is a collection of datasets ready to use with TensorFlow, Jax, ...

https://www.tensorflow.org/datasets

Apache License 2.0

4.28k stars 1.53k forks source link

[GSoC] Add Python 3 typing annotations to TFDS #1517

Open Conchylicultor opened 4 years ago

Conchylicultor commented 4 years ago

Task: Add Python 3 typing annotations to TFDS repository. Now that Python 2 support is officially dropped, we can add better typing to TFDS.

Impact: Everyone using TFDS with a static type checker

Instructions:

Add typing annotations to our API: (Typing annotation guide: https://docs.python.org/3/library/typing.html)
Check that the typing annotations are valid with https://github.com/google/pytype
Only update only a few files at a time (send multiple PR).
As TFDS has many files, many people can work on this issue at the same time, just coordinate with each other.
Updating the core/ is what matter the most. Then scripts/, Other files are optional
Typing are ignored in test files, so no need to add typing annotations to *_test.py

Complexity: Easy

abhinavsp0730 commented 4 years ago

Hi, I am Abhinav Prakash and I am working in this.

vijayphoenix commented 4 years ago

@Conchylicultor I was thinking of editing the my_dataset generator script, so that all new datasets will have Python3 typing annotations. For example,

def _generate_examples(self) -> [Dictionary]:

Please share your views and guide me on the same Thank you

Conchylicultor commented 4 years ago

@vijayphoenix, That's a good idea. To avoid boilerplate, you could add a core/types.py with:

KeyType = Union[str, bytes, int]
KeyExample = Tuple[KeyType, Dict[str, Any]]

And then have: _generate_examples() -> tfds.types.KeyExample:

ashutosh1919 commented 4 years ago

@Conchylicultor, I have added first PR for this issue which adds pytype in 3 files. I will add multiple PRs and I will only include 3-4 files per PR so that it will be easy for reviewer to review.

Also, @abhinavsp0730 and @vijayphoenix , Please stay coordinated so that we don't overlap PRs. I am working on tfds/core outer files. If you are working then please work on tfds/core/(decode/download/features/proto/utils) files. Also, please review my PR and also give your comments since you two are working on same issue.

Eshan-Agarwal commented 4 years ago

Hey! I am working on core(/decode/download)

Eshan-Agarwal commented 4 years ago

@Conchylicultor and @ashutosh1919 can you check this PR #1571 and let me know if I am mistaking ?

Ir1d commented 4 years ago

Hi! I'm working on core/utils