Further, the inclusion of examples is not only educative, but can become part of our tests with doctest. Things are harder to break too, when you're reminded in the source code of exactly how they're supposed to work and what types that are supposed to take and return.
In the process of writing this for the Crawler and Sorter, we would also properly work out keyword unpacking such that the user may pass the full range of initialization options (or other arguments when calling methods) supported by each class it wraps (giving greater control of the Tor process and controller, the Firefox process and driver, Xvfb, etc.). The docstring for the initialization of these internal objects in particular should be divided into sections, and might even have to omit certain options as it may turn out to be excessive. We'll also have to make sure there are no parameter name conflicts.
I think overall this can make more accessible a lot of functionality that already exists in the Crawler, can help catch mistakes before they happen, and should overall improve code quality.
Update: partially implemented; there are still a lot of classes/ functions that lack docstrings/ docstring tests, but we've been pretty good about not adding new code w/o them.
Of Python libraries that come to mind,
python-gnupg
has the nicest docstrings in my opinion. Poke around https://github.com/isislovecruft/python-gnupg/blob/master/gnupg/. You learn:Further, the inclusion of examples is not only educative, but can become part of our tests with doctest. Things are harder to break too, when you're reminded in the source code of exactly how they're supposed to work and what types that are supposed to take and return.
In the process of writing this for the Crawler and Sorter, we would also properly work out keyword unpacking such that the user may pass the full range of initialization options (or other arguments when calling methods) supported by each class it wraps (giving greater control of the Tor process and controller, the Firefox process and driver, Xvfb, etc.). The docstring for the initialization of these internal objects in particular should be divided into sections, and might even have to omit certain options as it may turn out to be excessive. We'll also have to make sure there are no parameter name conflicts.
I think overall this can make more accessible a lot of functionality that already exists in the Crawler, can help catch mistakes before they happen, and should overall improve code quality.
Related: #28.