search

cea-hpc / clustershell

Scalable cluster administration Python framework — Manage node sets, node groups and execute commands on cluster nodes in parallel.
https://clustershell.readthedocs.io/
422 stars 85 forks source link

Updating documentation for Python 3 #544

Open dwysocki opened 10 months ago

dwysocki commented 10 months ago

There are a lot of Python 2 code examples in the docs. A search for "print " (note the trailing space) finds 29 matching files, some of which are not offenders, but many are, including README.md. Presumably there are other issues as well, but that's the easy one to search.

I can make a pull request with fixes, but I need some info. I see Python 2.7 is still listed as supported, so presumably the examples should be Python 2/3 cross-compatible? Are there any other requirements?

degremont commented 10 months ago

Thanks @dwysocki ! Honestly, we are thinking of dropping 2.7 support in next release, so I'm not sure there is value in writting examples that are cross-compatible. Moreover, 2.7 support is there for compatibility for existing code base, not for people writing new projects from scratch.

If you can just send a PR to migrate them to Py3, that would be great!

dwysocki commented 10 months ago

Here is a task list with every object that has a docstring containing code examples, with line numbers inserted as comments

dwysocki commented 10 months ago

For future reference, this is the script I wrote to generate that list, in case it's useful to anybody (credit to Bing AI for writing the initial draft)

import doctest
import inspect
import importlib
import pkgutil

import ClusterShell

def get_docstrings(module):
    result = {}

    def store(name, val):
        if val:
            result[name] = val

    for name, obj in inspect.getmembers(module):
        if inspect.isclass(obj) or inspect.isfunction(obj) or inspect.ismodule(obj):
            store(f"{module.__name__}.{name}", get_examples(module.__name__, name, obj.__doc__))

        if inspect.isclass(obj):
            for method_name, method_obj in inspect.getmembers(obj):
                if inspect.isfunction(method_obj):
                    store(f"{module.__name__}.{name}.{method_name}", get_examples(module.__name__, f'{name}.{method_name}', method_obj.__doc__))

    if hasattr(module, '__path__'):
        for _, submodule_name, _ in pkgutil.iter_modules(module.__path__):
            result.update(get_docstrings(importlib.import_module(f'{module.__name__}.{submodule_name}')))

    return result

def get_examples(module_name, obj_name, docstring):
    if docstring:
        return doctest.DocTestParser().get_examples(docstring)

def print_examples(examples):
    source_width = max(max(len(line) for line in ex.source.splitlines())
                       for ex in examples)

    for ex in examples:
        for i, line in enumerate(ex.source.splitlines()):
            if i == 0:
                print(">>>", line.ljust(source_width), "# line:", ex.lineno)
            else:
                print("...", line)

def main():
    docstrings = get_docstrings(ClusterShell)

    for name, examples in docstrings.items():
        print(f"- [ ] `{name}`")
        print("```python")
        print_examples(examples)
        print("```")

if __name__ == "__main__":
    main()