Open HPLegion opened 4 years ago
@HPLegion thanks for submitting this. I'm afraid, w/o a reproducer this will be difficult to triage. Perhaps you could make a patch against the Numba source tree (which also uses sphinx for documentation) to expose the issue? Or if you have this in an open-source project, perhaps link to it? Perhaps you could try messing with different versions of sphinx/apidoc and see if the issue vanishes? Sorry I don't have any better ideas here.
I have gotten around to construct a small example #5765. In that process I also realised that the problem does not lie with sphinx apidoc
(which just creates the skeleton) but with autodoc
, which generates the actual reference documentation.
Okay, after stepping through sphinx I think I narrowed it down to the following:
njit
functions have the __wrapped__
attribute which I suppose contains the original python function. Sphinx can resolve this.
This field does not exist for vectorize
, hence autodoc thinks that it cannot document the function and skips it (it is not rejected based on any filters as far as I can tell, autodoc just gives up)
EDIT This is where the dispatcher updates the wrapper information https://github.com/numba/numba/blob/70f7b687bf7bc25a025a71aed5d74551f380fa5a/numba/core/dispatcher.py#L679 A similar step seems to happen for cfuncs https://github.com/numba/numba/blob/5f474010f8f50b3cf358125ba279d345ae5914ef/numba/core/ccallback.py#L45 But not for DUFuncs https://github.com/numba/numba/blob/5f474010f8f50b3cf358125ba279d345ae5914ef/numba/np/ufunc/dufunc.py#L79-L85
Some more digging has revealed that njit
functions get documented by autodoc because
methoddescriptor
__wrapped__
__module__
string.For njit
all these fields are set through the call to functools.update_wrapper, and the __get__
but not __set__
methods are implemented which qualifies them as methoddescriptors.
>>> from numba import njit
>>> import inspect
>>> @njit
... def f(x):
... return x
...
>>> f(1)
1
>>> inspect.isfunction(f)
False
>>> inspect.ismethoddescriptor(f)
True
>>> inspect.isroutine(f)
True
>>> hasattr(f, "__wrapped__")
True
>>> f.__module__
'__main__'
>>> f.__qualname__
'f'
>>> f.__wrapped__
<function f at 0x7f0b87558830>
>>> from numba import vectorize
>>> import inspect
>>> @vectorize
... def g(x):
... return x
...
>>> g(1)
1
>>> inspect.isfunction(g)
False
>>> inspect.ismethoddescriptor(g)
False
>>> inspect.isroutine(g)
False
>>> hasattr(g, "__wrapped__")
False
>>> g.__module__
'numba.np.ufunc.dufunc'
>>> g.__qualname__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'DUFunc' object has no attribute '__qualname__'
>>> g.__wrapped__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'DUFunc' object has no attribute '__wrapped__'
@HPLegion good to see you are getting somewhere investigating this. Do you think there might be a way to adapt this technique for @vectorize
too?
Yes, I think so.
TL;DR: I tried and numba hasn't exploded
lru_cache
from functools
to get a similar wrapper-example from the standard library. It turns out that functions decorated with lru_cache
do not pass the isfunction
check, they are indeed also methoddescriptors - so being a methoddescriptor seems like a reasonable choice for a decorated functionlru_cache
uses update_wrapper
to set its own fields to the appropriate values.
It is hence very much comparable to the jit
Dispatcher and both of them seem to get documented just fineupdate_wrapper
for a clean and standardized propagation of the decorated function attributes and implement __get__
but not __set__
for vectorize
(and maybe even further dispatchers), but this raises some questions
update_wrapper
sets a number of attributes, but by default also updates the entire __dict__
, this smells like a possibility for unforseen consequences (but it could be disabled)('__module__', '__name__', '__qualname__', '__doc__', '__annotations__')
crticial within numba and could break it if they were changed.Yesterday I set up a small branch where I simply called update_wrapper(self, py_func)
in the DUFunc
constructor and copied the __get__
implementation from the jit Dispatcher
. As far as I can tell that resulted in sphinx succesfully documenting vectorize-decorated functions, but I did not stress test it. There seems to be a plethora of Dispatchers for different targets and different ways to construct them so I am sure that this simple tweak did not fix all of them. However the good news is that it does not seem like I broke much in the process; all tests in tests/npyufunc
passed (I hope that was the right test set to focus on :stuck_out_tongue: )
lru_cache
calling update_wrapper
https://github.com/python/cpython/blob/4649202ea75d48e1496e99911709824ca2d3170e/Lib/functools.py#L765
update_wrapper
https://github.com/python/cpython/blob/4649202ea75d48e1496e99911709824ca2d3170e/Lib/functools.py#L33-L50
I had some time yesterday and found myself thinking that bending the inner workings of numba
for the sake of sphinx
may not be the way to go (even though there may or may not be other advantages to this). So I had a bit of a look for ways to fixing my problem and got inspired by celery
- they are offering a small sphinx extension as part of their package to allow auto-documentation of Celery (decorated) tasks.
The advantage to such an approach could be that it actually offers the possibility to customise the documentation behaviour. I hacked together a proof of concept, and it is not too terrifying, luckily Sphinx has a fairly comfortable extension API (but debugging event-driven programs turns out to be a nightmare :fearful:).
Here is a plaintext example of what could be done in principle. The documenter (see below) adds a prefix (@numba.vectorize
) to the signature and injects information about the precompiled signature into the docstring.
@numba.vectorize ebisim.plasma.clog_ei(Ni, Ne, kbTi, kbTe, Ai, qi)
*Precompiled signatures:[‘dddddl->d’]*
The coulomb logarithm for ion electron collisions.
Parameters:
* **Ni** (*float** or **numpy.ndarray*) – <1/m^3> Ion density.
* **Ne** (*float** or **numpy.ndarray*) – <1/m^3> Electron
density.
* **kbTi** (*float** or **numpy.ndarray*) – <eV> Ion
temperature.
* **kbTe** (*float** or **numpy.ndarray*) – <eV> Electron
temperature.
* **Ai** (*float** or **numpy.ndarray*) – Ion mass number.
* **qi** (*int** or **numpy.ndarray*) – Ion charge state.
Returns:
Ion electron coulomb logarithm.
Return type:
float or numpy.ndarray
Is an extension like this something that could be interesting to feature in numba, or is it just too likely to become a maintenance liability? :stuck_out_tongue: (The code as it is right now would likely need some bulletproofing.)
From my side this issue can be closed, since I found a solution that is working for me but if you want to keep this open to track the feature request, that's fine.
Extension code
"""
This sphinx extension aims to improve the documentation of numba-decorated
functions. It it inspired by the design of Celery's sphinx extension
'celery.contrib.sphinx'.
Usage
-----
Add the extension to your :file:`conf.py` configuration module:
.. code-block:: python
extensions = (...,
'numbadoc')
This extension adds two configuration fields, which determine the prefix
printed before jitted functions in the reference documentation.
Overwrite these values in your :file:`conf.py` to change this behaviour
.. code-block:: python
#defaults
numba_jit_prefix = '@numba.jit'
numba_vectorize_prefix = '@numba.vectorize'
With the extension installed `autodoc` will automatically find
numba decorated objects and generate the correct docs.
If a vecotrized function with fixed signatures is found, these are injected
into the docstring.
"""
from typing import List, Iterator
from sphinx.domains.python import PyFunction
from sphinx.ext.autodoc import FunctionDocumenter
from numba.core.dispatcher import Dispatcher
from numba.np.ufunc.dufunc import DUFunc
class NumbaFunctionDocumenter(FunctionDocumenter):
"""Document numba decorated functions."""
def import_object(self) -> bool:
"""Import the object given by *self.modname* and *self.objpath* and set
it as *self.object*.
Returns True if successful, False if an error occurred.
"""
success = super().import_object()
if success:
# Store away numba wrapper
self.jitobj = self.object
# And bend references to underlying python function
if hasattr(self.object, "py_func"):
self.object = self.object.py_func
elif hasattr(self.object, "_dispatcher") and \
hasattr(self.object._dispatcher, "py_func"):
self.object = self.object._dispatcher.py_func
else:
success = False
return success
def process_doc(self, docstrings: List[List[str]]) -> Iterator[str]:
"""Let the user process the docstrings before adding them."""
# Essentially copied from FunctionDocumenter
for docstringlines in docstrings:
if self.env.app:
# let extensions preprocess docstrings
# need to manually set 'what' to FunctionDocumenter.objtype
# to not confuse preprocessors like napoleon with an objtype
# that they don't know
self.env.app.emit('autodoc-process-docstring',
FunctionDocumenter.objtype,
self.fullname, self.object,
self.options, docstringlines)
# This block inserts information about precompiled signatures
# if this is a precompiled vectorized function
if getattr(self.jitobj, "types", []) and \
getattr(self.jitobj, "_frozen", False):
s = "| *Precompiled signatures:" + str(self.jitobj.types) + "*"
docstringlines.insert(0, s)
yield from docstringlines
class JitDocumenter(NumbaFunctionDocumenter):
"""Document jit/njit decorated functions."""
objtype = 'jitfun'
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
return isinstance(member, Dispatcher) and hasattr(member, 'py_func')
class VectorizeDocumenter(NumbaFunctionDocumenter):
"""Document vectorize decorated functions."""
objtype = 'vecfun'
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
return isinstance(member, DUFunc) and \
hasattr(member, '_dispatcher') and \
hasattr(member._dispatcher, 'py_func')
class JitDirective(PyFunction):
"""Sphinx jitfun directive."""
def get_signature_prefix(self, sig):
return self.env.config.numba_jit_prefix
class VectorizeDirective(PyFunction):
"""Sphinx vecfun directive."""
def get_signature_prefix(self, sig):
return self.env.config.numba_vectorize_prefix
def setup(app):
"""Setup Sphinx extension."""
# Register the new documenters and directives (autojitfun, autovecfun)
# Set the default prefix which is printed in front of the function signature
app.setup_extension('sphinx.ext.autodoc')
app.add_autodocumenter(JitDocumenter)
app.add_directive_to_domain('py', 'jitfun', JitDirective)
app.add_config_value('numba_jit_prefix', '@numba.jit', True)
app.add_autodocumenter(VectorizeDocumenter)
app.add_directive_to_domain('py', 'vecfun', VectorizeDirective)
app.add_config_value('numba_vectorize_prefix', '@numba.vectorize', True)
return {
'parallel_read_safe': True
}
(ping @esc )
Hi, thank you very much for this extension, but my decorated functions still don't appear in the docs. The only change I made is using @cuda.jit
instead of @numba.jit
with this line in conf.py
:
numba_jit_prefix = '@cuda.jit'
Do you have any idea why it doesn't work in this case?
Hej @soleti,
it's been a few days since I bashed this together, but my best guess at the moment is the following:
The Documenter classes have a method can_document_member
which sphinx uses to determine the extensions ability to deal with certain objects.
As they are written now, they probably simply return False for CUDA jit functions.
I see two possiblefixes here:
Let me know if that helps, and if yes, feel free to share your results here for future reference (if I have time one day, I may turn this into a tiny package, but more likely I will just document it in a better place (i.e. Luk's How-to-numba-repo))
I tried adapting the jit-documenter stuff but I couldn't get it to work. I adopted a solution that I am ashamed of but that works: running this before sphinx :)
find mypackage/ -name '*.py' -exec sed -i -e 's/@cuda/#@cuda/g' {} \;
Hi @soleti,
For me the following seems to work, feel welcome to give it a test spin and if yes expand on it.
(full disclaimer: I don't really know numba.cuda at all, so I am not aware if my check for CUDA funcs is even remotely sufficient or what nice gimmicks would be for documenting CUDA functions, also my Laptop has no GPU, so I only checked this without ever actually triggering the compilation of the CUDA decorated function)
Same sphinx extension as before with added cuda features: (You need to make sure that Sphinx loads this, by both adding it to the conf.py and making sure that it is actually on the current path for sphinx to find the module and import it, I called the file numbadoc.py (hence the suggested extension name in the docstring)
"""
This sphinx extension aims to improve the documentation of numba-decorated
functions. It it inspired by the design of Celery's sphinx extension
'celery.contrib.sphinx'.
Usage
-----
Add the extension to your :file:`conf.py` configuration module:
.. code-block:: python
extensions = (...,
'numbadoc')
This extension adds two configuration fields, which determine the prefix
printed before jitted functions in the reference documentation.
Overwrite these values in your :file:`conf.py` to change this behaviour
.. code-block:: python
#defaults
numba_jit_prefix = '@numba.jit'
numba_vectorize_prefix = '@numba.vectorize'
numba_cuda_jit_prefix = '@numba.cuda.jit'
With the extension installed `autodoc` will automatically find
numba decorated objects and generate the correct docs.
If a vecotrized function with fixed signatures is found, these are injected
into the docstring.
"""
from typing import List, Iterator
from sphinx.domains.python import PyFunction
from sphinx.ext.autodoc import FunctionDocumenter
from numba.core.dispatcher import Dispatcher
from numba.np.ufunc.dufunc import DUFunc
from numba.cuda.compiler import AutoJitCUDAKernel
class NumbaFunctionDocumenter(FunctionDocumenter):
"""Document numba decorated functions."""
def import_object(self) -> bool:
"""Import the object given by *self.modname* and *self.objpath* and set
it as *self.object*.
Returns True if successful, False if an error occurred.
"""
success = super().import_object()
if success:
# Store away numba wrapper
self.jitobj = self.object
# And bend references to underlying python function
if hasattr(self.object, "py_func"):
self.object = self.object.py_func
elif hasattr(self.object, "_dispatcher") and \
hasattr(self.object._dispatcher, "py_func"):
self.object = self.object._dispatcher.py_func
else:
success = False
return success
def process_doc(self, docstrings: List[List[str]]) -> Iterator[str]:
"""Let the user process the docstrings before adding them."""
# Essentially copied from FunctionDocumenter
for docstringlines in docstrings:
if self.env.app:
# let extensions preprocess docstrings
# need to manually set 'what' to FunctionDocumenter.objtype
# to not confuse preprocessors like napoleon with an objtype
# that they don't know
self.env.app.emit('autodoc-process-docstring',
FunctionDocumenter.objtype,
self.fullname, self.object,
self.options, docstringlines)
# This block inserts information about precompiled signatures
# if this is a precompiled vectorized function
if getattr(self.jitobj, "types", []) and \
getattr(self.jitobj, "_frozen", False):
s = "| *Precompiled signatures:" + str(self.jitobj.types) + "*"
docstringlines.insert(0, s)
yield from docstringlines
class JitDocumenter(NumbaFunctionDocumenter):
"""Document jit/njit decorated functions."""
objtype = 'jitfun'
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
return isinstance(member, Dispatcher) and hasattr(member, 'py_func')
class VectorizeDocumenter(NumbaFunctionDocumenter):
"""Document vectorize decorated functions."""
objtype = 'vecfun'
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
return isinstance(member, DUFunc) and \
hasattr(member, '_dispatcher') and \
hasattr(member._dispatcher, 'py_func')
class CUDAJitDocumenter(NumbaFunctionDocumenter):
"""Document jit/njit decorated functions."""
objtype = 'cudajitfun'
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
return isinstance(member, AutoJitCUDAKernel)\
and hasattr(member, 'py_func')
class JitDirective(PyFunction):
"""Sphinx jitfun directive."""
def get_signature_prefix(self, sig):
return self.env.config.numba_jit_prefix
class VectorizeDirective(PyFunction):
"""Sphinx vecfun directive."""
def get_signature_prefix(self, sig):
return self.env.config.numba_vectorize_prefix
class CUDAJitDirective(PyFunction):
"""Sphinx jitfun directive."""
def get_signature_prefix(self, sig):
return self.env.config.numba_cuda_jit_prefix
def setup(app):
"""Setup Sphinx extension."""
# Register the new documenters and directives (autojitfun, autovecfun)
# Set the default prefix which is printed in front of the function signature
app.setup_extension('sphinx.ext.autodoc')
app.add_autodocumenter(JitDocumenter)
app.add_directive_to_domain('py', 'jitfun', JitDirective)
app.add_config_value('numba_jit_prefix', '@numba.jit', True)
app.add_autodocumenter(VectorizeDocumenter)
app.add_directive_to_domain('py', 'vecfun', VectorizeDirective)
app.add_config_value('numba_vectorize_prefix', '@numba.vectorize', True)
app.add_autodocumenter(CUDAJitDocumenter)
app.add_directive_to_domain('py', 'cudajitfun', CUDAJitDirective)
app.add_config_value('numba_cuda_jit_prefix', '@numba.cuda.jit', True)
return {
'parallel_read_safe': True
}
Hey @HPLegion thank you very much for this. However, I get this error message
Could not import extension numbadoc (exception: cannot import name 'AutoJitCUDAKernel' from 'numba.cuda.simulator.compiler' (/usr/local/lib/python3.8/site-packages/numba/cuda/simulator/compiler.py))
which I think it's because I am running it on a machine without a GPU. Replacing it with FakeCUDAKernel
fixes the error, but the functions still don't appear in the docs.
that's a curious one, cause my laptop does not have a GPU (and that is where I tested). Which version of numba are you running?
Does this work for you (or does it show a different type)?
In [1]: from numba.cuda import jit as cjit
In [2]: @cjit
...: def f(a, b):
...: return a+b
...:
In [3]: type(f)
Out[3]: numba.cuda.compiler.AutoJitCUDAKernel
AutoJitCUDAKernel
was removed in 0.51 - it is now cuda.compiler.Dispatcher
: https://github.com/numba/numba/blob/master/numba/cuda/compiler.py#L773 - this is part of bringing the CUDA target into line with the CPU target.
Sweet, that will likely be the reason (I am still running 0.50.0 locally)
@soleti if you are running 0.51+ just exchange the AutoJitCUDAKernel to the cuda Dispatcher (both import and in CUDAJitDocumenter.can_document_member
)
Uhm it still doesn't work. I am doing
from numba.cuda.compiler import Dispatcher as CUDADispatcher
which gives me
Extension error:
Could not import extension numbadoc (exception: cannot import name 'Dispatcher' from 'numba.cuda.simulator.compiler' (/usr/local/lib/python3.8/site-packages/numba/cuda/simulator/compiler.py))
Hey! Thank you for the extension for autodoc. I wanted to add, for future people searching this, that this doesn't work with numba cuda device functions specifically, only kernels. To fix this, add this
from numba.cuda.compiler import DeviceFunctionTemplate
and change this
class CUDAJitDocumenter(NumbaFunctionDocumenter):
"""Document jit/njit decorated functions."""
objtype = 'cudajitfun'
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
return isinstance(member, AutoJitCUDAKernel)\
and hasattr(member, 'py_func')
to
class CUDAJitDocumenter(NumbaFunctionDocumenter):
"""Document jit/njit decorated functions."""
objtype = 'cudajitfun'
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
return (isinstance(member, AutoJitCUDAKernel) and hasattr(member, 'py_func')) or isinstance(member, DeviceFunctionTemplate)
Hope this helps someone, and thanks again!
@HPLegion, thanks for the sphinx extension for autodoc. I tried to change it to work with autosummary
+ @intrinsic
:
import numba
from typing import List, Optional
from sphinx.ext.autodoc import FunctionDocumenter
class NumbaDocumenter(FunctionDocumenter):
"""Sphinx directive that also understands numba decorators"""
objtype = 'function' # This must be assigned to 'function' to work!!
directivetype = 'function'
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
if isinstance(member, numba.core.extending._Intrinsic):
return True
return super(RBCDocumenter, cls).can_document_member(member, membername, isattr, parent)
def setup(app):
"""Setup Sphinx extension."""
# Register the new documenters and directives (autojitfun, autovecfun)
# Set the default prefix which is printed in front of the function signature
app.setup_extension('sphinx.ext.autodoc')
app.add_autodocumenter(NumbaDocumenter)
return {
'parallel_read_safe': True
}
Also, in order for this to work, one needs to update the intrinsic wrapper:
import functools
functools.update_wrapper(my_intrinsic, my_intrinsic._defn) # _defn is the inner function
Hi @HPLegion, thanks for the extension! Any idea on how to extend it to support @guvectorize
-decorated functions?
Hi @HPLegion, thanks for the extension! Any idea on how to extend it to support
@guvectorize
-decorated functions?
Here's my version of the extension that handles @guvectorize
d functions, if anyone's interested:
I am buried in other work at the moment, but with the collection as it is, this almost looks as if it might warrant releasing a micro-package that people can install or vendor directly into their codebase. If anyone feels the urge to create such a thing, I would certainly not hold a grudge :-P
Just had a brief look at the issue @gmarkall cross linked yesterday, seems like there are still issues linked to the missing magic wrapper information. Is there a dedicated issue for that?
Hi all. Found a simple way to document the Numba classes and methods in my code. Basically, I just added the following line to the Sphinx conf.py
(effectively disabling the JIT compilation, and allowing Sphinx autodoc to properly parse all functions and classes):
os.environ["NUMBA_DISABLE_JIT"] = '1'
Hello all,
I am adjusting/improving/cleaning up the documentation (pr#160) for a python package. However, I have been struggling with documenting jit-decorated functions. I am using autosummary
extension to generate recursively a page for every module/class/function (see SO#2701998 )
I have tried all the solutions mentioned here:
os.environ["NUMBA_DISABLE_JIT"] = '1'
to the conf.py
does nothing ;
WARNING: [autosummary] failed to import sarkas.potentials.
Possible hints:
and/or
```shell
Failed to import sarkas.potentials.
Possible hints:
Somehow it can't read the submodules anymore?
I am using Sphinx v4.5.0, but I have also tried Sphinx v6.2.1 and v7.2.6 with the same results.
I don't really understand what I am doing wrong any help is highly appreciated.
@SilvestriStefano
Hi,
unfortunately, it has been a while since I have thought about this, so it is a bit tricky to give good advice. However: Those errors do not look like sphinx problems. It looks like there is actually a problem with the package you are documenting. If you run a pure Python shell in the environment that you build the documentation with, can you actually import that package as expected or do you get a similar error?
Cheers
Hello @HPLegion ,
thanks for responding. The issue only arises whenever I add the extra extension in the conf.py
. Otherwise the documentation builds fine without warnings, except that the module functions are not present (see below)
The conf.py
with the extension looks like :
import os
import sys
import sarkas
sys.path.insert(0, os.path.abspath("../sarkas"))
sys.path.insert(0, os.path.abspath("../sarkas/time_evolution"))
sys.path.insert(0, os.path.abspath("../sarkas/utilities"))
sys.path.insert(0, os.path.abspath("../sarkas/potentials"))
sys.path.insert(0, os.path.abspath("../sarkas/tools"))
sys.path.insert(0, os.path.abspath("./_ext")) # <-------- this is where the extension file is located
extensions = [
"sphinx.ext.napoleon",
"sphinx.ext.autodoc",
"sphinx.ext.autosummary",
"numbadecoratordoc" # <-------- how i called the file
]
# ... more stuff ...
autodoc_mock_imports = ["yaml", "numba", "scipy", "optparse", "time", "pyfftw", "pyfiglet", "tqdm", "fmm3dpy"]
# Generate the API documentation when building
autosummary_generate = True
autosummary_generate_overwrite = True
# # A list of ignored prefixes for module index sorting.
modindex_common_prefix = ["sarkas."]
# Make sure the target is unique
autosectionlabel_prefix_document = True
# Napoleon settings
napoleon_google_docstring = False
napoleon_numpy_docstring = True
napoleon_include_init_with_doc = False
napoleon_include_private_with_doc = False
napoleon_include_special_with_doc = True
napoleon_use_admonition_for_examples = False
napoleon_use_admonition_for_notes = False
napoleon_use_admonition_for_references = False
napoleon_use_ivar = True
napoleon_use_param = False
napoleon_use_rtype = False
Hello again @HPLegion , I have done a little debugging and i found out a few things:
create_cells_array()
in the screen of my previous comment) they can create more problems like
Untyped global name 'zeros': Cannot determine Numba type of <class 'builtin_function_or_method'>
File "../sarkas/potentials/force_pp.py", line 538: def create_cells_array(box_lengths, cutoff):
cells_per_dim = zeros(3, dtype=int64)
^
It is not a big problem (to us anyway) as apparently we can go without the signature specifications.
- if I only use the decorators `@jit(nopython=True)` or `@njit` I get no errors and no warnings, however the decorated functions are still not appearing in the doc.
- If I modify the `JitDocumenter` class method `can_document_member` as follows
```python
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
return isinstance(member, Dispatcher) and hasattr(member, 'py_func') and member.__doc__ is None
the decorated functions (they should not have any signature specification) whose docstring is not empty do appear in the docs but without the prefix. I am confused as why this works though.
.numba_config.yaml
with the entry DISABLE_JIT: "1"
in the docs/
folder (as per numba documentation) and use your extension without modifications, all the decorated functions (even those without a docstring and those with signature specifications) appear in the documentation..numba_config.yaml
with the entry DISABLE_JIT: "1"
in the docs/
folder and do not use your extension, no decorated functions appears in the documentation.Hi Stefano (and others)
I just wanted to give you a heads-up that I won't be able to follow up on these issues myself for a while, unfortunately. I would like to offer more help, but time is limited at the moment.
I expect documentation to fail without the plugin (as you observed), since Sphinx gets confused by numba's decorators. I don't know why you see the flaky behviour in some cases though.
When I first looked into this, I really had to debug the Sphinx autodoc run to see why the documenters did not pick up on the decorated functions, and then I tried to come up with something that would be able to identify the decorated functions as more or less normally documentable objects. Then I used the chance to sprinkle in a few extras, maybe some of that broke over the last years. It is not particularly fun, but maybe a way for follow up investigations.
In the meantime I would encourage you to write on https://numba.discourse.group/ - there is a chance you will be able to get in touch with some knowledgable people who are not aware of this issue / disucussion. Judging by how lively this thread still is, this seems to affect a number of users. Maybe someone has the time and energy to really dive into this and come up with a good and maintainable lonterm solution :-)
Best of luck!
At the moment we are satisfied with how it works. However, I would prefer to understand the flaky behavior, so I will probably bug the people on discourse :P
Thank you for all your help :)
I use
sphinx
with theapidoc
extension to have a reference documentation generated automatically. For functions decorated withnjit
this works without any complications, forvectorize
however, the function disappears from the docs, indicating that apidoc either cannot find it or decides not to include it. (Sorry for the lack of better info).Since this is sphinx related I am not sure how to include a minimal example, but if it would help and I find some time I can try to set up a mini-repository.