Open bendichter opened 6 years ago
Thanks for reporting the issue.
What are the steps leading to this error ?
As a side note, you can now pip install nwb-docutils
, this will make available these command line tools
nwb_init_sphinx_extension_doc --project simulation_output --author "Ben Dicher and Kael Dai" --version "0.1.0" --release alpha --spec_dir ~/dev/nwb-extensions/simulation_output/spec/ --namespace_filename ~/dev/nwb-extensions/simulation_output/spec/simulation_output.namespace.yaml --default_namespace simulation_output --external_description ~/dev/nwb-extensions/simulation_output/docs/description.rst --external_release_notes ~/dev/nwb-extensions/simulation_output/docs/release_notes.rst --output ~/dev/nwb-extensions/simulation_output/docs
error:
cp: /Users/bendichter/anaconda3/lib/python3.6/site-packages/nwb_docutils/../../Legal.txt: No such file or directory
Copy of utils dir failed: ['cp', '/Users/bendichter/anaconda3/lib/python3.6/site-packages/nwb_docutils/../../Legal.txt', '/Users/bendichter/dev/nwb-extensions/simulation_output/docs/utils']
make apidoc
error:
PYTHONPATH=/Users/bendichter/dev/nwb-extensions/simulation_output/docs/utils:/Users/bendichter/dev/nwb-extensions/simulation_output/docs/source:/usr/local/hdf5: python /Users/bendichter/dev/nwb-extensions/simulation_output/docs/utils/generate_format_docs.py
python: can't open file '/Users/bendichter/dev/nwb-extensions/simulation_output/docs/utils/generate_format_docs.py': [Errno 2] No such file or directory
make: *** [apidoc] Error 2
That's just a path error. It should be nwb-extensions/simulation_output/docs/nwb_docutils/generate_format_docs.py
But when I fix that:
apidoc
error:
Bens-MacBook-Pro:docs bendichter$ make apidoc
PYTHONPATH=/Users/bendichter/dev/nwb-extensions/simulation_output/docs/nwb_docutils:/Users/bendichter/dev/nwb-extensions/simulation_output/docs/source:/usr/local/hdf5: python /Users/bendichter/dev/nwb-extensions/simulation_output/docs/nwb_docutils/generate_format_docs.py
/Users/bendichter/dev/nwb-extensions/simulation_output/docs/nwb_docutils/generate_format_docs.py:60: UserWarning: DISABLING RENDERING OF SPEC GRAPHS DUE TO IMPORT ERROR
warnings.warn('DISABLING RENDERING OF SPEC GRAPHS DUE TO IMPORT ERROR')
Generating output directory: /Users/bendichter/dev/nwb-extensions/simulation_output/docs/source/_format_auto_docs
Traceback (most recent call last):
File "/Users/bendichter/dev/pynwb/src/pynwb/form/spec/namespace.py", line 379, in load_namespaces
inc_ns = self.get_namespace(s['namespace'])
File "/Users/bendichter/dev/pynwb/src/pynwb/form/utils.py", line 348, in func_call
return func(self, **parsed['args'])
File "/Users/bendichter/dev/pynwb/src/pynwb/form/spec/namespace.py", line 240, in get_namespace
raise KeyError("'%s' not a namespace" % name)
KeyError: "'core' not a namespace"
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/bendichter/dev/nwb-extensions/simulation_output/docs/nwb_docutils/generate_format_docs.py", line 1356, in <module>
main()
File "/Users/bendichter/dev/nwb-extensions/simulation_output/docs/nwb_docutils/generate_format_docs.py", line 1263, in main
resolve=spec_resolve_type_inc)
File "/Users/bendichter/dev/nwb-extensions/simulation_output/docs/nwb_docutils/generate_format_docs.py", line 88, in load_nwb_namespace
namespace.load_namespaces(namespace_file, resolve=resolve)
File "/Users/bendichter/dev/pynwb/src/pynwb/form/utils.py", line 348, in func_call
return func(self, **parsed['args'])
File "/Users/bendichter/dev/pynwb/src/pynwb/form/spec/namespace.py", line 381, in load_namespaces
raise ValueError("Could not load namespace '%s'" % s['namespace'])
ValueError: Could not load namespace 'core'
make: *** [apidoc] Error 1
@bendichter the init_sphinx_extension_doc script needs to be updated to not copy the sources that are now in nwb-docutils but rather to use nwb-docutils directly.
I just push some fixes to nwb-docutils that account for the fact script are not anymore located in the nwb-schema/docs/utils
.
There is no intermediate errors anymore. Here are the steps so far:
Install from source:
pip install git+https://github.com/NeurodataWithoutBorders/nwb-docutils
Then:
mkdir -p /tmp/scratch && cd $_
git clone git@github.com:NeurodataWithoutBorders/nwb-extensions.git
cd nwb-extensions/extensions/simulation_output
nwb_init_sphinx_extension_doc \
--project test \
--author "Dr. Master Expert" \
--version "1.2.3" \
--release alpha \
--output my_extension_docs \
--spec_dir . \
--namespace_filename simulation_output.namespace.yaml \
--default_namespace simulation_output
cd my_extension_docs
make apidoc
The following error is reported:
Generating output directory: /tmp/nwb-extensions/extensions/simulation_output/my_extension_docs/source/_format_auto_docs
Traceback (most recent call last):
File "/home/jcfr/Projects/pynwb/src/pynwb/form/spec/namespace.py", line 379, in load_namespaces
inc_ns = self.get_namespace(s['namespace'])
File "/home/jcfr/Projects/pynwb/src/pynwb/form/utils.py", line 348, in func_call
return func(self, **parsed['args'])
File "/home/jcfr/Projects/pynwb/src/pynwb/form/spec/namespace.py", line 240, in get_namespace
raise KeyError("'%s' not a namespace" % name)
KeyError: "'core' not a namespace"
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/jcfr/.virtualenvs/pynwb-py35/bin/nwb_generate_format_docs", line 11, in <module>
load_entry_point('nwb-docutils', 'console_scripts', 'nwb_generate_format_docs')()
File "/tmp/nwb-extensions/extensions/simulation_output/my_extension_docs/nwb_docutils/generate_format_docs.py", line 1263, in main
resolve=spec_resolve_type_inc)
File "/tmp/nwb-extensions/extensions/simulation_output/my_extension_docs/nwb_docutils/generate_format_docs.py", line 88, in load_nwb_namespace
namespace.load_namespaces(namespace_file, resolve=resolve)
File "/home/jcfr/Projects/pynwb/src/pynwb/form/utils.py", line 348, in func_call
return func(self, **parsed['args'])
File "/home/jcfr/Projects/pynwb/src/pynwb/form/spec/namespace.py", line 381, in load_namespaces
raise ValueError("Could not load namespace '%s'" % s['namespace'])
ValueError: Could not load namespace 'core'
Makefile:170: recipe for target 'apidoc' failed
make: *** [apidoc] Error 1
@oruebel A new version of nwb-docutils was just published on pypi, the new version doesn't copy the source of utils into the extension.
In the process we understood the following:
core
namespaceKeeping the reference to the core
schema leads to the error posted above. Removing the line with - namespace: core
from allows to generate documentation:
core
and generating the documentationThis can be done by apply the patch associated with pull request #5
Which approach do we want to promote for extension documentation ?
If we decide to go with (1), we would need to:
core
namespace
orI think going with 1.a
would be more sensible. We should also probably leverage inter-sphinx capabilities to ensure references to core schema documentation are solved.
See http://www.sphinx-doc.org/en/stable/ext/intersphinx.html#module-sphinx.ext.intersphinx
To ensure extension A
that depends on extension B
can effectively resolve references to the documentation of extension B
, it would need to know how to find the corresponding documentation.
Few approaches are possible:
or
In both cases (1) and (2) the document generator needs to be fixed because the docs won't link back properly to the docs if you remove the 'core' from the schema.
At least for now, I think we can simplify this problem by focusing on what is needed for the 'core' docs from the nwb-schema repo and the extension docs from the new nwb-extensions repo.
If we want to render only the parts that are new to the extension then a possible solution would be to:
conf_doc_autoge
(i.e., the configuration file that is being used to configure generate_format_docs) Note, there is also an additional variant to this, in that we can set spec_resolve_type_inc
setting for extensions to True. In this case the docs will be always fully resolved (instead of just showing what is new). E.g., if you create an extension of TimeSeries this would show all fields inherited from TimeSeries plus all the changes from the extensions, i.e., it would show the full, final spec not just the parts that have changed.
I like the idea of setting spec_resolve_type_inc=True
. In fact, I like that in general, as opposed to requiring the user to follow links back through the documentation.
I have 2 example extensions up, each with documentation rendered, including the entire core. Right now it's difficult to know which classes belong to the extension, so I think limiting what is shown to new extension classes would be very useful.
make apidoc
gives the following error: