Update docs for "add_additional_resource" and other fixes

[GraemeWatt]

I finally got around to addressing #197 that I opened in June 2022. In doing so, I encountered many basic problems with my local installation (on macOS 13.5.2 with Python 3.11) including running the tests and building the docs. I've tried to fix all the problems that I encountered. Consequently, the scope of this PR is much larger than originally intended. I'll try to summarise the problems and my fixes below.

• I've added new sections "Adding resource links or files" in usage.rst to explain the add_additional_resource method of the Submission and Table objects. I extended the method to support an additional argument file_type="HistFactory" and modified the test_additional_resource_size function. I modified all five example Jupyter notebooks to use the add_additional_resource method. Closes #197.
• I added ROOT v6.28 to the test matrix in tests.yml.
• I changed https://zenodo.org/record/4946277 (v0.6.0 of hepdata_lib) written in the __init__ method of the Submission class to https://doi.org/10.5281/zenodo.1217998 (the Concept DOI). Closes #212.
• I've updated .readthedocs.yml following current recommendations and based on experience with the main HEPData/hepdata repo (closes #227). In particular, installing hepdata_lib should (hopefully!) build the module index (closes #138). I won't know if it works until I open this PR, and some further tweaks might be needed, so I'm going to open a draft PR to start with.
• I added the hepdata_lib.c_file_reader and hepdata_lib.helpers modules to the docs.
• The method for running the tests python setup.py test mentioned in dev.rst is deprecated and it does not even work for me ("Ran 0 tests in 0.000s"). It is not used in the CI (tests.yml) which instead uses python -m pytest tests. I've changed dev.rst to instead recommend using pytest tests. I didn't change another instance of python setup.py test in release.yml which does not seem to be used recently.
• The CI (tests.yml) has a command python -m pip install '.[test]' which gives WARNING: hepdata-lib 0.13.0 does not provide the extra 'test'. This is because extras_require needs to be defined in setup.py, which I've now done with the requirements needed to run the tests locally on my system. The existing CI still works despite this bug because the test requirements are all installed in the previous step. I've also removed the deprecated setup_requires and tests_require.
• I've added a new section "Building the documentation" at the end of dev.rst. I added Sphinx<7 to the docs/requirements.txt file as the build fails with Sphinx v7. I replaced m2r2 with sphinx-mdinclude to avoid a version conflict with Mistune when installing Jupyter in the same virtual environment.
• Running Sphinx locally gave many warnings/errors, for example, due to missing line breaks. I've added SPHINXOPTS = -W --keep-going to the Makefile and fail_on_warning: true to .readthedocs.yml. This should help avoid building broken docs. For example, the links to the Jupyter notebooks and Binder links in Further examples are not working properly.
• I've added an explanation to setup.rst to mention use of the standard venv module rather than virtualenv/virtualenvwrapper.
• I managed to get almost everything working under Python v3.11, so I extended the test matrix and pushed to GitHub. Then I realised I should run Pylint and found that the installed v2.9.6 is not compatible with Python 3.11, because it installs wrapt v1.12.1 via astroid. We could upgrade Pylint to a more recent version, but the pylintrc files would need to be updated, since they use many obsolete options. For now, I removed support for Python v3.11 and kept Pylint v2.9.6. I added a section on "Analysing the code" in dev.rst to get the developer to run the same commands as used in GitHub Actions. I got several error messages that I didn't understand, so for now I simply added # pylint: disable-msg comments to the offending lines.

---

:books: Documentation preview :books:: https://hepdata-lib--233.org.readthedocs.build/en/233/

[codecov-commenter]

Codecov Report

Merging #233 (0d77f11) into main (9330eb9) will increase coverage by 0.02%. The diff coverage is 100.00%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

@@            Coverage Diff             @@
##             main     #233      +/-   ##
==========================================
+ Coverage   88.52%   88.54%   +0.02%     
==========================================
  Files           4        4              
  Lines         976      978       +2     
  Branches      202      203       +1     
==========================================
+ Hits          864      866       +2     
  Misses         82       82              
  Partials       30       30              

Flag	Coverage Δ
unittests-3.10	88.54% <100.00%> (+0.02%)	:arrow_up:
unittests-3.6	88.21% <100.00%> (+0.02%)	:arrow_up:
unittests-3.7	88.21% <100.00%> (+0.02%)	:arrow_up:
unittests-3.8	88.34% <100.00%> (+0.02%)	:arrow_up:
unittests-3.9	88.34% <100.00%> (+0.02%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
hepdata_lib/__init__.py	90.57% <100.00%> (+0.05%)	:arrow_up:
hepdata_lib/helpers.py	77.69% <ø> (ø)

[matthewfeickert]

@GraemeWatt I'm about to go on holiday for a week and I won't have internet access, so while I might be able to do a first pass on this before I go to bed tonight I think I'll defer everything to your judgement and @clelange review. Thanks for this PR!

[GraemeWatt]

@GraemeWatt I'm about to go on holiday for a week and I won't have internet access, so while I might be able to do a first pass on this before I go to bed tonight I think I'll defer everything to your judgement and @clelange review. Thanks for this PR!

Sorry, I meant to comment that your review would be optional and only if you had spare time. I just thought I'd add you as a Reviewer since you contributed to improving this repository earlier in the year.

[GraemeWatt]

Hi @GraemeWatt - thanks, this changes quite a lot, but I agree with @matthewfeickert that things overall look good. I left a couple of suggestions/comments in the code.

Two general questions:

• Why did you change the formatting in so many places, in particular those lines/separators?
• Do we really need to disable pylint for missing imports in so many places (# pylint: disable-msg=E0401) or is this a setup issue on your side?

Thanks for the useful review! I've commented separately on your individual points. The formatting changes were necessary to avoid errors/warnings about incorrectly formatted .rst files when running Sphinx. I've now removed the # pylint: disable-msg comments that I added previously and fixed the Pylint errors by making some other changes (adding __init__.py, importing test_utilities.py as a relative module after hepdata_lib, adding math and array modules to extension-pkg-whitelist in pylintrc). My setup is not unusual (macOS 13.5.2 with Python 3.10), so I don't know why these errors didn't show up previously in the CI.

I also renamed the master branch to main as for other HEPData repositories (hope you don't mind). I updated the actions in the GitHub Actions workflows to hopefully close some of the open Dependabot PRs.

Is the release.yml workflow currently being used? If not, removing it from the repository would reduce maintenance. It seems to use many unofficial actions. I didn't update the python setup.py test line. If there's a reason why actions should not be upgraded, the dependabot.yml file should have an ignore option.

[clelange]

Is the release.yml workflow currently being used? If not, removing it from the repository would reduce maintenance. It seems to use many unofficial actions. I didn't update the python setup.py test line. If there's a reason why actions should not be upgraded, the dependabot.yml file should have an ignore option.

Yes, we should remove this action, there are better ones available now. I'll create an issue.

[clelange]

Actually, we have this in https://github.com/HEPData/hepdata_lib/issues/165 already, I renamed it slightly.

[clelange]

Thanks for addressing my comments, I'll merge this now.