dev: update dependencies, testing, packaging, and linting/formatting

lindsay-stevens commented 5 months ago

Overall this PR modernises pyxform and lints a swathe of code. Most of the commits are progressive application of linter rules, which hopefully makes it clearer what is being done and why - refer to pyproject.toml to see what rules were enabled in that commit.

Changes summary:

update prod and dev/test dependencies to latest versions for Python 3.8
update testing runner from nose 1.3.7 (2015) to built-in unittest, which can do everything required by pyxform. This also revealed some testing issues which are fixed:
- ResourceWarnings due to input XLS/X files not being closed properly. Presumably nose was suppressing warnings so these were not visible until switching to unittest.
- BadChoicesSheetHeaders test expected warnings that either don't exist or apply anymore. This test was not being run by nose for some unknown reason.
- AttributeColumnsTest test expected output was not updated for secondary instances, and the fix to missing itext media. This test was not being run by nose for some unknown reason.
update packaging to use flit since setuptools is no longer recommended for python. Another option is hatch but it looked like it might be a bit overkill and/or introduce OS dependency requirements just for package building.
update linting to use ruff, which has linter/formatter rules that encompass all the existing tools (black, flake8, isort) and many more. An initial batch of extra linting rule sets are enabled and applied.
remove unused and unmaintained sphinx based docs stub folder - not published anywhere and broken anyway

Why is this the best possible solution? Were any other approaches considered?

These changes support upgrading pyxform supported Python versions, since the tools replaced in this PR are either out of date already or will not be supported in future.

A lot of code in pyxform is/was crufty due to it's origins in Python 2, and as a natural result of having dozens of committers over many years. The increased linting moves the code towards a more coherent style that is more consistent with modern Python code. This should make contributions easier in future.

What are the regression risks?

The update of AttributeColumnsTest seems to have revealed a bug introduced by #380 for custom attributes and repeats (will backref this ticket). Nobody seems to have complained about this though so perhaps it's a low priority to fix.

The release publishing GitHub action may not be 100% right since I'm not sure there's a nice way to test that.

Does this change require updates to documentation? If so, please file an issue here and include the link below.

The code should be functionally equivalent for users. Where applicable, the README file has been updated.

Before submitting this PR, please make sure you have:

[x] included test cases for core behavior and edge cases in tests
[x] run ~nosetests~ unittest and verified all tests pass
[x] run ~black pyxform tests~ ruff check pyxform tests & ruff format pyxform tests to lint and format code
[x] verified that any code or assets from external sources are properly credited in comments

lindsay-stevens commented 4 months ago

@lognaturel thanks for the review! I'll wait for #694 to be decided before merging.

lognaturel commented 4 months ago

Let's go ahead and merge! I think we're going to let #694 simmer for a bit. We can always release off a special release branch if needed.

XLSForm / pyxform