Improve linting and execute on code

[paddyroddy]

This PR adds ruff with a lot of rules and changes (sorry). Anything unclear/disputed, please ask away. Have also changed the line length from 79 to 88 as I couldn't see a reason not to.

I haven't tried to add mypy here (#164). One for a separate PR.

[deprecated-napari-hub-preview-bot[bot]]

Preview page for your plugin is ready here: https://preview.napari-hub.org/quantumjot/BayesianTracker/204 Updated: 2023-03-07T15:43:42.943562

[codecov-commenter]

Codecov Report

Patch coverage: 87.98% and project coverage change: +1.01 :tada:

Comparison is base (0c61b83) 87.53% compared to head (43ff60d) 88.54%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #204 +/- ## ========================================== + Coverage 87.53% 88.54% +1.01% ========================================== Files 19 19 Lines 1548 1519 -29 ========================================== - Hits 1355 1345 -10 + Misses 193 174 -19 ``` | [Impacted Files](https://codecov.io/gh/quantumjot/BayesianTracker/pull/204?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+R+Lowe) | Coverage Δ | | |---|---|---| | [btrack/dataio.py](https://codecov.io/gh/quantumjot/BayesianTracker/pull/204?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+R+Lowe#diff-YnRyYWNrL2RhdGFpby5weQ==) | `0.00% <0.00%> (ø)` | | | [btrack/datasets.py](https://codecov.io/gh/quantumjot/BayesianTracker/pull/204?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+R+Lowe#diff-YnRyYWNrL2RhdGFzZXRzLnB5) | `70.37% <57.14%> (+11.54%)` | :arrow_up: | | [btrack/io/exporters.py](https://codecov.io/gh/quantumjot/BayesianTracker/pull/204?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+R+Lowe#diff-YnRyYWNrL2lvL2V4cG9ydGVycy5weQ==) | `83.33% <75.00%> (ø)` | | | [btrack/io/hdf.py](https://codecov.io/gh/quantumjot/BayesianTracker/pull/204?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+R+Lowe#diff-YnRyYWNrL2lvL2hkZi5weQ==) | `83.20% <77.50%> (+0.44%)` | :arrow_up: | | [btrack/io/utils.py](https://codecov.io/gh/quantumjot/BayesianTracker/pull/204?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+R+Lowe#diff-YnRyYWNrL2lvL3V0aWxzLnB5) | `90.90% <80.00%> (+0.21%)` | :arrow_up: | | [btrack/libwrapper.py](https://codecov.io/gh/quantumjot/BayesianTracker/pull/204?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+R+Lowe#diff-YnRyYWNrL2xpYndyYXBwZXIucHk=) | `97.05% <80.00%> (ø)` | | | [btrack/optimise/optimiser.py](https://codecov.io/gh/quantumjot/BayesianTracker/pull/204?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+R+Lowe#diff-YnRyYWNrL29wdGltaXNlL29wdGltaXNlci5weQ==) | `86.41% <83.33%> (+2.69%)` | :arrow_up: | | [btrack/btypes.py](https://codecov.io/gh/quantumjot/BayesianTracker/pull/204?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+R+Lowe#diff-YnRyYWNrL2J0eXBlcy5weQ==) | `83.57% <91.89%> (+1.27%)` | :arrow_up: | | [btrack/io/\_localization.py](https://codecov.io/gh/quantumjot/BayesianTracker/pull/204?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+R+Lowe#diff-YnRyYWNrL2lvL19sb2NhbGl6YXRpb24ucHk=) | `91.76% <92.85%> (-0.10%)` | :arrow_down: | | [btrack/utils.py](https://codecov.io/gh/quantumjot/BayesianTracker/pull/204?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+R+Lowe#diff-YnRyYWNrL3V0aWxzLnB5) | `92.07% <96.42%> (+2.17%)` | :arrow_up: | | ... and [10 more](https://codecov.io/gh/quantumjot/BayesianTracker/pull/204?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+R+Lowe) | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+R+Lowe). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+R+Lowe)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[p-j-smith]

Good news: everything is passing now. Bad news: this is a huge pr with a lot of changes. I'll try to summarise them here:

• tox and github actions changed:

  • Add support for tox>=4
  • remove use of tox-conda - it's not needed as all dependencies are installed using pip
  • remove separate napari test env. Having a separate napari test env means we run lots of the (non-napari) tests twice. Instead, let's run them all once
  • set environment variables in tox.ini for napari tests to run in CI

• type hint changes:

  • replace use of Union with the | operator
  • replace typing generics (List, Tuple, etc.) with standard collections (list, tuple etc.)
  • use numpy.typing to replace np.ndarray with npt.NDArray

• pydantic typing overrides:

  • for anything that pydantic validates, the above typing changes must be undone, i.e. use Union and List instead of | and list
  • this is necessary for Python 3.8 - even though we import annotations from __future__, pydantic will still complain that e.g. list is not subscriptable

• lots of formatting changes made by black, e.g. replace single quotes with double quotes

• remove magic numbers and replace with constants, e.g. TWO_DIM = 2

• modifying tests is usually a bad idea, but there are a couple of changes made here

  • in the function create_test_object, rename the parameter id to test_id so that we do not use the name of a builtin
  • apply type hint changes described above to the tests

• some changes we might want to reverse:

  • modifications to format of the data files, e.g. models/particle_config.json
  • Hypothesis.type has been renamed to Hypothesis.hypothesis_type, again to avoid using the name of a builtin. But this would be a breaking change for users and so probably a bad idea

Let me know if there's anything else that is not clear!

[quantumjot]

In general, I think this look great. I don't have time to thoroughly go through this, but a couple of thoughts:

modifications to format of the data files, e.g. models/particle_config.json

generally is fine, except we lose some readability from the users perspective - in particular, the matrices are not shaped correctly due to the formatting changes.

Hypothesis.type has been renamed to Hypothesis.hypothesis_type, again to avoid using the name of a builtin. But this would be a breaking change for users and so probably a bad idea

This is fine (actually good) - this is not really a public interface anyway, so users shouldn't be directly setting these.

[p-j-smith]

in particular, the matrices are not shaped correctly due to the formatting changes

okay, I'll make the matrices look nice again but leave the other formatting changes

This is fine (actually good) - this is not really a public interface anyway, so users shouldn't be directly setting these.

thanks for the clarification - let's stick with the new name then

[p-j-smith]

I've made those few changes and I think this is ready to be merged now, but would you prefer me to wait until #205 is merged first @quantumjot ?

[quantumjot]

I would go ahead and merge. #205 is pretty simple, but might need a new test adding