Update packages Jan 2024

[r-b-g-b]

First, thanks for the excellent software. I was having trouble resolving dependencies while installing py_irt into an existing environment since some of the pinned dependency versions are a bit old. I had a go at updating as many as I could, with the exception of torch which I left pinned to <2. The tests pass and my simple tests of the CLI work. I bumped the py_irt version to 0.4.14, but it might warrant a bump to 0.5.0; I'll defer to the package maintainers on that.

Most of the packages only needed minor version updates. Some are functionally equivalent to what is pinned already based on how poetry resolves fuzzy versions, e.g., pyro-ppl = "^1.6.0" (old) and pyro-ppl = "^1.8.6" (new) will both pull in the latest version with major version <2; similarly with numpy = ">=1.22.0" (old) and numpy = ">=1.24.4" (new).

Beyond the tests, I looked into any potentially breaking changes resulting from major updates to the computational packages.

Pandas ^1.1 ➡️ ^2.0.3

I browsed the list of backward incompatible API changes and searched the py_irt codebase for any relevant breaking changes (value_counts, datetime, timedelta). I didn't turn up anything that should affect this codebase.

scikit-learn =1.2 ➡️ ^1.3.2

Although the new pin ^1.3.2 wouldn't allow any major version upgrades, it did stand out as worth mentioning since the old version was pinned to a specific major-minor version, 1.2. Any idea about whether upgrading would cause an issue?

Pydantic ^1.8.2 ➡️ ^2.5.3

A fix required by the upgrade to Pydantic 2 was to set IrtConfig.protected_namespaces (docs) to avoid the following warning:

pydantic/typer CLI output

``` (base) ➜ py-irt git:(update-packages-1-24) poetry run py-irt --help /home/robert/.cache/pypoetry/virtualenvs/py-irt-3fGP65dd-py3.10/lib/python3.10/site-packages/pydantic/_internal/_fields.py:149: UserWarning: Field "model_type" has conflict with You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`. warnings.warn( ``` After setting `protected_namespaces` to `()`, we see the usual typer output ``` (base) ➜ py-irt git:(update-packages-1-24) ✗ poetry run py-irt --help The currently activated Python version 3.8.13 is not supported by the project (>=3.9,<3.12). Trying to find and use a compatible version. Using python3.10 (3.10.11) (base) ➜ py-irt git:(update-packages-1-24) ✗ poetry run py-irt train --help The currently activated Python version 3.8.13 is not supported by the project (>=3.9,<3.12). Trying to find and use a compatible version. Using python3.10 (3.10.11) Usage: py-irt train [OPTIONS] MODEL_TYPE DATA_PATH OUTPUT_DIR ╭─ Arguments ────────────────────────────────────────────────────────────────────────────────────╮ │ * model_type TEXT [default: None] [required] │ │ * data_path TEXT [default: None] [required] │ │ * output_dir TEXT [default: None] [required] │ ╰────────────────────────────────────────────────────────────────────────────────────────────────╯ ╭─ Options ──────────────────────────────────────────────────────────────────────────────────────╮ │ --epochs INTEGER [default: None] │ │ --priors TEXT [default: None] │ │ --dims INTEGER [default: None] │ │ --lr FLOAT [default: None] │ │ --lr-decay FLOAT [default: None] │ │ --device TEXT [default: cpu] │ │ --initializers TEXT [default: None] │ │ --config-path TEXT [default: None] │ │ --dropout FLOAT [default: 0.5] │ │ --hidden INTEGER [default: 100] │ │ --seed INTEGER [default: None] │ │ --deterministic --no-deterministic [default: no-deterministic] │ │ --log-every INTEGER [default: 100] │ │ --help Show this message and exit. │ ╰────────────────────────────────────────────────────────────────────────────────────────────────╯ ```

Tests

The Pytest tests run successfully and I also tested the CLI for train, train-and-evaluate, and evaluate on the test fixtures minitest.jsonlines and minitest.amortized.jsonlines. They ran without error and the loss decreased, but I'm not sure if there are any more subtle bugs I should be looking for.

Example output of `train`

``` (base) ➜ py-irt git:(update-packages-1-24) ✗ poetry run py-irt train 3pl test_fixtures/minitest.jsonlines my3pl --epochs 500 The currently activated Python version 3.8.13 is not supported by the project (>=3.9,<3.12). Trying to find and use a compatible version. Using python3.10 (3.10.11) [10:04:07] config: model_type='3pl' epochs=500 priors=None initializers=[] dims=None lr=0.1 lr_decay=0.9999 cli.py:109 dropout=0.5 hidden=100 vocab_size=None log_every=100 seed=None deterministic=False data_path: test_fixtures/minitest.jsonlines cli.py:111 output directory: my3pl cli.py:112 [10:04:07] amortized: False dataset.py:112 [10:04:07] Vocab size: None training.py:90 Training Model... cli.py:116 args: {'device': 'cpu', 'num_items': 4, 'num_subjects': 4} training.py:134 Parsed Model Args: {'device': 'cpu', 'num_items': 4, 'num_subjects': 4, 'priors': 'vague', training.py:147 'dropout': 0.5, 'hidden': 100, 'vocab_size': None} torch.Size([16]) torch.Size([16]) Training Pyro IRT Model for 500 epochs ┏━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┓ ┃ Epoch ┃ Loss ┃ Best Loss ┃ New LR ┃ ┡━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━┩ │ 1 │ 8017.0768 │ 8017.0768 │ 0.1000 │ │ 101 │ 148.6918 │ 144.4898 │ 0.0990 │ │ 201 │ 112.7594 │ 76.6643 │ 0.0980 │ │ 301 │ 98.9252 │ 64.1975 │ 0.0970 │ │ 401 │ 124.6769 │ 64.1975 │ 0.0961 │ │ 500 │ 95.0697 │ 64.1975 │ 0.0951 │ └───────┴───────────┴───────────┴────────┘ [10:04:12] Train time: 4.540265798568726 cli.py:122 ```

Python version support

Upgrading ipython required dropping support for Python 3.8, which seemed acceptable since official support ends Oct 2024.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (6d6d9a1) 66.92% compared to head (7b83739) 66.95%.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #58 +/- ## ========================================== + Coverage 66.92% 66.95% +0.02% ========================================== Files 15 15 Lines 1158 1159 +1 ========================================== + Hits 775 776 +1 Misses 383 383 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[jplalor]

Thanks! This may also help with #47. I added one comment re: Python 3.8 but after that change I think this should be good to go.

[r-b-g-b]

Done!

[jplalor]

Thanks!