adrhill commented 2 years ago

Draft PR for iNNvestigate 2.0 with support for the latest TensorFlow versions (tested on 2.7 and 2.8).

Commits are non-atomic, so this will have to be squash-merged.

Breaking changes

Due to the switch from TF1 to TF2 and therefore also from keras to tf.keras, this is a breaking release.

The following analyzers have temporarily been removed to get a first TF2 release out and will have to be re-implemented:

PatternNet
PatternAttribution
LRPZIgnoreBias
LRPEpsilonIgnoreBias

Running local reference tests

This PR includes thorough tests on reference layers and support for local reference tests on VGG, including reference tests of layer-wise relevances.

To run these local tests, generate the data by following the instructions at https://github.com/adrhill/test-data-innvestigate. Then copy the generated .hdf5 reference files into /tests/references/vgg16 and run pytest -m local.

codecov-commenter commented 2 years ago

Codecov Report

Merging #277 (a1ea049) into master (db8a58e) will decrease coverage by 1.18%. The diff coverage is 70.31%.

:exclamation: Current head a1ea049 differs from pull request most recent head a96d0b4. Consider uploading reports for the commit a96d0b4 to get more accurate results

@@            Coverage Diff             @@
##           master     #277      +/-   ##
==========================================
- Coverage   71.41%   70.22%   -1.19%     
==========================================
  Files          45       42       -3     
  Lines        4216     3369     -847     
  Branches      619      554      -65     
==========================================
- Hits         3011     2366     -645     
+ Misses       1017      846     -171     
+ Partials      188      157      -31

Impacted Files	Coverage Δ
tests/_debug.py	`0.00% <0.00%> (ø)`
tests/_monkeytype.py	`0.00% <0.00%> (ø)`
src/innvestigate/analyzer/relevance_based/utils.py	`32.35% <33.33%> (ø)`
src/innvestigate/utils/__init__.py	`18.18% <33.33%> (ø)`
tests/analyzer/test_reference_vgg16.py	`34.42% <34.42%> (ø)`
src/innvestigate/backend/graph.py	`47.22% <40.65%> (ø)`
tests/networks/base.py	`38.88% <41.93%> (+6.38%)`	:arrow_up:
src/innvestigate/layers.py	`48.09% <50.00%> (-29.69%)`	:arrow_down:
tests/networks/imagenet.py	`51.51% <50.00%> (+27.37%)`	:arrow_up:
src/innvestigate/tools/perturbate.py	`62.43% <57.14%> (-0.55%)`	:arrow_down:
... and 41 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update db8a58e...a96d0b4. Read the comment docs.

adrhill commented 2 years ago

1. Installing Poetry

To ensure reproducibility, install this repo and all of its dev dependencies with Poetry. Either use brew install poetry on macOS or run this cURL one-liner from the Poetry docs:

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -

2. Running precommit tests

Make sure you have an up-to-date local installation of Python (e.g. by running pyenv local 3.10.X) and run:

poetry install

The precommit tests can now be run via

poetry run pytest -m precommit

To only run tests on ResNet50, use

poetry run pytest -m resnet50

3. Publishing on PyPI

Instructions can be found in todo_before_release.md and in the Poetry docs on publish.

albermax / innvestigate