search

dianna-ai / dianna

Deep Insight And Neural Network Analysis
https://dianna.readthedocs.io
Apache License 2.0
48 stars 13 forks source link

Enable DIANNA to take images with multiple channels as input #334

Open elboyran opened 2 years ago

elboyran commented 2 years ago

DIANNA should be able to process images as inputs and models trained with more than 3 (e.g. RGB) channels.

Inspired by the Lorentz workshop CNN-post-processing use case. For example data and quick fix, see the link in the Lorenz code usecase branch by @gcroci2. Have in mind that the model in this code also accepts multiple (in this case 2) inputs. The latter is related to issue #333.

loostrum commented 2 years ago

This would be great indeed! I did some quick tests during the workshop: RISE works as-is with more than 3 channels, but it's good to know that it applies the same mask to every channel. LIIME doesn't work: the lime explainer (i.e. the external package we use) only accepts 3 channels. For greyscale we fixed this by duplicating the channels axis. For more than 3 channels we would need to submit a PR to lime or reimplement it ourselves. I haven't tested KernelSHAP.

elboyran commented 1 year ago

Also follows from #406