NKI-AI / ahcore

Ahcore is the AI for Oncology core computational pathology toolkit
Apache License 2.0
15 stars 1 forks source link

ZarrWriter is unable to access keywords from model output dictionary of foundational models #81

Closed AjeyPaiK closed 2 months ago

AjeyPaiK commented 3 months ago

Describe the bug When I'm using the zarrwriter callback to write extracted features to disc, it breaks since the dictionary returned by the feature encoder is not hasable.

To Reproduce Load a jit-ed foundational model and run feature extraction.

Expected behavior This is not clear to me yet. There are multiple ways to handle it.

  1. One way could be to specify what member of the model output dictionary actually needs to be written (cls token or patch embeddings) in the model config.
  2. Other way is to deal with this in the configuration for the zarrwriter.

Environment dlup version: 0.3.37 How installed: pip install -e .[dev] Python version: 3.11 Operating System: Linux

Additional context Screenshots of the model output dictionary and the corresponding error that the writer throws. image image