Difference in CoNSeP results

[FabianHoerst]

• TIA Toolbox version: 1.5.1
• Python version: 3.11.9
• Operating System: Ubuntu

Description

I tried to reproduce the CoNSeP results mentioned in the HoVerNet paper. For this, I used the test images of the CoNSeP dataset, performed inference and calculated the results given the compute_stats.py from the HoVerNet repository. However, I "just" achieve: [ 0.8365, 0.5349, 0.6565, 0.7670, 0.5051] vs. the referenced scores in the paper: [ 0.8530, 0.5710, 0.7020, 0.7780, 0.5470]

What I Did

import logging
import warnings
import os
os.environ["HOME"] = "/home/jovyan/hovernet"

if logging.getLogger().hasHandlers():
    logging.getLogger().handlers.clear()

import cv2
import joblib
import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np
from pathlib import Path

from tiatoolbox import logger
from tiatoolbox.models.engine.nucleus_instance_segmentor import NucleusInstanceSegmentor
from tiatoolbox.utils.misc import download_data, imread
from natsort import natsorted as sorted

ON_GPU = True
INPUT_FOLDER = "consep-test-images"
OUTPUT_FOLDER = "results"

if __name__ == "__main__":
    filelist = [str(f) for f in sorted(Path(INPUT_FOLDER).glob("*.png"))]
    inst_segmentor = NucleusInstanceSegmentor(
        pretrained_model="hovernet_original-consep",
        num_loader_workers=4,
        num_postproc_workers=4,
        batch_size=4,
    )
    tile_output = inst_segmentor.predict(
        filelist,
        save_dir=OUTPUT_FOLDER,
        mode="tile",
        on_gpu=ON_GPU,
        crash_on_exception=True,
    )

[shaneahmed]

The original implementation was in TensorFlow. TIAToolbox has implemented a version of HoVerNet for faster processing with different tile/patch sizes. For original code, please see https://github.com/vqdang/hover_net/tree/pytorch?tab=readme-ov-file#comparison-to-original-tensorflow-implementation

[FabianHoerst]

Thank you very much!