Empty of WSI inference

[jjhbw]

Hi @simongraham , its me again.

I got WSI-inference to work by using a smaller WSI, as you suggested in #14 . However, the output looks weird, see below. The PNG that's produced is all black, similar to the way the tile-level inference used to as described in #2 . The dat file is basically empty (473 bytes), which also seems wrong.

output_wsi_infer/
├── [   82]  dat
│   └── [  473]  TCGA-QF-A5YT-01Z-00-DX1.BA524390-A937-4356-B6AB-C686CD19EC7F.dat
└── [   82]  tissue
    └── [ 1.3M]  TCGA-QF-A5YT-01Z-00-DX1.BA524390-A937-4356-B6AB-C686CD19EC7F.png

The tail end of the script's stdout looks like the below. Note the Post Proc Gland & Lumen (6/0) at the end, which may be part of the problem.

|2023-02-11|14:22:06.183| [INFO] Inference Time: 6531.991563773714
|2023-02-11|14:29:27.674| [INFO] Nuclei Post Proc Time: 441.39994969870895
|2023-02-11|14:30:53.744| [INFO] Tissue Region Post Proc Time: 86.06777512375265
|2023-02-11|14:32:53.633| [INFO] Gland & Lumen Post Proc Time: 119.88807615451515
|2023-02-11|14:32:54.275| [INFO] Overall Time: 7185.857540397905
|2023-02-11|14:32:54.275| [INFO] Finish
/cerberus/.conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/cerberus/.conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/cerberus/.conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/cerberus/.conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/cerberus/.conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/cerberus/.conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/cerberus/.conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/cerberus/.conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/cerberus/.conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/cerberus/.conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/cerberus/.conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/cerberus/.conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))

Post Proc Gland & Lumen (6/0)

[simongraham]

Hi,

Yes, something seems to be wrong here. I am actively exploring the issue.

[simongraham]

All sorted now, along with switching to mat file output for tissue type prediction. Please verify and close the issue if you are happy.

[jjhbw]

Thanks, it seems to . However, I see your removed dataset.yml from the root of the repo in cc29a5f144ba2c6d6a5229511ab520b68ec8b1de, so now it fails with the below error:

/cerberus/.conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
Traceback (most recent call last):
  File "run_infer_wsi.py", line 142, in <module>
    infer.process_wsi_list(run_args)
  File "/cerberus/infer/wsi.py", line 972, in process_wsi_list
    ioconfig, ioconfig_pp, wsi_idx, wsi_basename, self.output_dir
  File "/cerberus/infer/wsi.py", line 639, in process_single_file
    with open("dataset.yml") as fptr:
FileNotFoundError: [Errno 2] No such file or directory: 'dataset.yml'

As a test, I simply un-deleted dataset.yml by partially reverting cc29a5f144ba2c6d6a5229511ab520b68ec8b1de and ran the script again. I'm waiting for the results now...

[simongraham]

Added it back to the repo 👍🏼

Fingers crossed all works fine now.

[jjhbw]

Judging by the filenames, I now get reasonable-looking output:

output_wsi_infer/
├── [   82]  dat
│   └── [ 546M]  TCGA-QF-A5YT-01Z-00-DX1.BA524390-A937-4356-B6AB-C686CD19EC7F.dat
└── [   82]  tissue
    └── [ 538M]  TCGA-QF-A5YT-01Z-00-DX1.BA524390-A937-4356-B6AB-C686CD19EC7F.mat

2 directories, 2 files

So this issue seems fixed, thanks!