search

DIAGNijmegen / pathology-hooknet-tls

MIT License
6 stars 0 forks source link

wholeslidedata #6

Open 4479917 opened 2 months ago

4479917 commented 2 months ago

Dear Mart,

I attempted to run Docker, and it seems that the preinstalled version of wholeslidedata is 0.1.0. However, I encountered some issues as certain files in this version appear to be unavailable. Specifically, I received the following error messages: 'ModuleNotFoundError: No module named 'wholeslidedata.extensions'' and 'ModuleNotFoundError: No module named 'wholeslidedata.annotation.structures'.'

In response, I uninstalled wholeslidedata 0.1.0 and installed version 0.0.16. Yet, I encountered another issue with this version, resulting in the error message: 'AttributeError: module 'wholeslidedata.iterators' has no attribute 'PatchConfiguration'.'

Could you please inform me which version of wholeslidedata you used, and do you have any guidance on how to proceed from here?

Thank you for your assistance.

martvanrijthoven commented 2 months ago

Dear 4479917,

Thank you for your interest in HookNet-TLS. Could you please provide me with the complete error trace? HookNet-TLS should work with the latests wholeslidedata version (main branch = 0.1.0). If you could provide me with the complete error trace, I would be happy to help to debug the problem.

Sorry for the inconvenience.

Best wishes, Mart

4479917 commented 2 months ago

Thank you very much for your prompt response.

I did this "python3 -m hooknettls hooknettls.default.image_path=/home/4479917/example.tif hooknettls.default.mask_path=/home/4479917/example_mask.tif"

It returns: /usr/local/lib/python3.8/dist-packages/wholeslidedata/image/wholeslideimage.py:78: UserWarning: spacing 0.5 outside margin (0.3%) for [50.0, 100.00135898590088, 200.00813007354736, 400.0380039215088, 800.0760078430176, 1600.1520156860352, 3201.694107055664, 6408.956146240234, 12840.243530273438, 25770.278930664062, 51903.521728515625], returning closest spacing: 50.0 warnings.warn( Process CommanderForkProcess-3: Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/wholeslidedata/buffer/patchcommander.py", line 74, in create_message return next(self._messages) StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/usr/local/lib/python3.8/dist-packages/concurrentbuffer/commander.py", line 71, in run self._message(buffer_id) File "/usr/local/lib/python3.8/dist-packages/concurrentbuffer/commander.py", line 74, in _message message = self._commander.create_message(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/wholeslidedata/buffer/patchcommander.py", line 77, in create_message return next(self._messages) StopIteration 2024-04-12 19:15:46.125181: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2024-04-12 19:15:46.612501: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 13797 MB memory: -> device: 0, name: Tesla T4, pci bus id: 0000:41:00.0, compute capability: 7.5 loading weights... /home/user/pathology-hooknet-tls/weights.h5 Create output folder Creating lock file: /output/images/examplehooknettls.lock Run inference Init writers... /usr/local/lib/python3.8/dist-packages/wholeslidedata/image/wholeslideimage.py:78: UserWarning: spacing 0.5 outside margin (0.3%) for [50.0, 100.00135898590088, 200.00813007354736, 400.0380039215088, 800.0760078430176, 1600.1520156860352, 3201.694107055664, 6408.956146240234, 12840.243530273438, 25770.278930664062, 51903.521728515625], returning closest spacing: 50.0 warnings.warn( write: /home/user/tmp/example_hooknettls.tif Creating: /home/user/tmp/example_hooknettls.tif.... Spacing: 50.0 Dimensions: (73703, 50706) Tile_shape: (1024, 1024) write: /home/user/tmp/example_hooknettls_heat1.tif Creating: /home/user/tmp/example_hooknettls_heat1.tif.... Spacing: 50.0 Dimensions: (73703, 50706) Tile_shape: (1024, 1024) write: /home/user/tmp/example_hooknettls_heat2.tif Creating: /home/user/tmp/example_hooknettls_heat2.tif.... Spacing: 50.0 Dimensions: (73703, 50706) Tile_shape: (1024, 1024) Applying... 0it [00:00, ?it/s] /usr/local/lib/python3.8/dist-packages/numpy/core/fromnumeric.py:3432: RuntimeWarning: Mean of empty slice. return _methods._mean(a, axis=axis, dtype=dtype, /usr/local/lib/python3.8/dist-packages/numpy/core/_methods.py:190: RuntimeWarning: invalid value encountered in double_scalars ret = ret.dtype.type(ret / rcount) average batch time: nan average prediction time: nan Saving... Segmentation fault (core dumped)

Not sure if it's an issue about wholeslidedata?

martvanrijthoven commented 2 months ago

It seems that your input image does not contain the correct spacings or wholeslidedata is reading the spacings wrong: [50.0, 100.00135898590088, 200.00813007354736, 400.0380039215088, 800.0760078430176, 1600.1520156860352, 3201.694107055664, 6408.956146240234, 12840.243530273438, 25770.278930664062, 51903.521728515625] The image should contain 0.5 and 2.0 values in this list.

Would it be possible to share the image with me? Then I can try to debug it.

Best wishes, Mart

4479917 commented 2 months ago

Dear Mart,

Thank you very much for your explanation.

For the first input, I used save_image_at_spacing.py from pathology-whole-slide-data to convert svs to tiff with spacing 0.5 and 2. It's file 424218_EAS43977_HE.tif. For the second input, it's file segmented_output.tif. Here in the link are the images: https://drive.google.com/drive/folders/1wABmWVJOr3QGXLWa_EScqOpvkEjyRsd_?usp=sharing

It does not have spacing warnings anymore, but it stopped moving forward after running a while. Here is the traceback: python3 -m hooknettls hooknettls.default.image_path=/home/4479917/424218_EAS43977_HE.tif hooknettls.default.mask_path=/home/4479917/424218_EAS43977_HE_mask.tif 2024-04-17 14:06:32.945552: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2024-04-17 14:06:33.331746: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 13797 MB memory: -> device: 0, name: Tesla T4, pci bus id: 0000:41:00.0, compute capability: 7.5 loading weights... /home/user/pathology-hooknet-tls/weights.h5 Create output folder Creating lock file: /output/images/424218_EAS43977_HEhooknettls.lock Run inference Init writers... write: /home/user/tmp/424218_EAS43977_HE_hooknettls.tif Creating: /home/user/tmp/424218_EAS43977_HE_hooknettls.tif.... Spacing: 0.5025998745463595 Dimensions: (41832, 40844) Tile_shape: (1024, 1024) write: /home/user/tmp/424218_EAS43977_HE_hooknettls_heat1.tif Creating: /home/user/tmp/424218_EAS43977_HE_hooknettls_heat1.tif.... Spacing: 0.5025998745463595 Dimensions: (41832, 40844) Tile_shape: (1024, 1024) write: /home/user/tmp/424218_EAS43977_HE_hooknettls_heat2.tif Creating: /home/user/tmp/424218_EAS43977_HE_hooknettls_heat2.tif.... Spacing: 0.5025998745463595 Dimensions: (41832, 40844) Tile_shape: (1024, 1024) Applying... 0%| | 0/953 [00:00<?, ?it/s]2024-04-17 14:06:35.098043: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8100 0%|▍ | 2/953 [00:06<43:05, 2.72s/it]

It stops here. Could you kindly provide some guidance on how to proceed? Thank you very much for your time and help!

martvanrijthoven commented 2 months ago

It might be that this is due to a shared memory issue. If you are using docker could you please try to add --shm-size=4G when starting the docker.

Please let me know if it still hangs or if you are not using docker.

4479917 commented 2 months ago

Thanks a bunch of for your help on this Mart! I added --shm-size=4G, now it can proceed. But stopped at the step of writing json file.

Here is the traceback: Applying... 0%| | 0/953 [00:00<?, ?it/s]2024-04-19 14:35:57.449930: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8100 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 953/953 [05:07<00:00, 3.10it/s] average batch time: 0.004079683488156616 average prediction time: 0.2824230131481876 Saving... Total time was 13453 Total reading time was 0 Total base writing time was 3787 Total pyramid downsampling time was 0 Total pyramid writing time was 9666 Total time determining min/max was 953 Copying from: /home/user/tmp/424218_EAS43977_HE_hooknettls.tif Copying to: /output/images/424218_EAS43977_HE_hooknettls.tif Removing tmp file... Copying done. Total time was 16500 Total reading time was 0 Total base writing time was 4932 Total pyramid downsampling time was 0 Total pyramid writing time was 11568 Total time determining min/max was 953 Copying from: /home/user/tmp/424218_EAS43977_HE_hooknettls_heat1.tif Copying to: /output/images/424218_EAS43977_HE_hooknettls_heat1.tif Removing tmp file... Copying done. Total time was 16944 Total reading time was 0 Total base writing time was 5026 Total pyramid downsampling time was 2 Total pyramid writing time was 11918 Total time determining min/max was 953 Copying from: /home/user/tmp/424218_EAS43977_HE_hooknettls_heat2.tif Copying to: /output/images/424218_EAS43977_HE_hooknettls_heat2.tif Removing tmp file... Copying done. Stopping iterator Releasing lock file /output/images/424218_EAS43977_HEhooknettls.lock writing json file: /output/images/filtered/424218_EAS43977_HE_hooknettls_tls_filtered.json Traceback (most recent call last): File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/user/pathology-hooknet-tls/hooknettls/main.py", line 8, in objects = build_config(config_reader.read()["default"]) File "/usr/local/lib/python3.8/dist-packages/dicfg/factory.py", line 124, in build_config return _ObjectFactory(deepcopy(config)).build_config() File "/usr/local/lib/python3.8/dist-packages/dicfg/factory.py", line 26, in build_config return self._build(self._configuration) File "/usr/lib/python3.8/functools.py", line 912, in _method return method.get(obj, cls)(*args, *kwargs) File "/usr/local/lib/python3.8/dist-packages/dicfg/factory.py", line 38, in _build_dict config[key] = self._build_object(value) File "/usr/local/lib/python3.8/dist-packages/dicfg/factory.py", line 67, in _build_object return attribute(args, *kwargs) File "/home/user/pathology-hooknet-tls/hooknettls/postprocessing.py", line 235, in gc_filtering filtered_tls_annotations = filter_gc_annotations( File "/home/user/pathology-hooknet-tls/hooknettls/postprocessing.py", line 147, in filter_gc_annotations if not in_tls(tls_tree, gc_annotation.geometry): File "/home/user/pathology-hooknet-tls/hooknettls/postprocessing.py", line 131, in in_tls tls_box = box(tls_annotation.bounds) AttributeError: 'numpy.int64' object has no attribute 'bounds'

I really appreciate your time and patience. Thanks a lot in advance.

martvanrijthoven commented 2 months ago

You are welcome :)

For your latest problem it seems that shapely2.x has updated the interface of strtree. I have updated the docker file to install shapely1.8.4, which should work with hooknet-tls code. I will soon try to update the code to make it compatible with shapely2.0, but if you now rebuild the docker with the updated dockerfile, it should not give this error anymore.

Sorry for the inconvenience. Please let me know if you still run into issues.

4479917 commented 2 months ago

Thank you very much for your great help!

Best, Ranran

4479917 commented 1 month ago

Dear Mart, I really appreciate this great work you have done in TLS detection. Is it possible for you to share the script used for pretraining the weights? Your assistance in this matter would be greatly valuable. Thank you very much in advance!

martvanrijthoven commented 1 month ago

Dear Ranran,

For training i used the wholeslidedata package en hooknet model. You can find example code to train it here: https://github.com/DIAGNijmegen/pathology-hooknet/blob/master/notebooks/HookNetPracticalGuide.ipynb

Please let me know if you have any further questions.

Best wishes, Mart

4479917 commented 1 month ago

Dear Mart,

Thank you for your guidance. I tried that ipynb, the example image /tmp/TCGA-21-5784-01Z-00-DX1.tif from https://drive.google.com/uc?id=1NefnQu3e0l4WR7Xb809gpVORXrKpJb0q cannot be found, so I downloaded TCGA-21-5784-01Z-00-DX1.svs from TCGA directly and then converted it to tif with spacings 0.5 and 2, but the size of this tif seems different from the example data.

I am wondering if the data you used to pretrain HookNet-TLS corresponds to the datasets provided in this repository (the annotations are 1020 annotations_approx_05um xml files which you uploaded recently, and the images are their corresponding 1020 TIF files, which need to be downloaded from TCGA?) Additionally, could you advise on the appropriate processing steps for the .svs images obtained from TCGA? Should they be converted to .tif format with spacings of 0.5 and 2, or is there another approach to ensure that the tif can fit the annotations you provided?

Thank you very much!

Best, Ranran

martvanrijthoven commented 1 month ago

Dear Ranran,

I have seen your email and responded, lets discuss this further in our meeting :).

Best wishes, Mart