where is the yolo_head.py

[lu-kyo]

Hi, thank you for the impressive work from your team. I have learned a lot. When I am running the code for run_patch and run_wsi, I always encounter an error at this location: "/project/DPDS/Xiao_lab/s181141/yolo_mask/metayolo/models/yolo_head.py". However, this file does not exist. Where can I find and modify yolo_head.py? Thanks for your help kynoa

[impromptuRong]

Hi Kynoa,

Are you using docker or setup the environment with the conda file? Also which model are you using? The lung cancer model or the breast cancer model or the model built by yourself? Would you mind also attaching the conda environment details here? conda env export > test.yaml

[lu-kyo]

Hi,again, thanks for your help. At the beginning, I did not connect Docker with the environment. Now, I have analyzed the results. Thanks a lot

[LIYAOSONGLIYAOSONG]

Thank you for your excellent work. I have gained a lot from your work on 'A Deep Learning Approach for Histology Based Nucleus Segmentation and Tumor Microenvironment Characterization'. When I tried to run the run_ctch_inference. py file in the pycharm environment<! python run_patch_inference.py --data_path /usr/users/space_lys/wsi-master_0/wsi-master/datasets/breast_cancer --model brca ’>， Also encountered error: 'Traceback of TorchScript, original code (most recent call last):' File "/project/DPDS/Xiao_lab/s181141/yolo_mask/metayolo/models/yolo_head.py", line 346, in forward if compute_masks and n_obj_per_image[idx] > 0: m = mask_probs[idx] mask_labels = self.mask_indices[r['labels'].clamp(min=0.)] # convert -100 to 0, obj_labels 0(unclassified), 1~nc

index = torch.arange(len(m),  device=mask_labels.device)
r['masks'] = m[index, mask_labels][:, None]
Runtime Error: tensors used as indicators must be long, int, byte or boolean tensors "However, this file does not exist. Where can I find and modify yolo_ head. py?

[impromptuRong]

Hi Liyao,

The model is deployed as torchscript model, so the source code of model definition is not included in this repo. Is it possible to share the image patch that cause the error? I can take a look and get back to you shortly

[LIYAOSONGLIYAOSONG]

Hello, thank you very much for your timely reply, which gave me some ideas. The dataset I downloaded is the address https://sites.google.com/view/nucls/ mentioned in your paper. I would like to ask if the source code of the model definition can be put in the warehouse without affecting the operation of each .py file. This way, it will be more convenient to modify it every time an error occurs, without bothering you.

The following is my parameter configuration for the run_patch_inference.py file: if name == 'main': parser = argparse.ArgumentParser('Patch inference with HD-Yolo.', add_help=True) parser.add_argument('--data_path', required=True, type=str, help="input data filename or directory.") # parser.add_argument('data_path', nargs ='?', type=str, help="Input data filename or directory.") parser.add_argument('--meta_info', default='meta_info.yaml', type=str, help="A yaml file contains: label texts and colors.") parser.add_argument('--model', default='brca', type=str, help="Model path, torch jit model." ) parser.add_argument('--output_dir', default=None, type=str, help="Output folder.") parser.add_argument('--device', default='cuda', choices=['cuda', 'cpu'], type=str, help='Run on cpu or gpu.') parser.add_argument('--mpp', default=None, type=float, help='Input data mpp.')

parser.add_argument('--batch_size', default=64, type=int, help='Number of batch size.')

parser.add_argument('--num_workers', default=64, type=int, help='Number of workers for data loader.') parser.add_argument('--box_only', action='store_true', help="Only save box and ignore mask.")

parser.add_argument('--export_text', action='store_true', help="If save_csv is enabled, whether to convert numeric labels into text.")

In the configs.py file, only the following code was changed, and the others were unchanged: MODEL_PATHS = { 'lung': './selected_models/benchmark_lung/lung_best.float16.torchscript.pt', 'brca': '/usr/users/space_lys/hd_wsi-master_0/hd_wsi-master/weights/fold3_epoch201.float16.torchscript.pt', ...... } 非常感谢，再次，期待您的回复。

[impromptuRong]

Hi Liyao,

I uploaded the model code to the following repository: https://github.com/impromptuRong/hd_yolo.git. Feel free to give it a try!

(Note: The code is about 4 years old and no longer maintained—sorry for any inconvenience.)

Due to the rapid evolution of similar models, the hd_wsi pipeline now serves more as a general WSI inference framework that can integrate with various models.

Btw, do you happen to know which specific image/image_id in NuCLS that lead to the error? We have the NuCLS dataset downloaded and I would love to know which corner case triggered the error.

[LIYAOSONGLIYAOSONG]

Hello, OK, thank you very much for your prompt reply. When the code executes the data breast_cancer/rgb/TCGA-A1-A0SP-DX1_id-5ea4095addda5f8398977ebc_left-11868_top-53956_bottom-54243_right-12182.png, an error message is displayed. I think it may be caused by the mismatch between my environment and the source environment, rather than the data problem. I will try again later. I will show you all the errors after running the code: ================================== Load hdyolo model: /usr/users/space_lys/wsi-master_0/wsi-master/weights/fold3_epoch201.float16.torchscript.pt to cuda (nms: {'conf_thres': 0.15, 'iou_thres': 0.45, 'max_det': 1000.0} Dataset configs: {'mpp': 0.25, 'patch_size': 512, 'padding': 64, 'page': 0, 'labels': ['bg', 'tumor', 'stromal', 'immune', 'blood', 'macrophage', 'dead', 'other'], 'labels_color': {-100: '#949494', 0: '#ffffff', 1: '#00ff00', 2: '#ff0000', 3: '#0000ff', 4: '#ff00ff', 5: '#ffff00', 6: '#0094e1', 7: '#646464'}, 'labels_text': {-100: 'unlabeled', 1: 'tumor', 2: 'stromal', 3: 'sTILs', 4: 'blood', 5: 'macrophage', 6: 'dead', 7: 'other'}, 'classes': ['bg', 'tumor', 'stromal', 'sTILs', 'blood', 'macrophage', 'dead', 'other']} Inputs: /usr/users/space_lys/wsi-master_0/wsi-master/datasets/breast_cancer (5232 files observed). Outputs: ./patch_results/20241013T190706 ============================ ============================ /usr/users/space_lys/wsi-master_0/wsi-master/datasets/breast_cancer/rgb/TCGA-A1-A0SP-DX1_id-5ea4095addda5f8 398977ebc_left-11868_top-53956_bottom-54243_right-12182.png Traceback (most recent call last): File "/usr/users/space_lys/wsi-master_0/wsi-master/run_patch_inference.py", line 197, in main(args) File "/usr/users/space_lys/wsi-master_0/wsi-master/run_patch_inference.py", line 125, in main outputs = analyze_one_patch( File "/usr/users/space_lys/wsi-master_0/wsi-master/run_patch_inference.py", line 47, in analyze_one_patch outputs = model (inputs, compute_masks=compute_masks)[1] File "/usr/users/venv/venv_lys/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, *kwargs) File "/usr/users/venv/venv_lys/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(args, **kwargs) RuntimeError: The following operation failed in the TorchScript interpreter. Traceback of TorchScript, serialized code (most recent last call): File "code/torch/metayolo/models/y olo_head.py", line 295, in forward mask_indices = self.mask_indices _96 = annotate(List[Optional[Tensor]], [torch.clamp(r["labels"], 0.)]) mask_labels = torch.index(mask_indices, _96) ~~~ <--- HERE index = torch.arange(torch.len(m), dtype=None, layout=None, device=ops.prim.device(mask_labels)) _97 = annotate(List[Optional[Tensor]], [index, mask_labels]) Traceback of TorchScript, original code (most recent call last): File "/project/DPDS/Xiao_lab/s181141/yolomask/metayolo/models/yolo head.py", line 346, in forward if compute_masks and n_obj_per_image[idx] > 0: m = mask_probs[idx] mask_labels = self.mask_indices[r['labels'].clamp(min=0.)] # convert -100 to 0, obj_labels 0(unclassified), 1~nc ~~~~~~~~~~~~~ <--- HERE index = torch.arange(len(m), device=mask_labels.device) r['masks'] = m[index, mask_labels][:, None] RuntimeError: tensors used as indices must be long, int, byte or bool tensors

Then for this error, as I said before, the path /project/DPDS/Xiao_lab/s181141/yolo_mask/metayolo/models/yolo_head.py cannot be found. I will try again later. Thank you again for your reply.

[LIYAOSONGLIYAOSONG]

Of course, if you have any solution to this error, thank you for sharing it. My running environment is Python-3.10.12 torch-2.0.0+cu118 CUDA:0 (NVIDIA GeForce RTX 3090, 24253MiB)