search

DearCaat / RRT-MIL

[CVPR 2024] Feature Re-Embedding: Towards Foundation Model-Level Performance in Computational Pathology
85 stars 8 forks source link

Issue With PLIP #12

Open Abdullah-Tauqeer01 opened 4 months ago

Abdullah-Tauqeer01 commented 4 months ago

Hey I am facing an issue in forwarding the image batch through the PLIP model, I followed everything exactly as instructed in the readme file, and I have installed the clam_latest environment from official clam repository, the error is given below:

quantimb02 44 % cd /local/data0/home/atauqeer/Code/CLAM ; /usr/bin/env /local/data0/software/miniconda3-23.11/envs/clam_latest/bin/python /local/data0/home/atauqeer/pc_vscode/q01/.vscode-server/extensions/ms-python.debugpy-2024.8.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher 49945 -- /local/data0/home/atauqeer/Code/CLAM/extract_features_fp.py --data_h5_dir /local/data0/data/CAMELYON_16_AND_17/CAMELYON16/Processed_CAMELYON_512X512/ --data_slide_dir /local/data0/data/CAMELYON_16_AND_17/CAMELYON16/images/ --csv_path /local/data0/data/CAMELYON_16_AND_17/CAMELYON16/Processed_CAMELYON_512X512/process_list_autogen.csv --feat_dir /local/data0/data/CAMELYON_16_AND_17/CAMELYON16/Processed_CAMELYON_512X512/Features_dir/ --batch_size 256 --slide_ext .tif initializing dataset 0%| | 0/379 [00:00<?, ?it/s] progress: 0/379 normal_001 downsample [1. 1.] downsampled_level_dim [ 97792 221184] level_dim [ 97792 221184] name normal_001 patch_level 0 patch_size 512 save_path /local/data0/data/CAMELYON_16_AND_17/CAMELYON16/Processed_CAMELYON_512X512/patches

feature extraction settings transformations: Compose( ToTensor() Normalize(mean=(0.48145466, 0.4578275, 0.40821073), std=(0.26862954, 0.26130258, 0.27577711)) ) processing a total of 8 batches 0%| | 0/8 [05:58<?, ?it/s] 0%| | 0/379 [06:00<?, ?it/s] Traceback (most recent call last): File "/local/data0/software/miniconda3-23.11/envs/clam_latest/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/local/data0/software/miniconda3-23.11/envs/clam_latest/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/local/data0/home/atauqeer/pc_vscode/q01/.vscode-server/extensions/ms-python.debugpy-2024.8.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/main.py", line 39, in cli.main() File "/local/data0/home/atauqeer/pc_vscode/q01/.vscode-server/extensions/ms-python.debugpy-2024.8.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main run() File "/local/data0/home/atauqeer/pc_vscode/q01/.vscode-server/extensions/ms-python.debugpy-2024.8.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file runpy.run_path(target, run_name="main") File "/local/data0/home/atauqeer/pc_vscode/q01/.vscode-server/extensions/ms-python.debugpy-2024.8.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path return _run_module_code(code, init_globals, run_name, File "/local/data0/home/atauqeer/pc_vscode/q01/.vscode-server/extensions/ms-python.debugpy-2024.8.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code _run_code(code, mod_globals, init_globals, File "/local/data0/home/atauqeer/pc_vscode/q01/.vscode-server/extensions/ms-python.debugpy-2024.8.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code exec(code, run_globals) File "/local/data0/home/atauqeer/Code/CLAM/extract_features_fp.py", line 111, in output_file_path = compute_w_loader(output_path, loader = loader, model = model, verbose = 1) File "/local/data0/home/atauqeer/Code/CLAM/extract_features_fp.py", line 42, in compute_w_loader features = model(batch) File "/local/data0/software/miniconda3-23.11/envs/clam_latest/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/local/data0/software/miniconda3-23.11/envs/clam_latest/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, *kwargs) File "/local/data0/home/atauqeer/Code/CLAM/models/plip.py", line 10, in forward return self.model(input).image_embeds File "/local/data0/software/miniconda3-23.11/envs/clam_latest/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/local/data0/software/miniconda3-23.11/envs/clam_latest/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, kwargs) File "/local/data0/software/miniconda3-23.11/envs/clam_latest/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 1298, in forward vision_outputs = self.vision_model( File "/local/data0/software/miniconda3-23.11/envs/clam_latest/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/local/data0/software/miniconda3-23.11/envs/clam_latest/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(args, kwargs) File "/local/data0/software/miniconda3-23.11/envs/clam_latest/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 849, in forward hidden_states = self.embeddings(pixel_values) File "/local/data0/software/miniconda3-23.11/envs/clam_latest/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/local/data0/software/miniconda3-23.11/envs/clam_latest/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(args, **kwargs) File "/local/data0/software/miniconda3-23.11/envs/clam_latest/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 189, in forward embeddings = embeddings + self.position_embedding(self.position_ids) RuntimeError: The size of tensor a (257) must match the size of tensor b (50) at non-singleton dimension 1

DearCaat commented 4 months ago

I'm sorry, but I can't offer any valuable replies about your question. But I think you can print the size of the embedding and pos for the model runtime and check if you have any error about the size when cropping the patches.