About data_preparation.ipynb in nsd.

[zhangdanfeng888]

Hello, I don't understand this part:

"Run the code(https://github.com/styvesg/nsd/blob/master/data_preparation.ipynb) to obtain the textual descriptions of the stimulus images from NSD in the COCO dataset. Rename the corresponding file as "trn_cocoID_correct.npy" or "test_cocoID_correct.npy" and save it in the path "your_folder/data/utils_data/"."

The file(data_preparation.ipynb) has too much code, should I run all this code in sequence? Or can you provide me the “trn_cocoID_correct.npy” and “test_cocoID_correct.npy” files? Thanks a lot.

[ReedOnePeck]

https://github.com/ReedOnePeck/MindDiffuser/tree/main/data/utils_data

[ReedOnePeck]

Download this file cocoID.zip

[zhangdanfeng888]

Download this file cocoID.zip下载此文件 cocoID.zip

Thanks. I download the cocoID.zip then unzip it, it only contains four files: cocoId_correct_sub1.npy, cocoId_correct_sub2.npy, cocoId_correct_sub5.npy and cocoId_correct_sub7.npy, should I rename these four files to "trn_cocoID_correct_sub.npy" or "test_cocoID_correct_sub.npy”"?

[ReedOnePeck]

No, thanks for your reminder, I have revised the readMe

[zhangdanfeng888]

Okay, I'll try that in a minute. Thank you. By the way, there is a slight error here: It should be "environment_1.yaml", haha.

[ReedOnePeck]

Thanks for your reminder

[zhangdanfeng888]

@ReedOnePeck I find another question when I run python Feature extractor/Structural_feature_extraction.py:

Traceback (most recent call last): File "Feature extractor/Structural_feature_extraction.py", line 113, in main() File "Feature extractor/Structural_feature_extraction.py", line 89, in main stim_sub1_val_CLIP = CLIP_extraction(args.stim_sub1_val_multi, args.target_layers) File "Feature extractor/Structural_feature_extraction.py", line 59, in CLIP_extraction feature_maps = FE.get_all_feature_maps(model, stimulus_loader, layers_to_retain=target_layers, File "/data/zhangdf/sihuo/MindDiffuser/packages/feature_extraction_detach.py", line 230, in get_all_feature_maps feature_maps = get_empty_feature_maps(model, next(iter(inputs))[:3], input_size, File "/data/zhangdf/sihuo/MindDiffuser/packages/feature_extraction_detach.py", line 196, in get_empty_feature_maps empty_feature_maps = get_feature_maps(model, inputs, layers_to_retain, remove_duplicates) File "/data/zhangdf/sihuo/MindDiffuser/packages/feature_extraction_detach.py", line 169, in get_feature_maps model(inputs) File "/home/inspur/anaconda3/envs/MindDiffuser/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1128, in _call_impl result = forward_call(*input, kwargs) File "/data/zhangdf/sihuo/MindDiffuser/src/CLIP/clip/model.py", line 224, in forward x = self.conv1(x) # shape = [, width, grid, grid] File "/home/inspur/anaconda3/envs/MindDiffuser/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1128, in _call_impl result = forward_call(input, kwargs) File "/home/inspur/anaconda3/envs/MindDiffuser/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 447, in forward return self._conv_forward(input, self.weight, self.bias) File "/home/inspur/anaconda3/envs/MindDiffuser/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 443, in _conv_forward return F.conv2d(input, weight, bias, self.stride, RuntimeError: Input type (torch.cuda.DoubleTensor) and weight type (torch.cuda.FloatTensor) should be the same

What should I do to solve this question?

[ReedOnePeck]

RuntimeError: Input type (torch.cuda.DoubleTensor) and weight type (torch.cuda.FloatTensor) should be the same As you can see from this, the incoming data is in a different format that causes the error, and you should use pycharm's debug to check the points where the error occurred by hitting the breakpoint. Go to CSDN or Google to find a solution to the problem.

[zhangdanfeng888]

@ReedOnePeck Hello, sorry to trouble you again. When I run python Feature extractor/Structural_feature_selection.py, I get "cuda out of memory", could you share files from "index_save_path" in the code? Thanks a lot.

[ReedOnePeck]

Sorry for replying you so late. I have uploaded the required documents for participant 1.

[zhangdanfeng888]

Sorry for replying you so late. I have uploaded the required documents for participant 1.

Thanks for your reply. I want to run "python Feature decoding/Semantic_feature_decoding.py", but I don't know where is the file in the code. Could you tell me where are these files? Are they root in "Feature extractor"? By the way, I modify the "ROIs" to ["nsd_train_fmriavg_nsdgeneral_sub1"], am I right?

[ReedOnePeck]

1. （1）'trn_file_ex' and 'val_file_ex' refer to the folders where the preprocessed fMRI data are stored （2）The other four file names refer to the path where you save the semantic features. You should run the files in "Feature extractor" to save the features first. （3）You don't need to pay attention to recons_img_idx, it seems to be a mistake I made when organizing the code.

2. ROI refers to different brain region numbers, such as V1, V2, IT, etc. It depends on the format of your preprocessed fMRI data. For specific ROI entries, you can refer to line 21of MindDiffuser/Feature decoding /Semantic_feature_decoding.py