Open fatemehkalantari1993 opened 1 year ago
Thank you. Let me confirm two points.
Thank you. I have a question about these features that you used. did you extract image features with Caffe and brain features with Fasl2lir???
did you extract image features with Caffe and brain features with Fasl2lir???
Yes, the shared image and decoded features were extracted using Caffe and decoded using FastL2LiR. Furthermore, the weights of VGG-19 used in our PyTorch reconstruction code were actually imported from Caffe, making them compatible with the image features extracted using the Caffe model.
Thank you very much. I ran your code with your data with the generator and without the generator and the resulting figures were the same as the paper. But errors in with generator code were bigger than without the generator. I think this problem is due to calculate errors in the construction function.
for n02190790_15121.mat
with generator results: iteration = 193; err = 61842.234375 iteration = 194; err = 61800.95703125 iteration = 195; err = 61347.01171875 iteration = 196; err = 61385.69921875 iteration = 197; err = 61229.046875 iteration = 198; err = 61087.90625 iteration = 199; err = 60957.53515625 iteration = 200; err = 60826.23046875
without generator results and function: recon_image, loss_list = reconstruct(feat, encoder, generator=None, layer_mapping=layer_mapping, optimizer=optim.SGD, image_size=encoder_input_shape, crop_generator_output=True, preproc=image_preprocess, postproc=image_deprocess, output_dir=save_dir, save_snapshot=True, snapshot_dir=snapshots_dir, snapshot_interval=10, snapshot_ext='jpg', snapshot_postprocess=normalize_image, return_loss=True, **opts)
iteration = 195; err = 47284.10546875 iteration = 196; err = 47321.4609375 iteration = 197; err = 47359.74609375 iteration = 198; err = 47416.3671875 iteration = 199; err = 47473.265625 iteration = 200; err = 47544.5234375
Thank you very much. I ran your code with your data with the generator and without the generator and the resulting figures were the same as the paper. But errors in with generator code were bigger than without the generator. I think this problem is due to calculate errors in the construction function. I attached codes.
for n02190790_15121.mat
with generator results: iteration = 193; err = 61842.234375 iteration = 194; err = 61800.95703125 iteration = 195; err = 61347.01171875 iteration = 196; err = 61385.69921875 iteration = 197; err = 61229.046875 iteration = 198; err = 61087.90625 iteration = 199; err = 60957.53515625 iteration = 200; err = 60826.23046875
without generator results and function: recon_image, loss_list = reconstruct(feat, encoder, generator=None, layer_mapping=layer_mapping, optimizer=optim.SGD, image_size=encoder_input_shape, crop_generator_output=True, preproc=image_preprocess, postproc=image_deprocess, output_dir=save_dir, save_snapshot=True, snapshot_dir=snapshots_dir, snapshot_interval=10, snapshot_ext='jpg', snapshot_postprocess=normalize_image, return_loss=True, **opts)
iteration = 195; err = 47284.10546875 iteration = 196; err = 47321.4609375 iteration = 197; err = 47359.74609375 iteration = 198; err = 47416.3671875 iteration = 199; err = 47473.265625 iteration = 200; err = 47544.5234375
On Mon, 09 Mordad 1402 05:42 AM, Shuntaro Aoki @.***> wrote:
did you extract image features with Caffe and brain features with Fasl2lir??? Yes, the shared image and decoded features were extracted using Caffe and decoded using FastL2LiR. Furthermore, the weights of VGG-19 used in our PyTorch reconstruction code were actually imported from Caffe, making them compatible with the image features extracted using the Caffe model. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
thanks a lot for your help. as you know, I am working in the image reconstruction field.
first step: I extracted image features with MatConvNet that didn't work well in reconstruction. then, I extracted Caffe image features and then I extracted brain features with fastl2lir and finally, I did reconstruction and my results were very close to your result.
second step: I update my Ubuntu to the Ubuntu 22 version and I couldn't install Caffe on it. Therefore, I used Torch for image feature extraction. Then, I used Torch image features in Fasl2lir and extract brain features. recently. you uploaded new codes that did reconstruction with Torch. I ran this code with my torch features, but I received big errors. when I ran this code with your data, I received big errors, too. I report this problem to you and you modified your code and errors decreased with your features.
But, when I run this code with my features that extract in the same way (fastl2lir) I still get big errors. The only difference is that I extract image features with Torch. I am attached my code. Please look at them.
I need to use Torch because I want to use resnet and I don't have a strong system in Iran to train this network in Caffe and get resnet.caffemodel. please help me.
for example in this code I extract conv5_1 image features in Torch:
import numpy as np from google.colab.patches import cv2_imshow import pandas as pd import cv2 import os import torch from torchvision.datasets import ImageFolder from torchvision.transforms import ToTensor from torch import optim, nn from torchvision import models, transforms import torchvision.io as io import torchvision.transforms as transforms from PIL import Image import imghdr from bdpy.dataform import Features, load_array, save_array
Load the VGG19 model
vgg19 = models.vgg19(pretrained=True)
Access the features module
features = vgg19.features
Define a new model that outputs features from the conv3_1 layer
model = nn.Sequential(features[0:12])
Set the model to evaluation mode
vgg19.eval()
import numpy as np from google.colab.patches import cv2_imshow import pandas as pd import cv2 import os import torch from torchvision.datasets import ImageFolder from torchvision.transforms import ToTensor from torch import optim, nn from torchvision import models, transforms import torchvision.io as io import torchvision.transforms as transforms from PIL import Image import imghdr from bdpy.dataform import Features, load_array, save_array
Load the VGG19 model
vgg19 = models.vgg19(pretrained=True)
Access the features module
features = vgg19.features
Define a new model that outputs features from the conv3_1 layer
model = nn.Sequential(features[10])
conv3_1_features = nn.Sequential(*list(vgg19.features.children())[:29])
Set the model to evaluation mode
conv3_1_features.eval()
Load an image
data_dir='/content/drive/MyDrive/image_feature_python/resultes/pytorch_image_feat_training/pytorch/VGG19/conv5_1/' img_dir='/content/drive/MyDrive/matconvnet/data/training'
Get image files
imagefiles = [] for root, dirs, files in os.walk(img_dir): imagefiles = [os.path.join(root, f) for f in files if imghdr.what(os.path.join(root, f))]
imgs_path = np.array(imagefiles)
image name
images_name=[] images=imagefiles for f in files: name=f[:-5] images_name.append(name)
print ('Image num: %d' % len(imagefiles))
for n in range(imgs_path.shape[0]): print(n) img = io.read_image(imgs_path[n])