code not running

[maithal]

Hi,

I saw in the closed issue this same problem was addressed however the model accuracy solution ddn't work. The code is hanging at preds[idx], _ = utils.get_preds(model, images[idx], 'imagenet', batch_size=args.batch_size)

I am using the ReNet50 model that was used by the original code. The batch size was reduced to 16 since the CUDA error was occurring. Can you tell me how to rectify this issue?

[cg563]

I have never encountered this issue before. Can you check which line of utils.get_preds the code is hanging at?

[maithal]

The code is not getting hung in the getpreds function but in the while loop. the code outputs preds[idx], but the while condition while preds.ne(labels).sum() > 0 is never met. Is there any specific reason for this? I also tried using fewer test images but nothing changed.

[cg563]

This is likely due to the model not correctly classifying the input images. Can you check the model's accuracy on the ImageNet test set is as expected? Please note that the code expects images to be unnormalized -- https://github.com/cg563/simple-blackbox-attack/blob/master/run_simba.py#L40 takes care of that.

[maithal]

I appreciate your quick response!

I am using the pretrained Inception_v3 model from torchvision.models. I used the same image as input to the same model and obtained some probability output but when I try using the same image as input to run_simba code it gets stuck in an infinite while loop. Also, the image I am inputting is unnormalized.

Can you please provide additional information on how your data_root looks like? Did you create a folder name '1' inside val folder where the input image is stored?

[cg563]

The data_root directory follows how ImageNet is typically setup, as in https://csinva.io/blog/misc/imagenet_quickstart/readme. Each subdirectory of val should correspond to images for a different class. Maybe this is the reason for the infinite loop?

If you forward your images through the model, does it give correct predictions?

[maithal]

I tried checking the model accuracy for the given image and it turns out the accuracy is good and all the predictions were correct. However, I am working on data_root directory to check if that is causing the issue.

[maithal]

Hello,

I was finally able to get to the core reason why this issue is occurring. It looks like for the given dataset is PyTorch pretrained ResNet50 & inception_v3 model is used, then preds.ne(labels) never sum to 0. But my main concern is why isn't the model converging for Inceptionv3 & ResNet50?

Thanks!

[cg563]

I'm a bit confused -- are you using pre-trained ResNet-50 or Inception-v3?

https://github.com/cg563/simple-blackbox-attack/blob/master/run_simba.py#L186 is just trying to sample a batch of samples until all of them are correctly classified. If the model is accurate enough, this should not be a problem. The only reason why that loop wouldn't terminate is if the pre-trained models have a very low accuracy on the dataset that you're using, which could be due to a variety of reasons, e.g., improper pre-processing, wrong class labels.

[maithal]

Yes, I am using PyTorch models ResNet-50 or Inception-v3 pretrained model. However, it's running for ResNet18 and I appreciate your prompt response.