search

Trusted-AI / adversarial-robustness-toolbox

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
https://adversarial-robustness-toolbox.readthedocs.io/en/latest/
MIT License
4.74k stars 1.15k forks source link

Update (path to) imagenet labels in notebook attack_defense_imagenet #109

Closed jinglin80 closed 5 years ago

jinglin80 commented 5 years ago

I try to run the notebook example: detector-for-CIFAR10. However, CIFAR10 cannot be loaded. Following is the code and the error: (x_train, y_train), (x_test, ytest), min, max_ = load_dataset('cifar10')

class_descr = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']

UnpicklingError Traceback (most recent call last)

in ----> 1 (x_train, y_train), (x_test, y_test), min_, max_ = load_dataset('cifar10') 2 3 class_descr = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck'] ~/anaconda3/lib/python3.6/site-packages/art/utils.py in load_dataset(name) 566 return load_mnist() 567 if "cifar10" in name: --> 568 return load_cifar10() 569 if "stl10" in name: 570 return load_stl() ~/anaconda3/lib/python3.6/site-packages/art/utils.py in load_cifar10(raw) 403 for i in range(1, 6): 404 fpath = os.path.join(path, 'data_batch_' + str(i)) --> 405 data, labels = load_batch(fpath) 406 x_train[(i - 1) * 10000: i * 10000, :, :, :] = data 407 y_train[(i - 1) * 10000: i * 10000] = labels ~/anaconda3/lib/python3.6/site-packages/art/utils.py in load_batch(fpath) 380 content = cPickle.load(file_) 381 else: --> 382 content = cPickle.load(file_, encoding='bytes') 383 content_decoded = {} 384 for key, value in content.items(): UnpicklingError: pickle data was truncated
beat-buesser commented 5 years ago

Hi @jinglin80 Thank you for raising this issue.

Please try running the notebook one the branch dev.

This issue is the same as #63. We have already fixed it on branch dev with #98. We will soon update branch master.

jinglin80 commented 5 years ago

I run the notebook on the branch dev. However, I got the same error message. The CIFAR-10 dataset cannot be loaded properly.

UnpicklingError Traceback (most recent call last)

in ----> 1 (x_train, y_train), (x_test, y_test), min_, max_ = load_dataset('cifar10') 2 3 num_samples_train = 100 4 num_samples_test = 100 5 x_train = x_train[0:num_samples_train] ~/anaconda3/lib/python3.6/site-packages/art/utils.py in load_dataset(name) 566 return load_mnist() 567 if "cifar10" in name: --> 568 return load_cifar10() 569 if "stl10" in name: 570 return load_stl() ~/anaconda3/lib/python3.6/site-packages/art/utils.py in load_cifar10(raw) 403 for i in range(1, 6): 404 fpath = os.path.join(path, 'data_batch_' + str(i)) --> 405 data, labels = load_batch(fpath) 406 x_train[(i - 1) * 10000: i * 10000, :, :, :] = data 407 y_train[(i - 1) * 10000: i * 10000] = labels ~/anaconda3/lib/python3.6/site-packages/art/utils.py in load_batch(fpath) 380 content = cPickle.load(file_) 381 else: --> 382 content = cPickle.load(file_, encoding='bytes') 383 content_decoded = {} 384 for key, value in content.items(): UnpicklingError: pickle data was truncated
beat-buesser commented 5 years ago

I'm not able to reproduce your error message. The notebook runs fine on branch dev.

Please try cleaning your ART data directory ~/.art/data/. Maybe your previous download got corrupted.

jinglin80 commented 5 years ago

Could it because I do not have torch == 0.4.0 as required in test_requirements.txt?

beat-buesser commented 5 years ago

I think the torch version should not be a problem here.

Which version of Python are you using? Could you please show the output of ls -lh ~/.art/data/?

jinglin80 commented 5 years ago

The problem solved!!! Thank you.

However, when I run the notebook attack_defense_imagenet, I got following error:

[image: image.png]

On Wed, Jul 10, 2019 at 5:04 AM Beat Buesser notifications@github.com wrote:

I think the torch version should not be a problem here.

Which version of Python are you using? Could you please show the output of ls -lh ~/.art/data/?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/IBM/adversarial-robustness-toolbox/issues/109?email_source=notifications&email_token=AJL4ETY77GGIQQLW4HASNTLP6WQZHA5CNFSM4H65RED2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZS2IOA#issuecomment-509977656, or mute the thread https://github.com/notifications/unsubscribe-auth/AJL4ET2L4FK56TSK33UJJ3LP6WQZHANCNFSM4H65REDQ .

beat-buesser commented 5 years ago

How did you solve it?

beat-buesser commented 5 years ago

I cannot see the image in you previous message. Could you please upload the image or its message again?

jinglin80 commented 5 years ago

I delete the old version, reinstall dev version, and clear ~.art/data.

On Wed, Jul 10, 2019 at 11:45 AM Beat Buesser notifications@github.com wrote:

How did you solve it?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/IBM/adversarial-robustness-toolbox/issues/109?email_source=notifications&email_token=AJL4ET6HLCDLQAYX4MBOSADP6X7YJA5CNFSM4H65RED2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZT4LSI#issuecomment-510117321, or mute the thread https://github.com/notifications/unsubscribe-auth/AJL4ETZLR7PNF76RUGBVWCLP6X7YJANCNFSM4H65REDQ .

jinglin80 commented 5 years ago

Here you goes.

The code (from attack_defense_imagenet notebook) I run: from os.path import join, abspath, expanduser

Get Imagenet labels

path = expanduser('~/git/nemesis') with open(join(path, "imagenet/labels.txt"), "r") as f_input: class_names = eval(f_input.read())

Get some data

image_file = join(path,'test_api/clean100/n04479046_2998224x224.jpg') image = image.load_img(image_file, target_size=(224, 224)) img = image.img_toarray(image) plt.imshow(img / 255) img = img[None, ...]

Predict for clean image

pred = classifier.predict(img) label = np.argmax(pred, axis=1)[0] confidence = pred[:,label][0] print('Prediction:', class_names[label], '- confidence {0:.2f}'.format(confidence))

The error message I got:

FileNotFoundError Traceback (most recent call last)

in 3 # Get Imagenet labels 4 path = expanduser('~/git/nemesis') ----> 5 with open(join(path, "imagenet/labels.txt"), "r") as f_input: 6 class_names = eval(f_input.read()) 7 FileNotFoundError: [Errno 2] No such file or directory: '/Users/jinglin/git/nemesis/imagenet/labels.txt'
beat-buesser commented 5 years ago

Thank you! This is indeed an error. It looks like the file with the imagenet class names got lost.

I'll rename this issue accordingly and try to fix it soon.

beat-buesser commented 5 years ago

@jinglin80 I have fixed the notebook on branch development_issue_109. If you have time, please let me know if it works for you.