search

openphilanthropy / unrestricted-adversarial-examples

Contest Proposal and infrastructure for the Unrestricted Adversarial Examples Challenge
Apache License 2.0
327 stars 62 forks source link

spsa_with_random_spatial attack not showing any progress #56

Closed sibyjackgrove closed 5 years ago

sibyjackgrove commented 5 years ago

I was able to get results from clean, spatial_grid (1 hr 18 mintue) and, common_corruptions (16 minutes). But the spsa_with_random_spatial seems to be stuck and not showing any progress in number of iterations even ater waiting for 15 minutes. Is this normal or could this be a bug? I am using a Google Colab instance with GPU acceleration.

image

nottombrown commented 5 years ago

Hmm did this end up making progress after you waited longer? If not then can you tell me what cleverhans version you are using?

pip freeze | grep cleverhans On Wed, Oct 17, 2018 at 12:51 PM Siby Jose Plathottam < notifications@github.com> wrote:

I was able to get results from clean, spatial_grid (1 hr 18 mintue) and, common_corruptions (16 minutes). But the spsa_with_random_spatial seems to be stuck and not showing any progress in number of iterations even ater waiting for 15 minutes. Is this normal or could this be a bug? I am using a Google Colab instance with GPU acceleration.

[image: image] https://user-images.githubusercontent.com/25213730/47112405-d720c880-d21b-11e8-8112-12da59471852.png

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/google/unrestricted-adversarial-examples/issues/56, or mute the thread https://github.com/notifications/unsubscribe-auth/AASt33GopS5VlmEsp0qmP5aWcV2mFHWZks5ul4pWgaJpZM4XlGQs .

sibyjackgrove commented 5 years ago

Not it didn't make progress. In fact, it made the whole notebook hang and I had to restart the runtime. It seems I am using clerverhans 2.1.0.

image

nottombrown commented 5 years ago

Investigating now. It could be that the colab kernel ran out of memory and the runtime had to restart?

I think that running the full evaluation on colab is unlikely to work, because it takes ~12 hours on a single P100 GPU. This seems like it's probably not colab's use-case.

I would recommend looking at the deep learning images for google cloud because that's what we test on currently.