openphilanthropy / unrestricted-adversarial-examples

Contest Proposal and infrastructure for the Unrestricted Adversarial Examples Challenge
Apache License 2.0
327 stars 62 forks source link

Release dataset version 0.0.4 #60

Closed nottombrown closed 5 years ago

nottombrown commented 5 years ago

This PR does the following:

Researchers can upgrade to the new dataset by pulling the repo and running the following command

pip install -e bird-or-bicycle --upgrade

Bad bird datapoints that are now removed

-3b2f2608d3f04aa1
-79ac2f60e526e683
-1ab16de89590e4c5
-99636d7b4ea3e233
-93cd5a5b526d5e15
-ff721aa6906a239b
-7aba5c4ae8c57916
-d84af53461a42947
-85785cc1a59e8412
-3aa31e2c8960ec33
-7485d8abf0b9297b
-954c63e11aa1232e
-ef14662cd1c2d115
-f616e4fc0d3836b3
-2e8cd9f60546ac55
-001c9606f49c59c4
-598820468ca2957d
-d0a53873d9b5045c
-6b3797dec5bea3c7
-cae73b8b09db3e93
-e9e1abec98923fbe
-386aca1b6dbc7ab3
-a69a9641ac3c1fad
-397906f964c1ba37
-466724fc1a2f5ea8
-b0cf7e6ba0f7c941
-4fba9527aa917ef8
-2f493ee0e8482bd1
-17fbc19521a99b25
-8366a15ab29977f8
-41a87bc25d7eafd0
-defb4c991b77aa01
-21f2aff1121d13be
-e0726c3ba7829ff4
-4b347ef789b71f2f
-d1125249cd57e7a7
-8e385d43da95930c
-68775c9d8051dcce
-bcab0463d981755e
-8a084a78ead09bbb
-9ae97d6e4190b7b4
-9c59fb41857ecfda
-603c2f061ce7dbd3
-1b0fb5b533b24537
-1ba42f8e412b76b3
-72460159badf7b64

Bad bicycle datapoints that are now removed

-8ae043c7b7448219
-80a1a3e948880d93
-189467f4fea0a56a
-c1338874b4a98446
-842121bda37f25a2
-d6bcc9441f2e2317
-63390b707dbb7022
-91370c5768f8dcfa
-410b9c96ebc154ef
-a585f18d33cb9ab5
-1c7ee7e854fa4d19
-8a650416f08b3860
-2886fdcc63085b01
-9dc74e41a54c6688
-33f3dd75e994bc9b
-ed55ba03dfdd99d7
-5fd811193609c657
-ef5ad509b85abb72
-37e6a15259882305
-ede99f867f9f6f2b
-b879da8e1ade53a1
-315034ca9726da08
-717815eaf2ef2ca4
-394bd46368ac2e40
-5aea87bab366c4ad
-8e7b9ef0c246a18b
-055b2035b7557d4f
-802846b84af1e218
-7c9064473e6a6a78
-f44d753798ef7bf5
-cf74a4afc09463de
-626281361aa05d80
-3de4c2b947bcb15d
-e2c737ad4e6c742b
-05aea1cd2c7641b3
-cac51471e70ed382
-87241e2568cd46aa
-8c20158094acab48
-9dca3efa0d64899e
-977074739a92ea75
-aafaf3228318cc7b
-f451d2ea5f218cee
-3830b17dc5980dd8
-7a4f3c534956200f
-8354f24d0006eacf
-79b6814d3471cdd7
-d9084e9ba4e51173
-079d91186621a4d9
-96f73c14700f8944
carlini commented 5 years ago

How did you find these images?

nottombrown commented 5 years ago

@carlini - These bad images were due to the issue with the tasker pipeline. I filed this as an incident and will directly export data from the tasker database myself in the future.

We had been aware of the issue earlier, but I had failed to commit my changes that used the updated csv file.

nottombrown commented 5 years ago

I have manually verified that all the images in the following dataset folders appear unambiguous to me and are not truncated:

nottombrown commented 5 years ago

@carlini can I get a quick review?

carlini commented 5 years ago

Let me take a look

carlini commented 5 years ago

It all seems fine