example for providing data from fgsm attack on cifar10 and how to set the output folder for layers.py

[jS5t3r]

How I would set the params for FGSM:

python generate_samples.py -m cifar10 --aa FGSM --gpu 0

and then for layers.py

(detection) user@cerberus16:~/adversarial-detection/expts$ python layers.py --model-type cifar10 --dm proposed  -b 32 -o "layers_test"
Files already downloaded and verified
Files already downloaded and verified
Calculating layer embeddings for the train data:

Number of labeled samples per class:
class 0, count = 5000, proportion = 0.1000
class 1, count = 5000, proportion = 0.1000
class 2, count = 5000, proportion = 0.1000
class 3, count = 5000, proportion = 0.1000
class 4, count = 5000, proportion = 0.1000
class 5, count = 5000, proportion = 0.1000
class 6, count = 5000, proportion = 0.1000
class 7, count = 5000, proportion = 0.1000
class 8, count = 5000, proportion = 0.1000
class 9, count = 5000, proportion = 0.1000

Number of predicted samples per class:
class 0, count = 5000, proportion = 0.1000
class 1, count = 5000, proportion = 0.1000
class 2, count = 5000, proportion = 0.1000
class 3, count = 5000, proportion = 0.1000
class 4, count = 5000, proportion = 0.1000
class 5, count = 5000, proportion = 0.1000
class 6, count = 5000, proportion = 0.1000
class 7, count = 5000, proportion = 0.1000
class 8, count = 5000, proportion = 0.1000
class 9, count = 5000, proportion = 0.1000

Calculating layer embeddings for the test data:

Number of labeled samples per class:
class 0, count = 1000, proportion = 0.1000
class 1, count = 1000, proportion = 0.1000
class 2, count = 1000, proportion = 0.1000
class 3, count = 1000, proportion = 0.1000
class 4, count = 1000, proportion = 0.1000
class 5, count = 1000, proportion = 0.1000
class 6, count = 1000, proportion = 0.1000
class 7, count = 1000, proportion = 0.1000
class 8, count = 1000, proportion = 0.1000
class 9, count = 1000, proportion = 0.1000

Number of predicted samples per class:
class 0, count = 1000, proportion = 0.1000
class 1, count = 1002, proportion = 0.1002
class 2, count = 1003, proportion = 0.1003
class 3, count = 989, proportion = 0.0989
class 4, count = 1016, proportion = 0.1016
class 5, count = 994, proportion = 0.0994
class 6, count = 1007, proportion = 0.1007
class 7, count = 980, proportion = 0.0980
class 8, count = 1014, proportion = 0.1014
class 9, count = 995, proportion = 0.0995

Test set accuracy = 0.9545

Number of layers = 8

Layer: 1
Original dimension = 3072. Train data size = 50000. Sub-sample size used for dimension reduction = 10000
Intrinsic dimensionality: 24

Searching for the best number of neighbors (k) and projected dimension.
INFO:helpers.knn_classifier:Using NPP for dimension reduction.
INFO:helpers.dimension_reduction_methods:Applying PCA as first-level dimension reduction step
INFO:helpers.dimension_reduction_methods:Number of nonzero singular values in the data matrix = 3072
INFO:helpers.dimension_reduction_methods:Number of principal components accounting for 99.5 percent of the data variance = 847
INFO:helpers.dimension_reduction_methods:Dimension of the PCA transformed data = 847
INFO:helpers.dimension_reduction_methods:Dimension of the projected subspace = 240
INFO:helpers.dimension_reduction_methods:Solving the generalized eigenvalue problem to find the optimal projection matrix.
INFO:helpers.knn_classifier:Performing cross-validation to search for the best combination of number of neighbors and projected data dimension:
Traceback (most recent call last):
  File "layers.py", line 308, in <module>
    main()
  File "layers.py", line 291, in main
    search_dimension_and_neighbors(embeddings, labels, indices_sample, model_file, output_file, n_jobs)
  File "layers.py", line 105, in search_dimension_and_neighbors
    n_jobs=n_jobs
  File "adversarial-detection/expts/helpers/knn_classifier.py", line 151, in knn_parameter_search
    labels_test_pred = knn_model.predict_multiple_k(data_proj[ind_te, :], k_range)
  File "adversarial-detection/expts/helpers/knn_classifier.py", line 414, in predict_multiple_k
    nn_indices, nn_distances = self.index_knn.query(X, k=k_list[-1])
  File "adversarial-detection/expts/helpers/knn_index.py", line 235, in query
    return self._query(data, self.index_knn[0], k)
  File "adversarial-detection/expts/helpers/knn_index.py", line 252, in _query
    nn_indices, nn_distances = index.query(data, k=k)
  File "/home/user/.conda/envs/detection/lib/python3.7/site-packages/pynndescent/pynndescent_.py", line 886, in query
    self._init_search_graph()
  File "/home/user/.conda/envs/detection/lib/python3.7/site-packages/pynndescent/pynndescent_.py", line 842, in _init_search_graph
    self._search_graph.transpose()
  File "/home/user/.conda/envs/detection/lib/python3.7/site-packages/scipy/sparse/lil.py", line 437, in transpose
    return self.tocsr(copy=copy).transpose(axes=axes, copy=False).tolil(copy=False)
  File "/home/user/.conda/envs/detection/lib/python3.7/site-packages/scipy/sparse/lil.py", line 462, in tocsr
    _csparsetools.lil_get_lengths(self.rows, indptr[1:])
  File "_csparsetools.pyx", line 109, in scipy.sparse._csparsetools.lil_get_lengths
ValueError: Buffer has wrong number of dimensions (expected 1, got 2)

[jayaram-r]

It looks like an error produced by the PyNNDescent library that is used for approximate nearest neighbor search. The error messages are a little weird. Can you tell me which version of PyNNDescent and Numba packages are you using? I can try to reproduce the error.

[jS5t3r]

pynndescent               0.3.3             py_0 
numba                     0.46.0           py37h962f231_0 

all: https://gist.github.com/jS5t3r/63f418656f82ed1e5a6179c8b6b0883b

conda env export > environment.yml: https://gist.github.com/jS5t3r/8ff34c52eea6753468035b0a8c025967 or conda list -e > environment.txt: https://gist.github.com/jS5t3r/b92c0e9998483151045eeed83929f2a9

you can use: conda create -n detection --file environment.yaml

[jS5t3r]

can you just tell me the exact Python Version 3.x?   This information is missing in your README.

[machengcheng2016]

@jS5t3r @jayaram-r I think it is Python 3.7.5. Please refer to this file

[jayaram-r]

Thanks @machengcheng2016 I hope that resolved the error @jS5t3r

[jayaram-r]

For future reference, using PyTorch 1.3.1 and other library versions given in the Readme file seems to solve the error. Works on a GTX 1080 TI, but not a TITAN V GPU.

To reduce the memory usage while extracting the layer embeddings, change the value of num_samples on this line: https://github.com/jayaram-r/adversarial-detection/blob/94fd0881a3eef179e66301629c9a5e348ce46bd1/expts/helpers/utils.py#L67