generate a new trained model

[IPKeohane]

Ying mentioned the idea of training the model with a subset of the data and then evaluating it on a different subset, which makes a ton of sense. So far, we have been able to figure out almost every step of the process except how to train a new model. We have it working where we can run data (benign, adversarial examples, etc.) through our defenses and then evaluate the results of the model post-defense against the true values. So far we have been using the "model-mnist-cnn-clean.h5" model and were wondering where the script is that generates that so that we can put in a custom subset of the data as an input into it to make a new trained model file.

[MENG2010]

I am sorry, but which assignment you are talking about? I can upload the script, but none of the assignments requires that. I would suggest you write down a breakdown and plan (like Issue https://github.com/csce585-mlsystems/project-athena/issues/25) for your assignment, so I can give more specific comments.

[IPKeohane]

Task 2 option 2. The task is to evaluate how an ensemble of weak defenses performs as a defense. The rough plan we have is to 1) pool together a collection of weak defenses into an ensemble. 2) put several different sets of examples through that ensemble defense. This does this by applying each defense and then a model is run to try and classify the "number" that the input is. 3) With all of these different "guesses" for each weak defense, the collection of probabilities is then synthesized for each example and one collective overall probability can be assigned to each example.

My question is, there is a model (cnn i believe) that is being applied after each weak defense to try and identify the class. We didn't generate that model or pick what training data to give that model. That model already exists in the repository and is called somewhere in the athena.predict line of code. I was wondering about how to generate a new model, and just like wanted to know more about what model is being called during that athena.predict call.

[MENG2010]

I am not sure either I misunderstood your breakdown or you were not on the right track.

The strategy model can be of any type and uses any technique (CNN, SVM, linear regression, etc.). Given an input, the strategy model takes (1) the collection of predictions (probabilities or logits) from individual weak defenses or (2) outputs from the inner layer(s) of individual weak defenses as its input and produces a final predicted label. The input-shape of your strategy model varies and relies on values you collected from the weak defenses. So, the script for training a weak defense (model-mnist-cnn-clean.h5) does NOT work for this task (the input-shapes are different, you will not be able to train this model using such architecture). The training of the strategy model is an INDEPENDENT machine learning task, separated from the ATHENA project, you need to do some research and decide what model you want to use. This task is an opening task.

For simplicity (consider the time issue), I would suggest you use the raw predictions (either probabilities or logits) as the input of your strategy model. Please check the corresponding video and Jupiter notebook tutorials for detailed information.

Once you trained your strategy model, you can evaluate data in the following process (a high-level description in partial pseudo code):

# load the ensemble
pool, _ = load_pool(*parameters)
ensemble = Ensemble(classifiers=list(pool.keys()))
# load the learned strategy model
decider = load the strategy model

# -------------------------
# evaluate data
# -------------------------
# load data
data = np.load(path_to_the_data)
# load true labels
y_true = np.load(path_to_the_true_label)

# collect the raw predictions from individual weak defenses in the ensemble
raw_preds = ensemble.predict(data, raw=True)
# transpose to correct shape
transposed_preds = np.transpose(raw_preds, (1, 0 2))
# get the final outputs
# I presume that you implemented a predict() function to produce the final prediction
y_preds = decider.predict(transposed_preds) 

# compute the error rate
# refer to the eval_model.py, or tutorial for Task1.

You can also refer to Cody's breakdown for this assignment.

[IPKeohane]

Thanks! The strategy model part is what we don't have yet. We've generated the raw predictions, but are trying to figure out how to create the strategy model that takes in those raw predictions. My follow up questions is trying to figure out whether we are doing one of the following two things.

Are we simply looking at the probability output of the raw predictions to get an overall prediction? Like if for each input data the weak defenses all say the number is a "7" so we label that a "7" (basic example, just for thinking about it). This is what I get from looking at your examples and the notebook, like we are using an ensemble of weak predictions to make one strong prediction.

Reading the description from Cody's team they talk about training a new model in keras using the outputs of these weak defenses. Are we supposed to train a new model with the predictions from these existing weak defenses as the training data?

[MENG2010]

To finish this assignment, you need to implement the following sub-tasks by yourself:

1. Decide the training data (which decides the input-shape of your model).
2. Choose a model (the model type, architecture, hyperparameters, etc.) and a framework you want to implement your model on (e.g., PyTorch, TensorFlow, Keras, etc.).
3. Implement the script to create, train (including save), load, and evaluate your strategy model, and a predict function for your model. How to implement this relies on the model type and architecture, the framework you chose in step 2, as well as the format you save the model. So, I cannot implement this for you. For example, if you check the loading functions in utils/models.py, you can see that we implemented different loading functions for the lenet models (which were saved in h5 format) and svm models (which were saved in a pickle format). You can use load_lenet to load a h5 model and load_svm to load a pickle format model. Other than that, you need to implement the corresponding loading function by yourself.
4. Prepare the training and test data (please refer to the discussion in #25 ).
5. Train & save the strategy model.
6. Evaluate the Undefended Model (model-mnist-cnn-clean.h5), the Vanilla ATHENA variants (using AVEP and MV strategies respectively), ATHENA + your strategy model, and the PGD adversarial trained model, on some AEs we uploaded in the data folder.
7. Analyze the experiment results and work on the report.

[MENG2010]

You need to decide your own training data, raw probabilities and raw logits are two simple examples.

You can also use outputs from hidden layers of weak defenses, which require modification/refactor in models/keraswrapper.py (maybe also in models/athena.py). Refer to the study Forest Agostinelli, Michael R. Anderson, and Honglak Lee. Adaptive Multi-Column Deep Neural Networks with Application to Robust Image Denoising. NIPS 2018. for detailed information on this idea.

There are other options but you need to do some research on this.