Setup of maip

[Tim-Cunnane]

Hi, I am currently in the process of setting up the maip platform on the VM, however I am having a couple of issues with this. I have the UI started but if I go into any section of the system it will say error{} in the top right corner of the screen. Would you be able to give me a hand with this?

Thanks

[strongcourage]

Hello Tim,

Did you install it by running the script or using the Dockerfile? On which system did you perform the installation? I have only tested it on Ubuntu 20.04.6 LTS - Focal. If possible, can you share a screenshot of the error?

Best, Manh

[Tim-Cunnane]

Hi Manh,

I have ran the start-client.sh scrip and I am able to get on to the UI. I am running it on a Ubuntu 20.04.6 LTS-focal. I have attached image below of the error when trying to get into the all arrears of the system like build, models, predict, etc.

Thanks for any help Tim

[strongcourage]

Hi Tim,

It seems that you have not started the server yet. Please make sure you run the script start-maip.sh first on one terminal and then the script start-client.sh on another terminal.

Best, Manh

[Tim-Cunnane]

Hi,

ahhhh perfect, that is sorted now appreciate it

Appreciate the help with this.

Thanks, Tim

[Tim-Cunnane]

Hi,

Apologies now whenever I build a new model it doesn’t appear within the all model section.

Also when i change from activity classification to anomaly detection and then I change to build a model it will automatically revert back to activity classification.

Best, Tim

[strongcourage]

Sorry, changing the app on client side is not handled properly now. I will fix this issue soon to make the state “app” more persistent on page navigation. If you want to change the AI application, please update this line with values ad or ac (by default). Also keep in mind that the repo is under development :D.

[Tim-Cunnane]

Hi,

Just want to check are we able to build model on the client side becuase when I build a model, it say that it is successfully built but in the All models section there is none within the section

Thanks

[strongcourage]

You are trying to build models for which applications? For activity classification ('ac'), it normally takes several seconds to build a model. For 'ad', it takes a bit longer, several minutes to complete, especially if you upload pcap files. You can also check the log on the terminal where you are running the server to see if there are any problems. Perhaps, you can start with the pcap files first.

[Tim-Cunnane]

trying to bulid models for 'ad', I built a model about 15 minutes ago with the existing pcaps provided and the system indicated that the model was successfully built, however there is no models within the All models section of the system which is a bit strange as seen in the below

[strongcourage]

I guess you may encounter some issues in parsing the upload pcap files. I just checked and can build models on my local machine. Thus, I recently added the mmt's reports of 2 pcap files, so that you can select them without uploading pcap files (commit 993b4cb).

On the server, you should see the log like this:

Finally, after 1 or 2 minutes, you should see the newly built model in the table of all models.

At least, running the latest version, you should see the model model_2 there (for performance testing purpose). Hope this helps.

[Tim-Cunnane]

Hi,

Hope you had a good Christmas

I am still have issue with the building of the models and they are not appearing in the All models section. Once I build the model the logs will appear like the below:

It has the POST requests however there is no GET requests. Would you now why this would be happening? Any help would be appreciated

Thanks, Tim

[strongcourage]

Hi Tim,

First, you need to check whether there are any issues in the model building process in the log training_f3044...log. If everything is ok, you will see the new model located at deep-learning/models or you can run the GET request /models.

Hope this helps.

[Tim-Cunnane]

Hi,

get this module not found error, how would this be installed?

[strongcourage]

Hello, I see, please follow this script to install necessary python libraries: https://github.com/Montimage/maip/blob/main/install.sh#L13C1-L13C55. Also pay attention to the version of Python 3. If you installed it on Ubuntu 20.04, it should be fine.

[Tim-Cunnane]

Hi,

I am receiving this after all of the modules have been installed and I have tried to build a model, just wondering if you have seen this issue before.

[edmdo]

AVX needs to be supported by the CPU. Try this: https://stackoverflow.com/questions/58198180/run-tensorflow-2-0-on-cpu-without-avx

[Tim-Cunnane]

Would it be possible if you could create a model with the latest reports for normal traffic and bot traffic in ad and change the training parameters to be different than the model that is pushed already?

Let me know if this is possible.

[Tim-Cunnane]

Just checking if the above is possible?

Thanks Tim

[strongcourage]

Hi Tim,

Yes, we can build new models by fine-tuning the parameters, such as the number of epochs and the size of batches for CNN and SAE. This should be done using the POST request /build or on "Build Model" tab.

[Tim-Cunnane]

Hi,

perfect, currently having issues with the avx instructions as it is saying that it isn’t supported on my devices, so trying to get something sorted for as I am currently a final year uni student looking into this software.

Would you know the params that are used for the model 2 that is already in the repo in anomaly detection section

[strongcourage]

I think that you can generally ignore this warning concerning avx instructions, and TensorFlow should still run as expected on your system, please check.

The model_2 is built as shown in the video demo with following parameters: number of epochs for CNN 5, number of epochs for SAE 2, size of batches for CNN 16, size of batches for SAE 32. The video showed that using 2 epochs for the SAE resulted in better performance compared to using 3 epochs for the tested training pcaps.

[Tim-Cunnane]

Get this within the logs of the system whenever I build a model and after checking the all model there is no model that are appearing. Just wanting to check what way do you find is best to start the platform, to allow for me to be able to build a model.

I ran the server with the using the installation instruction on the website

[Tim-Cunnane]

Would you have a break down of the exact AI python libraries that are used with in the system? and how they are used?

[strongcourage]

Sorry, it seems that you need to fix this issue in order to run tensorflow. According to this, what you can try is to uninstall tensorflow and then install tensorflow-cpu version 2.11.0.

All python libraries I've used are here https://github.com/Montimage/maip/blob/main/src/server/deep-learning/requirements.txt: lime and shap for XAI, the rest for extracting features, building and running AI models.

[Tim-Cunnane]

That is no problem I will have to look into it in the mean time, Would it be possible to upload a model that has the 3 epochs for SAE and different batch size than the one that is upload just for testing purposes?

[strongcourage]

Sure, I uploaded this model here.

[Tim-Cunnane]

Thank you for all of your help appreciate it

[Tim-Cunnane]

Hi,

There currently is an issue with the model 1 that was uploaded as there is no building status within the training folder how would this be fixed or could this be fixed.

[Tim-Cunnane]

Hi,

Would you have the pcap that you have used for the predict section that I could use for testing purposes, these include the youtube pcap, web cairn.pcap and the botnet honeypot.pcap?

[strongcourage]

Hi Tim,

Sorry, I forgot adding building files of model_1, please check the commit c50315f. Also, please find the pcaps here for testing. You can also use normal pcaps or malicious pcaps by searching for "botnet pcap files" on Internet.

Best.

[Tim-Cunnane]

Hi,

No bother, appreciate your help

Thanks,

[Tim-Cunnane]

Hi,

Sorry for hassling so much, I am having issue with the predict section when a model is picked and I add a pcap file then click predict, the system seem to be unresponsive would there be any reason for this?

[Tim-Cunnane]

Hi,

There is also issues with the SHAP and LIME where I am getting a 304 response error with the model. Have you seen this before?

Thanks

[strongcourage]

Hi Tim,

As you don't use the GUI, for prediction, you need to run 2 APIs in sequence as follows: 1) POST /mmt/offline to produce analysis report of existing pcap files in /maip/pcaps/, for example with the body request:

{
  "fileName": "youtube7.pcap"
}

If ok, you will find the reports in csv in the folder /maip/src/server/mmt/outputs/. In case you want to upload a new pcap file, please use the POST /pcaps request.

2) POST /predict using the following body request for youtube7.pcap:

{
  "predictConfig": {
    "modelId": "model_1",
    "inputTraffic": {
      "type": "report",
      "value": {
        "reportId": "report-youtube7.pcap-624b4fe1-422c-4b81-bc4a-2d3a47919b8a",
        "reportFileName": "1695745314.129769_0_youtube7.pcap.csv"
      }
    }
  }
}

I plan to improve the POST /predict to support "type": "pcap" so that the first POST request is executed automatically.

I added the reports of 3 testing pcap files in the latest commit so that you don't need to run the first step for those pcap files.

Regarding the issues of XAI, can you provide the logs? Thanks.

[Tim-Cunnane]

Hi,

How would you start the GUI to allow for me to use the predict if that would be easier?

This is what I am seeing on the logs for the XAI

[strongcourage]

Producing SHAP explanations can take a significant amount of time, even hours/days, depending on the size of the dataset or the number of samples being explained. I'd recommend running LIME first, as it is fast. You see several GET /api/xai requests, which are intended to continuously track the status of the XAI process. You can wait for the XAI process to complete and then disregard those GET requests.

To initiate the GUI, start the server first and then the client, following the steps you've already executed. Hope that helps.

[Tim-Cunnane]

Hi,

That is no bother at all, will have to run it and leave it, just want to be sure, is this likely to be the same for the attacks as it is getting the same responses in the logs

Apologies I miscommunication i am on the GUI trying to run the task but they just have the loading wheel

[Tim-Cunnane]

Hi,

Hope you are keeping well,

Just looking to see if there is any information in the pcap files that have been uploaded to the repo in terms of:

What traffic is used within the pcaps? For the bot pcap what network attack utilised such as DDOS or Denial of Service? and for the honeypot pcap where is this obtained?

Hopefully you will be able to help me.

Thanks Tim

[strongcourage]

Hello Tim,

The pcap files are extracted from this public dataset. The botnet attacks may consist of remote shell, file upload/download, capturing screenshots and key logging, etc. The example honeypot pcap can be obtained from multiple sources, like our honeypots or public datasets. Hope this help.

Best.

[Tim-Cunnane]

Hi,

Hope you are keeping well.

I am currently in the process of trying to create models for analyzing the best model parameters for my dissertation to show the accuracy numbers and F1 score for all of the models , apart from the 2 already made within the repo. I was wondering if it would be possible for some more model to be created and added to the repo.

Please see below the model that i would need created. models.txt

I appreciate all of the help and hope that this is possible to be completed.

Thanks Tim

[Tim-Cunnane]

Hi,

Just want to check if the above is possible?

Thanks

[strongcourage]

Hi Tim,

I will try to build those models this week.

Best,

[Tim-Cunnane]

Hi,

thank you very much, appreciate you help.

Kind regards

[strongcourage]

Hello Tim,

I recently built different models you've requested (https://github.com/Montimage/maip/commit/413de42aa85ab4b7fa5657ae7d3ee60b2d19fd3c). Please share with us if you find any interesting results during your testing phase, e.g., which configurations perform best. Thank you.

Best.

[Tim-Cunnane]

Hi,

Thanks for this, I am also currently trying to run a gan drvien attack on model 2 to test the robustness and get the performance score. how long would this usually take as this has been running quite a while for me so far?

Also, at the start of the attack there was a few errors that came up on GUI but then went away would this be a problem?

Thanks

[strongcourage]

Hello Tim,

Thank you for reporting the bugs. I fixed it in the latest commit, so that you can run all 3 attacks against model_2. Actually I added the dataset generated using CTGAN dataset for model_2. I will later update the API to automatically produce those GAN datasets with different configurations.

Since you're currently playing with our tool, if it's ok for you, I'll soon share our evaluation google form with you via email. Thus, you can provide us with your feedback to help us improve the tool. Thanks.

Best,

[Tim-Cunnane]

Hi,

Just out of curiosity, how would would this usually take to complete the CTGAN attack becuase i have been running it for a couple of hours now? I am trying to get the resilience scores from before and after the attack on model 2.

Thanks

[strongcourage]

Hello,

It's matter of seconds, around 10s. I would suggest you check the server logs to see if there are any errors. My logs are as follows:

[SERVER] MAIP Server started on: http://0.0.0.0:31057
GET /api/models 200 56.496 ms - -
GET /api/models 304 32.456 ms - -
/home/strongcourage/maip/src/server/deep-learning/trainings/model_2/datasets/Train_samples_view.csv
GET /api/models/model_2/datasets/train/view 200 280.014 ms - -
OPTIONS /api/attacks/poisoning/ctgan 204 0.257 ms - 0
Command to be coppied
python3 /home/strongcourage/maip/src/server/deep-learning//attacks.py model_2 ctgan 20
POST /api/attacks/poisoning/ctgan 200 22.589 ms - 94
GET /api/attacks 200 0.835 ms - 112
GET /api/attacks 304 0.493 ms - -
Process has completed:  python3 [
  '/home/strongcourage/maip/src/server/deep-learning//attacks.py',
  'model_2',
  'ctgan',
  20,
  ''
]
The log can be found at:  /home/strongcourage/maip/src/server/logs/attacks_ctgan_model_2_20.log
Going to callback on closed
Finish performing poisoning attack using CTGAN
GET /api/attacks 200 0.282 ms - 113
/home/strongcourage/maip/src/server/deep-learning/attacks/model_2/ctgan_poisoned_dataset.csv
GET /api/attacks/poisoning/ctgan/model_2/view 200 196.298 ms - -
GET /api/models 304 3.205 ms - -
OPTIONS /api/models/model_2 204 0.144 ms - 0
OPTIONS /api/models/model_2/predictions 204 0.117 ms - 0
OPTIONS /api/models/model_2/build-config 204 0.320 ms - 0
GET /api/attacks/model_2/datasets 200 6.501 ms - 117
GET /api/models/model_2/predictions 200 5.896 ms - -
GET /api/models/model_2/build-config 200 6.210 ms - 544
1695736470832
GET /api/models/model_2 200 22.546 ms - -
OPTIONS /api/retrain/model_2 204 0.232 ms - 0
model_2
{
  trainingDataset: 'ctgan_poisoned_dataset.csv',
  testingDataset: 'Test_samples.csv',
  training_parameters: {
    nb_epoch_cnn: 5,
    nb_epoch_sae: 2,
    batch_size_cnn: 16,
    batch_size_sae: 32
  }
}
{
  modelId: 'model_2',
  trainingDataset: 'ctgan_poisoned_dataset.csv',
  testingDataset: 'Test_samples.csv',
  training_parameters: {
    nb_epoch_cnn: 5,
    nb_epoch_sae: 2,
    batch_size_cnn: 16,
    batch_size_sae: 32
  }
}
{
  modelId: 'model_2',
  trainingDataset: 'ctgan_poisoned_dataset.csv',
  testingDataset: 'Test_samples.csv',
  training_parameters: {
    nb_epoch_cnn: 5,
    nb_epoch_sae: 2,
    batch_size_cnn: 16,
    batch_size_sae: 32
  }
}
/home/strongcourage/maip/src/server/deep-learning/trainings/310bc008-a58d-4811-8f1d-fa30f2dfd414/
/home/strongcourage/maip/src/server/deep-learning/trainings/model_2/datasets/Test_samples.csv
/home/strongcourage/maip/src/server/deep-learning/attacks/model_2/ctgan_poisoned_dataset.csv
/home/strongcourage/maip/src/server/deep-learning/trainings/model_2/datasets/Test_samples.csv
Command to be coppied
python3 /home/strongcourage/maip/src/server/deep-learning//retrain.py /home/strongcourage/maip/src/server/deep-learning/attacks/model_2/ctgan_poisoned_dataset.csv /home/strongcourage/maip/src/server/deep-learning/trainings/model_2/datasets/Test_samples.csv /home/strongcourage/maip/src/server/deep-learning/trainings/310bc008-a58d-4811-8f1d-fa30f2dfd414/results 5 2 16 32
{
  retrainConfig: '/home/strongcourage/maip/src/server/deep-learning/trainings/310bc008-a58d-4811-8f1d-fa30f2dfd414/retrain-config.json',
  retrainId: '310bc008-a58d-4811-8f1d-fa30f2dfd414'
}
POST /api/retrain/model_2 200 29.039 ms - 187
GET /api/retrain 200 0.664 ms - 333
GET /api/retrain 304 0.405 ms - -
GET /api/retrain 304 0.451 ms - -
GET /api/retrain 304 0.424 ms - -
GET /api/retrain 304 0.396 ms - -
GET /api/retrain 304 0.381 ms - -
GET /api/retrain 304 0.335 ms - -
GET /api/retrain 304 0.333 ms - -
GET /api/retrain 304 0.295 ms - -
GET /api/retrain 304 0.291 ms - -
GET /api/retrain 304 0.577 ms - -
GET /api/retrain 304 0.459 ms - -
Process has completed:  python3 [
  '/home/strongcourage/maip/src/server/deep-learning//retrain.py',
  '/home/strongcourage/maip/src/server/deep-learning/attacks/model_2/ctgan_poisoned_dataset.csv',
  '/home/strongcourage/maip/src/server/deep-learning/trainings/model_2/datasets/Test_samples.csv',
  '/home/strongcourage/maip/src/server/deep-learning/trainings/310bc008-a58d-4811-8f1d-fa30f2dfd414/results',
  5,
  2,
  16,
  32
]
The log can be found at:  /home/strongcourage/maip/src/server/logs/retraining_310bc008-a58d-4811-8f1d-fa30f2dfd414.log
Going to callback on closed
Finish retraining the model
GET /api/retrain 200 0.266 ms - 334
OPTIONS /api/models/310bc008-a58d-4811-8f1d-fa30f2dfd414/predictions 204 0.175 ms - 0
GET /api/models/310bc008-a58d-4811-8f1d-fa30f2dfd414/predictions 200 0.939 ms - -

[Tim-Cunnane]

Hi, That is strange this is what I am getting

would you know why?

Thanks Tim

[Tim-Cunnane]

Apologise it is now this,