search

andypitcher / IoT_Sentinel

IoT SENTINEL : Automated Device-Type Identification for Security Enforcement in IoT
https://arxiv.org/pdf/1611.04880.pdf
MIT License
41 stars 32 forks source link

classifier used in the IoT Sentinel paper #5

Open steseb opened 4 years ago

steseb commented 4 years ago

Hi, could you please share the parameters of your Random Forest classifier used in the paper? Or if possible the code of the classifier?

We are designing a different device-type identification mechanism. In the evaluation part, we're using your pcap files (many thanks again for sharing) as input data and a comparison would be nice.

Thanks,

andypitcher commented 4 years ago

Hi,

Sorry for the late reply, the project code is a bit everywhere, I will update the repo with the missing information. For now, you can have a look to this file https://github.com/Mozhdehm/IoT_Sentinel/blob/master/src/classification_IoTs.py (it's my colleague's repo that contains the rforest part)

Cheers,

steseb commented 3 years ago

Thanks Andy, we actually reimplemented your classifier to compare our performance. With the same dataset and a 5-fold validation we achieved an average F-score around 0.74. But we experienced a big difference in the execution time in case the random forest is not able to provide a unique match and the edit distance needs to be used.

We are currently using the normalized_damerau_levenshtein_distance from pyxDamerauLevenshtein but our results (on a server) are one order of magnitude slower than the ones reported in your paper. About tens of minutes per sample when a couple of sequences per device-type are considered. Could you please confirm me if that's the same library you used?

Thanks again,