Bug: Extra features are not filtered out in prediction

deflect-ca / baskerville

Security Analytics Engine - Anomaly Detection in Web Traffic

Other

32 stars 5 forks source link

Bug: Extra features are not filtered out in prediction #49

Open mkaranasou opened 4 years ago

mkaranasou commented 4 years ago

Conditions:

Have a scikit model, e.g. the test model
Have payload_size_average in configuration under the extra features (which is not included in the model's features)
Run irawlog pipeline.

Result: Key error for payload_size_average when predicting.

Note: I haven't tested it with a pyspark model, but it shouldn't be a problem since the part of the code that throws key error is only used in the scikit-learn workflow.

mkaranasou commented 4 years ago

Shortened output:

  File "/home/spark/baskerville/src/baskerville/models/anomaly_model_sklearn.py", line 125, in predict_dict
    x = [extract_features_in_order(x, detector.features)]
  File "/home/spark/baskerville/src/baskerville/models/anomaly_model_sklearn.py", line 41, in extract_features_in_order
    return [feature_dict[feature] for feature in model_features]
  File "/home/spark/baskerville/src/baskerville/models/anomaly_model_sklearn.py", line 41, in <listcomp>
    return [feature_dict[feature] for feature in model_features]
KeyError: 'payload_size_average'