KeyError: '\x02'

[jibanes]

linux-x86_64 idapro 7.2

Traceback (most recent call last):
  File "/opt/ida/python/ida_idaapi.py", line 572, in IDAPython_ExecScript
    execfile(script, g)
  File "/xxx/pigaios/sourceimp_ida.py", line 661, in <module>
    main()
  File "/xxx/pigaios/sourceimp_ida.py", line 647, in main
    importer.import_src(database)
  File "/xxx/pigaios/sourceimp_ida.py", line 581, in import_src
    if self.find_initial_rows():
  File "/xxx/pigaios/sourceimp_core.py", line 541, in find_initial_rows
    score, reasons, ml, qr = self.compare_functions(match_id, bin_id, ATTRIBUTES_MATCHING)
  File "/xxx/pigaios/sourceimp_core.py", line 283, in compare_functions
    self.ml_model = self.ml_classifier.load_model()
  File "/xxx/pigaios/ml/pigaios_ml.py", line 223, in load_model
    return joblib.load(filename)
  File "/usr/lib/python2.7/dist-packages/joblib/numpy_pickle.py", line 577, in load
    obj = _unpickle(fobj, filename, mmap_mode)
  File "/usr/lib/python2.7/dist-packages/joblib/numpy_pickle.py", line 507, in _unpickle
    obj = unpickler.load()
  File "/usr/lib/python2.7/pickle.py", line 864, in load
    dispatch[key](self)
KeyError: '\x02'```

[joxeankoret]

Check this: https://github.com/joxeankoret/pigaios/issues/19#issuecomment-442018350

[jibanes]

thank you joxean

[jibanes]

Looks like my output is slightly different from comment #19 see below, it looks like it's missing BernoulliNB/GradientBoostingClassifier/RandomForestClassifier.

/xxx/pigaios/ml$ ./pigaios_ml.py -multi -t [Sat Mar 30 08:17:58 2019] Using the Pigaios Multi Classifier [Sat Mar 30 08:17:58 2019] Loading data... [Sat Mar 30 08:18:08 2019] Fitting data with CPigaiosMultiClassifier(None)... Fitting DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None, max_features=None, max_leaf_nodes=None, min_impurity_split=1e-07, min_samples_leaf=1, min_samples_split=2, min_weight_fraction_leaf=0.0, presort=False, random_state=None, splitter='best') [Sat Mar 30 08:18:26 2019] Predicting... [Sat Mar 30 08:18:57 2019] Correctly predicted 13983 out of 19075 (false negatives 5092 -> 26.694626%, false positives 441 -> 0.044100%) [Sat Mar 30 08:18:57 2019] Total right matches 1013542 -> 99.457057% [Sat Mar 30 08:18:57 2019] Saving model... /xxx/pigaios/ml$

[jibanes]

probably a difference in this:

ML_CLASSIFIERS = [ (tree.DecisionTreeClassifier, "Decision Tree Classifier", []), ]

Should I add the other classifiers in there?

[joxeankoret]

No. Back in the day I used to use various, then realized it wasn't worth it and now I'm just using a decision tree classifier. So, the output being different is normal.

[jibanes]

Thank you