VirusTotal / yara-python

The Python interface for YARA
http://virustotal.github.io/yara/
Apache License 2.0
648 stars 179 forks source link

Unable to build extension with profiling support #155

Open sbruno opened 4 years ago

sbruno commented 4 years ago

I'm trying to build yara-python with profiling support.

The command python setup.py build --enable-profiling succeeds, but the resulting extension fails with "libyara compiled without profiling support" message when calling profiling_info() on a rules object.

yara-python.c seems to be outdated with respect to libyara in the profiling support.

First of all the preprocessor definition is PROFILING_ENABLED when it should be YR_PROFILING_ENABLED

However after changing that I got errors about clock_ticks not being member of YR_RULE and YR_STRING.

I saw in the code that clock_ticks was renamed as time_cost at some point, but later it was removed.

The only way I was able to compile the extension with the support for profiling was going back to version 3.11 in both yara-python and libyara, and also do some changes in the yara-python.c code (rename clock_ticks as time_cost and commenting out the addition of time_costs from YR_STRINGs since it seems to be member of YR_RULE only in this version).

So my modified yara-python.c at line 1619 looks like this:

#ifdef PROFILING_ENABLED
  PyObject* object;
  PyObject* result;

  YR_RULES* rules = ((Rules*) self)->rules;
  YR_RULE* rule;
  YR_STRING* string;

  char key[512];
  uint64_t time_cost;

  result = PyDict_New();

  yr_rules_foreach(rules, rule)
  {
    time_cost = rule->time_cost;

    /*yr_rule_strings_foreach(rule, string)
    {
      time_cost += string->time_cost;
    }*/

    snprintf(key, sizeof(key), "%s:%s", rule->ns->name, rule->identifier);

    object = PyLong_FromLongLong(time_cost);
    PyDict_SetItemString(result, key, object);
    Py_DECREF(object);
  }

  return result;
#else
  return PyErr_Format(YaraError, "libyara compiled without profiling support");
#endif

My first question is if I'm losing something because of not adding the YR_STRINGs time costs, since I don't know what they represented.

The second is if it would need too much rework in yara-python.c to support profiling with the latest libyara code. Looking at the code, it seems that profiling information is now stored in "scanners", and I don't see any mention of scanners in yara-python.c.