CERT-Polska / karton

Distributed malware processing framework based on Python, Redis and S3.
https://karton-core.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
381 stars 45 forks source link

Tags failing to update on existing file #231

Closed aBUDmdBQ closed 12 months ago

aBUDmdBQ commented 12 months ago

We're having trouble updating tags within this Karton producer/consumer:

from karton.core import Karton, Task, Resource

class DemoMalwareIdentifier(Karton):
    """
    Identifies the custom Demo Malware family.
    """
    identity = "karton.i_demo-malware"
    filters = [
        {
            "type": "sample",
            "stage": "recognized",
        }
    ]

    def process(self, task: Task) -> None:
        sample = task.get_resource("sample")
        if task.payload["sample"].sha256 == '<redacted>':
            tags: list = ["stage:1", "family:demo-malware"]
            tag_task = Task(
                headers={"type": "sample", "stage": "identified", "family": "demo-malware"},
                payload={"sample": sample, "tags": tags},
            )
            self.send_task(tag_task)

if __name__ == "__main__":
    DemoMalwareIdentifier().loop()

The code is based off of many other Karton modules within your Repo, but the tags fail to appear in the MWDB GUI.

We have some modules running that successfully set tags on newly created database objects. We even have some more complex modules updating tags for existing objects (basically using the same code as shown above), but this simple one just won't work.

We are currently running on mwdb-core v2.10.2 and karton-core v5.2.0 (but also tried karton-core v5.3.0).

Any ideas would be much appreciated at this point.

psrok1 commented 12 months ago

I see that you are using "stage": "identified". Service that is actually adding tags to MWDB is karton-mwdb-reporter. As you can see, the only accepted stage types are recognized, analyzed and unrecognized (if report_unrecognized option is set to true).

You can add your own filters to karton-mwdb-reporter by deriving from MWDBReporter and making your own customized reporter that supports additional identified stage: https://karton-core.readthedocs.io/en/latest/service_configuration.html#customizing-ready-made-karton-services

Just make sure that you don't send stage:recognized from Karton that accepts stage:recognized as it will cause a loop.

Let me know if my answer is helpful for you.

aBUDmdBQ commented 12 months ago

Got it! This was exactly the info we were missing. Thank you very much!