ikelos
commented
2 weeks ago Yep, it's all from when we introduced the ModuleRequirement. Some plugins don't need a kernel/OS, so just ask for a LayerRequirement, and others do. The configs are just trees of values, which are meaningless if the tree doesn't match up with what the plugin asks for. I'm kinda slowly working on fixing that, but it would be nice to be able to generate configs for virtual machines found by vmscan and have them work with all the other plugins.
To do that, I've already introduced a small change recently where TranslationLayerRequirement/SymbolTableRequirement/ModuleRequirements now record what type of requirement they were when the config is recorded. That should help us reuse some down the line, but it's really fluke that these all work (because all plugins by convention alone, use the same name for their ModuleRequirement, and for their LayerRequirements). It's exceptionally brittle and people became aware of it and started using it before it got a lot of love/attention.
Generally a --config is only guaranteed good for the same version of the same plugin as was used to generate it. That's it, that's as much as you get. By fluke that happens to be enough to be reusable across a whole bunch of plugins, but that's literally fluke. There is no surefire way of ensure that an arbitrary plugin's requirements will line up with another plugin's requirements. The best we can do is try to make them fit, and if not run the whole process over again (which we do, so no one should find that things don't work, they may just take longer).
Sadly "fix it" isn't really an option. Happy to document more clearly what it does, but I'm concerned that if people didn't realize exactly how it worked themselves, it might have been taught as some kind of panacea, rather than what it is a mechanism for rebuilding the state of a singular run from a particular plugin.
- is kinda light on details... 5;P It's not really broken, it's doing exactly what it was supposed to, just not necessarily what people thought it did?
2a) Yep, probably explaining to people what it does might help them understand, but it may not placate them.
2b) I don't think this will be possible, and I don't think that's a problem either. The complaint is not that they get bad data it's that it doesn't speed anything up? If they want to manually tinker with their config to match up the requirements of the plugin, that's fine, asking volatility to do that automatically is tricky and is on the cards, but not in the "parity-release" time frame, I'm afraid.
Plugins only fail if they only provide a config generated from a run of a different plugin? Where did they get the idea that configs were transferable between plugins? The --help is woefully unspecific about what a configuration is and how it works, and I can see how people might assume that a configuration is global, so those should be quick to make more accurate but otherwise I don't think --config is really documented anywhere?
atcuno
eve-mem
@ikelos in the workshops, we show
--save-configand--configearly on when showing new Vol3 features so that people get the performance benefit when running many plugins to solve the labs/exercises.This then leads them to running plugins like mftscan, the base yarascan, etc. with --config and the plugin not working.
I know at some point we discussed why this happened, but I cannot find it anywhere (it might have been on a call). I think I remember it being something about plugins that don't access kernel memory and just scan a physical layer, so they don't build the full config but I could be off about that.
With this issue in mind, I think we should either:
1) Fix it to work with these affected plugins
Or if 1) is too difficult or undesirable, then:
2a) Document it in our read the docs, so that we can link to it in training materials, tickets, Slack, etc.
2b) If possible, produce a warning when these plugins are run and
--configis set. I know direct command line argument access is abstracted a bit from plugins, but if these plugins could check in terms of "did the user set --config" then the plugins could warn not to use --config with them and to use -f directly instead. This would make full automation of plugins a bit weird, but switching from --config to -f for users running plugins directly from the command line should be pretty easy.Thoughts? Other options? I think this will lead to documentation either way, but I don't have a strong opinion on fixing it outright or making people switch between --config and -f as long as the plugins can warn about it, instead of just failing in a weird way like now.