simsong
commented
3 years ago bulk_extractor does have an object-oriented interface. What it doesn't have is object-oriented linkage down to the scanners. However, it has a clearly defined object-based API, in which an object is provided as the sole argument to the scanner function. The scanner function is called with "C" linkage. And the scanner calls object interfaces on objects that are passed in to the function.
The reason that the linkage is "C" and not C++ is two-fold. First and most important, it allows scanners can be loaded dynamically on a variety of platforms. The goal was for bulk_extractor to be run as a drop-in program in a certified environment, but to allow end-users to write their own modules and have them load at run-time.
The second reason is that I tried to do run-time loading of C++ shared libraries and I couldn't get it to work in a way that was reliable.
A third reason is that the current "C" linkage is easier to make work with things like Lambda, easier to call between languages (I was able to call C functions from Python, but not C++ functions, due to name mangling; YMMV), and the like.
However, you can certainly write a class that supports the C linkage but the uses virtual functions to call each of its scanner parts. This is left as an exercise for the reader. I've written a few of these; one of them is used for the flex-based scanners, but they don't replace the actual linkage section.
While working on this C++17 rewrite, I realized that the a scanner linkage prototype could be written with the C++ template system. Then you would instantiate the linkage for each of your named scanners and subclass it. Again, I tried some prototypes, but then gave up on it.
I have a very limited amount of time to work on this project, so I'm putting my efforts into making sure that BE2 supports C++17 and thereby supports modern operating systems. It's been a lot of work. I've done a complete refactoring of the BE13_API system and it is much cleaner now. At some point in the future this might be worth doing.
jonstewart
The concept of a scanner was built to be simple—just define a single function!
But, that function must adhere to a certain protocol. It must check the phase and then take appropriate action. These phases and actions correspond to lifetime concerns—initialization, config check, doing work, destruction. This is a good indication that there's an object-oriented design waiting to be defined for scanners, where a new scanner would be a class that inherits from a scanner base class. Inheritance from the base class would likely use the CRTP to provide for a common factory/construction method.
A wrinkle of lifetime management in a multithreaded processing application is that the first scanner may be initialized for the purpose of one-time setup and/or configuration/command-line checking, but then a scanner object of each type may be created for each thread in the thread pool. Typically this secondary initialization can be done via a
clone()method that takes a "prototype" instance and then simply invokes the copy constructor. In this way a scanner instance can have state which is reused from sbuf to sbuf, generally to avoid reallocation but also perhaps useful when doing some kind of accumulate/reduce operation (like creating histograms)—each thread's instance can create its result of the partition of sbufs it receives and then those partial results can be aggregated at the end of processing, thread-safe and very efficient (presumably the operations would be associative and commutative).While complicated framework APIs aren't fun, bulk_extractor isn't some terrible Java framework with a bewildering number of classes. A very simple base class API could be defined for scanners. The resulting code for scanners would be then be cleaner, simpler, and more idiomatic.
It may be possible to define such an interface that's then used by the existing function-based API, to avoid an all-or-nothing refactoring.