[Proposal] Contributing scratch re-implementation of the C++ SDK

[jasondellaluce]

Motivation

The current SDK C++ is faulty and not usable for a variety of reasons, both in its current mantainance state and its design as a whole. As a primer, the SDK is not up to date with the latest plugin API version 3.0.0, thus producing plugins that are not loadable by the latest Falco. Second, the design relies on virtual inheritance interfaces, which bring the following concerns (just to name a few):

• Plugin capabilities are composable and implemented implicitly, which can only implemented by going through the diamond OOP problem, which is not handy nor optimal
• The base interfaces rely on virtual function methods to define "traits" to be implemented, which is inherently a slow option. Virtual function are implemented by compilers through vtables, and invoking a virtual function can be ~10x-20x slower than an inlined function call depending on the situation. Now that plugins support the syscall event stream, this overhead caused by the SDK itself is not acceptable.
• The current design hides some internal components that make use of dynamic casting at init time to check that the user-defined plugin type adheres to each of the capabilities interfaces. This is costly and error prone.

Proposal

I worked on re-implementing the SDK from scratch with a whole new design based on mixins, implicit interfaces based on type traits, RAII, and intensive function inlining. Although I hate to re-propose a design for the third time, I think these changes would be worth the effort. Plus, the current SDK is non usable, had no traction in the community, and is still a sandbox-level project in our organization, meaning that we consider it still in its early stages of development.

The progress of the big refactor can be checked on a branch of my fork: https://github.com/jasondellaluce/plugin-sdk-cpp/tree/refactor/performance-design-2 I invite anyone interested to take a look, as that will heavily help on understanding the technical approach.

Apprach

Goals

1. Plugins to be defined as simple structs/classes respecting type traits, one for each capability
2. Plugin capabilities can be implemented one-by-one in composition
3. All the machinery and the C type definitions of the falcosecurity/libs plugin API definitions should be hidden, everything should be covered by a zero-cost C++ abstraction (issues #12, #13)
4. When a type trait is not respected, the compiler is meant to provide guidance on the issue and the expected function signatures
5. The cost of the SDK should be close to zero, meaning that the end binary should almost be equivalent as implementing each plugin extern "C" symbol manually

Goal 1,2

Defining a plugin will look like:

#include <falcosecurity/sdk.h>

struct my_plugin
{
    virtual ~my_plugin() = default;
    std::string get_name() { return "cpptest"; }
    std::string get_version() { return "0.1.0"; }
    std::string get_description() { return "some description"; }
    std::string get_contact() { return "some contact"; }
    ...
    bool init(falcosecurity::init_input& i) { return true; }
    ...
};

FALCOSECURITY_PLUGIN(my_plugin);
FALCOSECURITY_PLUGIN_FIELD_EXTRACTION(my_plugin);
... // (other capabilities)

As visible, no virtual function or override marker is present. The SDK makes the heavy lift of figuring out if the type complies with the expected traits or not.

Goal 3

Users should not deal with the complexity of the C API and the event definitions of falcosecurity/libs. For example, here's how an event can be produced for the event sourcing capability:

falcosecurity::result_code next_event(falcosecurity::event_writer& evt)
{
    falcosecurity::events::pluginevent_e_encoder enc;
    enc.set_data((void*)"hello world", strlen("hello world") + 1);
    enc.encode(evt);
    return falcosecurity::result_code::SS_PLUGIN_SUCCESS;
}

Goal 4

If a type trait is not respected, and a function is either missing or has the wrong signature, the compiler is capable of detecting it through the static assertions of the SDK and will provide errors such as:

struct my_plugin
{
    ...
    const char* get_name() { return "cpptest"; }
    ...
}

# g++ -std=c++0x -fPIC -I../../include/ -shared -o libtest.so test.cpp
In file included from ../../include/falcosecurity/internal/plugin_mixin.h:21,
                 from ../../include/falcosecurity/internal/symbols_async.h:20,
                 from ../../include/falcosecurity/sdk.h:20,
                 from test.cpp:1:
...
test.cpp:250:1:   required from here
../../include/falcosecurity/internal/plugin_mixin_common.h:71:66: error: static assertion failed: expected signature: std::string get_name()
   71 |                                    decltype(&Plugin::get_name)>::value,
      |                                                                  ^~~~~
make: *** [Makefile:30: libtest.so] Error 1

Note how the SDK prompts an error and suggests the right/missing function signature : error: static assertion failed: expected signature: std::string get_name()

Goal 5

The new design makes use of widespread inlining wherever possible, and some hacks involing templates and macros to avoid any performance penalty.

For example, we want the following two examples to produce almost the same assembly output, to prove that the SDK adds no extra overhead:

Without the SDK

...
extern "C" uint32_t plugin_get_id() { return 999; };
...

With the SDK

struct my_plugin
{
    ...
    uint32_t get_id() { return 999; };
    ...
}
...

Compilation output with the SDK (x86_64)

0000000000005733 <plugin_get_id>:
    5733:   f3 0f 1e fa             endbr64 
    5737:   55                      push   %rbp
    5738:   48 89 e5                mov    %rsp,%rbp
    573b:   48 8d 05 7e 0b 04 00    lea    0x40b7e(%rip),%rax        # 462c0 <_ZN13falcosecurity9_internalL17s_plugin_sourcingE>
    5742:   48 8d 15 77 0b 04 00    lea    0x40b77(%rip),%rdx        # 462c0 <_ZN13falcosecurity9_internalL17s_plugin_sourcingE>
    5749:   48 89 55 e8             mov    %rdx,-0x18(%rbp)
    574d:   48 89 45 f0             mov    %rax,-0x10(%rbp)
    5751:   48 8b 45 f0             mov    -0x10(%rbp),%rax
    5755:   48 89 45 f8             mov    %rax,-0x8(%rbp)
    5759:   b8 e7 03 00 00          mov    $0x3e7,%eax
    575e:   90                      nop
    575f:   90                      nop
    5760:   5d                      pop    %rbp
    5761:   c3                      ret  

As visible, no virtual function call is involved and the function is as minimal as it gets, thus confirming that the compiler is capable of squashing most of the SDK abstractions. Of course, with more complex function symbols extra SDK work will be required, but this gives the idea.

[jasondellaluce]

cc @falcosecurity/core-maintainers, @falcosecurity/plugin-sdk-cpp-maintainers

What's everyone's feeling?

[incertum]

Big +1 for example if we want to follow through with approaches like https://github.com/falcosecurity/falco/pull/2655 and expose more sophisticated functionality over the plugins framework we urgently need this. Exposing syscalls hot path to plugins at least for me just marks the beginning.

[FedeDP]

+1 from me! Also, big thank you for doing this @jasondellaluce , our Plugins Master!

[leogr]

big +1 from me! :heart_eyes: