Modelling stateful C-APIs with no struct/class equivalence.

[NikLeberg]

Hi there @hgarrereyn. Super exciting cutting-edge technology you have built here! Thank you very much for open-sourcing it.

I have a simple stateful C-API:

static int g_state;

void setState(int state) {
    g_state = state;
}

void run(void) {
    if (g_state == 123) {
        assert(0);
    }
}

If setState(123) and afterwards run() is invoked, the assert gets triggered.

I played around a bit and tried to come up with a schema.yaml for it:

struct_buggy_api:
  type: struct
  name: buggy_api
  c_headers: [buggy_api.h]
  static_methods:
  - void setState(int state);
  - void run();

Schema generation with gfuzz gen cpp works, but running the fuzzer afterwards produces following error:

[*] Loading: schema.json
[*] GraphFuzz: loading trees from cache...
[!] No initializer tree for type: "buggy_api" (type: 0)
[!] No finalizer tree for type: "buggy_api" (type: 0)
[*] After validation: total scopes: 4
[*] After validation: usable scopes: 3
[!] Schema is invalid. (Run with "--graphfuzz_ignore_invalid" to continue with usable scopes).

So if I understand correctly, it has no way of creating a "buggy_api" type. But the static functions don't really need one.

What would be a correct way of modelling this API? I would very much like your guidance.

[hgarrereyn]

Hey, thanks so much for trying out GraphFuzz!

Your use case is definitely right on the edge of what GraphFuzz is designed to do. The issue (as you figured out) is that these pure global functions effectively have no inputs and outputs and therefore GraphFuzz doesn't know how to link them together.

In the wild, most "global" C endpoints we encountered still required some context object; for example in sqlite3 api, you need to pass a sqlite3 * to the sqlite3_db_config endpoint. Those types of global endpoints are still bounded by the lifetime of the provided objects and so we don't see this problem.

In this case, we just need to define a fake "global context" object that GraphFuzz can use to understand how endpoints are allowed to be linked together. There are a few different ways to do this. I think the easiest way is to define a helper C++ header file with a struct that just proxies our APIs:

api_helper.h

extern "C" {
    #include "buggy_api.h"
}

struct GlobalContext {
public:
    GlobalContext() {}
    void _run(void) { run(); }
    void _setState(int state) { setState(state); }
};

Then we can define our schema.yaml like the following. Importantly, because libFuzzer is doing in-memory fuzzing and we have persistent global state, we need to reset the state each iteration so that our test cases are deterministic. GraphFuzz lets you define a few "hooks" which run at certain points, one of which is the initializer hook as you can see below: (I think this is currently missing from the documentation but I will add it soon)

schema.yaml

config:
  type: config
  # Invoked before every test case. Reset global state so each
  # test case is deterministic.
  initializer: |
    setState(0);

struct_GlobalContext:
  type: struct
  name: GlobalContext
  headers: [api_helper.h]
  methods:
  - GlobalContext()
  - ~GlobalContext()
  - void _run();
  - void _setState(int);

You can imagine all of the graphs we will generate like this will be linear. We create a GlobalContext at the beginning, call a bunch of methods, and then delete the GlobalContext. So effectively it accurately models how we expect to be able to use the global endpoints (invoke them in any order).

I was able to find the test crash with this harness:

#28090  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 597 rss: 478Mb L: 1920/3953 MS: 1 Custom-
#28327  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 602 rss: 478Mb L: 1896/3953 MS: 3 ShuffleBytes-Custom-CustomCrossOver-
#28477  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 593 rss: 478Mb L: 1054/3953 MS: 5 CustomCrossOver-CustomCrossOver-Custom-CustomCrossOver-CustomCrossOver-
#28803  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 600 rss: 478Mb L: 872/3953 MS: 1 CustomCrossOver-
#29372  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 599 rss: 478Mb L: 910/3953 MS: 5 CustomCrossOver-CrossOver-Custom-CustomCrossOver-CustomCrossOver-
#29383  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 599 rss: 478Mb L: 690/3953 MS: 1 CustomCrossOver-
#29422  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 600 rss: 478Mb L: 868/3953 MS: 6 CustomCrossOver-ChangeBit-Custom-ShuffleBytes-Custom-CustomCrossOver-
#29468  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 601 rss: 478Mb L: 2080/3953 MS: 1 CustomCrossOver-
#29561  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 603 rss: 478Mb L: 436/3953 MS: 4 CustomCrossOver-CopyPart-Custom-CustomCrossOver-
#30292  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 593 rss: 478Mb L: 266/3953 MS: 1 CustomCrossOver-
#30703  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 590 rss: 478Mb L: 3625/3953 MS: 1 Custom-
#30773  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 591 rss: 478Mb L: 1240/3953 MS: 6 CustomCrossOver-CrossOver-Custom-CustomCrossOver-CustomCrossOver-CustomCrossOver-
#30826  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 592 rss: 478Mb L: 542/3953 MS: 3 CustomCrossOver-CustomCrossOver-CustomCrossOver-
#31377  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 592 rss: 478Mb L: 2074/3953 MS: 1 CustomCrossOver-
#31757  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 599 rss: 478Mb L: 1890/3953 MS: 9 ChangeBinInt-Custom-ChangeBinInt-Custom-ChangeBit-Custom-ChangeBit-Custom-Custom-
#31794  REDUCE cov: 12 ft: 91 corp: 43/39Kb exec/s: 588 rss: 478Mb L: 536/3953 MS: 3 CopyPart-Custom-Custom-
fuzz_exec: ./buggy_api.h:12: void run(): Assertion `0' failed.
==29== ERROR: libFuzzer: deadly signal
    #0 0x523c09 in __sanitizer_print_stack_trace (/harness/in/fuzz_exec+0x523c09)
    #1 0x434106 in fuzzer::Fuzzer::CrashCallback() (/harness/in/fuzz_exec+0x434106)
    #2 0x43415f in fuzzer::Fuzzer::StaticCrashSignalCallback() (/harness/in/fuzz_exec+0x43415f)
    #3 0x7fa092c6997f  (/lib/x86_64-linux-gnu/libpthread.so.0+0x1297f)
    #4 0x7fa092280e86 in __libc_signal_restore_set /build/glibc-uZu3wS/glibc-2.27/signal/../sysdeps/unix/sysv/linux/nptl-signals.h:80
    #5 0x7fa092280e86 in gsignal /build/glibc-uZu3wS/glibc-2.27/signal/../sysdeps/unix/sysv/linux/raise.c:48
    #6 0x7fa0922827f0 in abort /build/glibc-uZu3wS/glibc-2.27/stdlib/abort.c:79
    #7 0x7fa0922723f9 in __assert_fail_base /build/glibc-uZu3wS/glibc-2.27/assert/assert.c:92
    #8 0x7fa092272471 in __assert_fail /build/glibc-uZu3wS/glibc-2.27/assert/assert.c:101
    #9 0x55498e in run (/harness/in/fuzz_exec+0x55498e)
    #10 0x5550f0 in GlobalContext::_run() (/harness/in/fuzz_exec+0x5550f0)
    #11 0x554d5c in shim_2 (/harness/in/fuzz_exec+0x554d5c)
    #12 0x558071 in LLVMFuzzerTestOneInput (/harness/in/fuzz_exec+0x558071)
    #13 0x434847 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/harness/in/fuzz_exec+0x434847)
    #14 0x43f0b4 in fuzzer::Fuzzer::MutateAndTestOne() (/harness/in/fuzz_exec+0x43f0b4)
    #15 0x44071f in fuzzer::Fuzzer::Loop(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, fuzzer::fuzzer_allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) (/harness/in/fuzz_exec+0x44071f)
    #16 0x42fadc in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/harness/in/fuzz_exec+0x42fadc)
    #17 0x4229a2 in main (/harness/in/fuzz_exec+0x4229a2)
    #18 0x7fa092263c86 in __libc_start_main /build/glibc-uZu3wS/glibc-2.27/csu/../csu/libc-start.c:310
    #19 0x422a19 in _start (/harness/in/fuzz_exec+0x422a19)

and this minimized down (gfuzz min) into the following test case:

#include "api_helper.h"

#define MAKE(t) static_cast<t *>(calloc(sizeof(t), 1))

int main() {
    GlobalContext *var_0;
    {
        var_0 = MAKE(GlobalContext);
        GlobalContext ref = GlobalContext();
        *var_0 = ref;
    }
    GlobalContext *var_1;
    {
        var_0->_setState(123);
        var_1 = var_0;
    }
    GlobalContext *var_2;
    {
        var_1->_run();
        var_2 = var_1;
    }
    {
        free(var_2);
    }
}

Hopefully this helps answer you question (and hopefully in the future we can do more of this type of global api wrapping automatically)! If you are interested in applying GraphFuzz to a larger API surface or mixing global endpoints and stateful endpoints, I'd be happy to try to help figure out the best way to do that.

[NikLeberg]

Hey, thank you very much for your in depth response.

I totally agree that most API have some sort of context that goes with it. I kinda figured that a helper is needed. Thank you for your solution, it worked flawlessly!

I really really love the flexibility of the schema with its exec. Because of it I was able to come up with following alternative schema.yaml for my contrived API:

struct_buggy_api:
  type: struct
  name: globalContext
  c_headers: [buggy_api.h, globalContext.h]
  methods:
  - globalContext():
      outputs: [globalContext]
      exec: |
        setState(0); // reset state
  - ~globalContext():
      inputs: [globalContext]
      exec: |
        // pass
  - void setState(int state):
      inputs: [globalContext]
      args: [int]
      outputs: [globalContext]
      exec: |
        setState($a0);
  - void run():
      inputs: [globalContext]
      outputs: [globalContext]
      exec: |
        run();

with an additional globalContext.h:

typedef struct {
    int dummy;
} globalContext;

Besides the need for exec, do you see any drawbacks with such a schema?

Since you mention undocumented parts, you probably already know, but following is not documented:

• type: config in schema
• graphfuzz_try() and graphfuzz_bail() functions
• libFuzzer extra arguments --graphfuzz-<opt>

[hgarrereyn]

Awesome, that's a totally valid approach! In fact, that sort of thing is what I was going to suggest originally but wasn't sure if it would be more confusing. But it seems like you've really embraced the exec model which is cool to see! I also like how you are enforcing state reset inside the globalContext constructor.

I don't see any efficiency differences w.r.t using a C++ GlobalContext struct or doing that manually with custom endpoint definitions like you have so I think it's really a stylistic preference. It's a bit hard for me to judge this early which format will be easiest to apply to existing projects (and comfortable enough for developers to actually maintain it). Certainly there's also room for some syntactic sugar automation here given how boilerplate the globalContext structure is.

And I'll definitely follow up on the documentation front, thanks for pointing out the missing parts!