mc-imperial / dredd

Framework for evaluating C/C++ compiler testing tools
Apache License 2.0
11 stars 3 forks source link

Incorrect Behavior in __dredd_enabled_mutation Due to Bit Manipulation on 32-bit Integer Literals #230

Closed JonathanFoo0523 closed 2 months ago

JonathanFoo0523 commented 3 months ago

Example

#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <threads.h>

/* ################################ Dredd Prelude ######################################### */
static thread_local int __dredd_some_mutation_enabled = 1;
static int __dredd_enabled_mutation(int local_mutation_id) {
  static thread_local int initialized = 0;
  static thread_local uint64_t enabled_bitset[520];
  if (!initialized) {
    int some_mutation_enabled = 0;
    const char* dredd_environment_variable = getenv("DREDD_ENABLED_MUTATION");
    if (dredd_environment_variable) {
      char* temp = malloc(strlen(dredd_environment_variable) + 1);
      strcpy(temp, dredd_environment_variable);
      char* token;
      token = strtok(temp, ",");
      while(token) {
        int value = atoi(token);

        int local_value = value - 0;
        if (local_value >= 0 && local_value < 33265) {
          enabled_bitset[local_value / 64] |= (1 << (local_value % 64));
          some_mutation_enabled = 1;
        }
        token = strtok(NULL, ",");
      }
      free(temp);
    }
    initialized = 1;
    __dredd_some_mutation_enabled = some_mutation_enabled;
  }

  return enabled_bitset[local_mutation_id / 64] & (1 << (local_mutation_id % 64));
}
/* ##################################################################################### */

int main() {
    putenv("DREDD_ENABLED_MUTATION=9007");
    if (__dredd_enabled_mutation(9007)) {
        printf("9007 is enabled\n");
    }
    if (__dredd_enabled_mutation(8975)) {
        printf("8975 is enabled\n");
    }

    return 0;
}

results in

9007 is enabled
8975 is enabled

Expected result

9007 is enabled
afd commented 3 months ago

This sound pretty serious - are you able to dig into what is going wrong in the prelude code?

Then, I wonder how we could write a regression test for this. Perhaps a program like:

int main() { printf("0\n"); printf("1\n"); printf("2\n"); ... }

with a large number of printfs. I think that dredd will only try statement deletion mutations, so that enabling mutant i should cause the print for i to be omitted. This could be used to test whether things are working correctly - what do you think?

JonathanFoo0523 commented 3 months ago

I guess the problem is related to int and int literals represented as 32-bits. Changing 1to 1ull and returning a uint64_t type instead of int seems to fix it.

I think the example program for the regression test will be mutated to:

int main() {
    if (__dredd(...)) { printf("0\n"); }
    if (__dredd(...)) { printf("1\n"); }
    if (__dredd(...)) { printf("2\n"); }
    ...
}

We will need to actually run the program to observe that one print statement is missing from stdout, right? Most of the regression tests just compare the original source file and the mutated source file, instead of compiling and running the mutated binary.

JonathanFoo0523 commented 3 months ago

Attempt to fix with #231

If the fix is acceptable, I will rewrite the expected dredd output for the single_file regression test.

JamesLee-Jones commented 3 months ago

Attempt to fix with #231

If the fix is acceptable, I will rewrite the expected dredd output for the single_file regression test.

Running .dev_shell.sh.template at the root of the repo followed by scripts/regenerate_single_file_expectations.sh should do this for you automatically.