esbmc / esbmc-ai

Automated Code Repair suite powered by ESBMC and LLMs.
https://pypi.org/project/esbmc-ai/
GNU Affero General Public License v3.0
27 stars 4 forks source link

OCM: ESBMC-AI helper methods #87

Open Yiannis128 opened 1 year ago

Yiannis128 commented 1 year ago

There is an issue when comparing results in the equivalence script for certain functions. Functions that return: pointer, array, etc. There is no way to know how the results and equivalence should be checked. Moreover, there are no configurable options on how equivalence should be checked. This issue proposes the implementation of special functions that ESBMC-AI can detect that configure the verification process, these functions start with __ESBMCAI_ and are used by ESBMC-AI, so they aren't invoked by the rest of the source code.

The functions can be included in the source code for configuration reasons, but can be excluded from compilation by ignoring all functions beginning with __ESBMCAI_. The following functions are proposed:

int __ESBMCAI_EQ_X

Overrides the equality logic that ESBMC (the backend) will use. The function starts with __ESBMCAI_EQ_ indicating it is an ESBMC-AI special function for equality checking. The get_from_index names the function that this special function will act as the equality operation for. The parameters for the function should match the return data type of the function.

int __ESBMCAI_EQ_get_from_index(struct LinkedList *a, struct LinkedList *b)
{
    // Do comparison here and return bool result.
    return 0;
}

The function that __ESBMCAI_EQ_get_from_index will act as the equality operator for:

struct LinkedList *get_from_index(struct LinkedList *root, int index)

In the function equivalence script, if a valid function is detected, then the comparison will be changed from:

assert($function_assert_old == $function_assert_new);

To:

assert(__ESBMCAI_EQ_get_from_index($function_assert_old, $function_assert_new));

This allows for any type of function with any data type to be tested.

__ESBMCAI_CFG_X_

Allows for the configuration of properties of function X. Configurable properties:

Example:

// Configures ESBMC-AI __VERIFIER_nondet_int() > 10 for input parameter index in function get_from_index
int __ESBMCAI_CFG_MINVALUE_get_from_index_index() {
  return 10;
}

Z __ESBMCAI_SETVALUE_X_Y(Z Y)

When function X is being checked for partial equivalence, will call function __ESBMCAI_SETVALUE_X_Y for parameter Y. The parameter to the function will be Z Y where Z is the datatype of the parameter Y in question. The return value of __ESBMCAI_SETVALUE_X_Y will be used as a parameter.

Example, we have a function to be optimized:

struct LinkedList* get_from_index(struct LinkedList* root, int index);

We need to supply rules to that function because the index and the LinkedList provided need to coincide with each other. We can configure both values to relate like so:

Root parameter:

struct LinkedList* __ESBMCAI_SETVALUE_get_from_index_root(struct LinkedList* root) {
  // Logic for constructing a good LinkedList
  return root;
}

Index parameter:

int index __ESBMCAI_SETVALUE_get_from_index_index(int index) {
  // Logic for constructing a good index
  return index;
}

How can we ensure communication between each method? As they don't have means of communicating, this is where the built-in seed function will come into place.

int __ESBMCAI_SEED();

Returns a random seed that for each parameter of the function will be consistent. So same value when called inside __ESBMCAI_SETVALUE_get_from_index_root and __ESBMCAI_SETVALUE_get_from_index_index, this can be used to synchronize behavior.

Tasks

Yiannis128 commented 1 year ago

Please comment/propose any other properties that should be configured by OCM.

Yiannis128 commented 11 months ago

Need to review if Lua should be used as a scripting frontend for users to correctly configure ESBMC-AI OCM.