Create basic evolutionary algorithm

[lisv]

Following the meeting on the 30th of October

Steps:

1. Connect to JSON-RPC API and being able to send request (individuals in the population) -> Currently possible for Ripple
2. Represent genes -> Now done in this way: https://ibb.co/r7z0xVT
3. Generate population -> stuck :( cannot find complete specification of rippled methods
4. Next population (crossover and mutate)
5. Evaluate population (ranking) -> for first version will be done by looking at HTTP response codes only (and encouraging diversity), the contents of the response is however also important, e.g. 200 requests with error messages could potentially be very relevant (look at later)
6. Loop (new populations) until convergence/stopping criteria in Main method

[lisv]

Links to some (incomplete?) specification for Ripple JSON-RPC API:

• https://xrpl.org/public-rippled-methods.html
• https://xrpl.org/admin-rippled-methods.html (but should not be available unless 'you connect to rippled on a host and port that the rippled.cfg file identifies as admin')
• https://xrpl.org/rippleapi-reference.html (not sure why but this list is not the same as the methods in the aforementioned links)

The layout however makes it not easy to automatically get an overview of all methods and corresponding params. How do I get such an overview to use as a basis for the population generation?

EvoMaster (https://github.com/EMResearch/EvoMaster/blob/master/docs/blackbox.md) performs black-box testing by using the schema of APIs, which are available for various REST APIs in the OpenAPI directory (swagger format). Unfortunately, I could not find something like this yet for RPC APIs.

There is something similar for RPC (OpenRPC: https://spec.open-rpc.org/) but this format is not yet used (not by Ripple and probably not by many others either as it seems quite new).

[apanichella]

I would suggest starting with something simple, like the hill climber or (mu, lambda)-Evolutionary Algorithm. These algorithms use only the mutation operator and no crossover. The crossover could be a bit complicated in this case.

[lisv]

Following the meeting on the 12th of November

• Create a schema (potentially based on OpenRPC format) for the Ripple methods and arguments
• Generate initial population based on this schema
• Fitness function based on diversity in both input (regarding parameter values) and output (look at least at HTTP codes)
• Only use mutation operator in EA to create next generation
• Create runnable tool, for now use time (e.g. 3 hours) as stopping criterion

[lisv]

Update

• I have used the OpenRPC format to create a specification for the public rippled methods (https://xrpl.org/public-rippled-methods.html). Since there are so many, I have now only included account methods, so not all methods are included in the specification yet.

• Most elements (that I need) of the OpenRPC are required by the rules of the format and are in a prespecified format. Unfortunately, this does not hold for the 'schema' element of a parameter (which can be structured in various ways with different elements to represent the type of a parameter). This will likely cause problems later for other OpenRPC's. For now I have built code that can only deal with all the different ways that 'schema' is formatted in the Ripple OpenRPC.

• An initial population is created based on the OpenRPC specification: each individual has a certain method with corresponding parameters. The required parameters are always included in the individual, optional parameters can be included. Because the population is generated based on the specification, all created requests are valid according to the API. It is currently not possible to create invalid requests (e.g. a method with parameters of another method).

• Structure for fitness function is created, but currently fitness of an individual is a randomly generated value. I think I will have to do more research in order to find an appropriate way to calculate the fitness, e.g. certain ways to measure the coverage based on different outputs (since it is black-box).

• BooleanGene is mutated to either the other boolean value, or to a different type (some parameters can be of multiple types, e.g. a String of an Integer/Long).

• StringGene and LongGene cannot yet be mutated. I am not sure yet how I could best do this. And should I deviate from the specified (valid) ranges?

• Stopping criterion for the EA is now a number of generations, but this can be easily changed to time in the future.

[lisv]

Update

• The algorithm is now a (mu, lambda)-Evolutionary Algorithm and works based on an OpenRPC specification for any JSON-RPC API
• The Boolean, String and Long Gene can all be mutated
• Different fitness functions will be created and evaluated