search

jMetal / jMetalSP

A framework for Big Data Optimization with multi-objective metaheuristics
MIT License
46 stars 26 forks source link
dynamic-optimization evolutionary-algorithms metaheuristic-framework multi-objective-optimization streaming-data-processing

readme

jMetalSP: A framework for Big Data Optimization with multi-objective metaheuristics

jMetalSP is a software platform for dynamic multi-objective Big Data Optimization. It combines the features of the jMetal multi-objective optimization framework with the Apache Spark cluster computing system. jMetal provides the optimization infrastructure for implementing both the Big Data optimization problems and the dynamic metaheuristic to solve them; Spark allows to manage the streaming data sources, allowing to effectively use the computing power of Hadoop clusters when processing large amounts of data.

Please, note that jMetalSP is a project in continuous development. If you have any question or suggestion of desirable features to be included, feel free to contact us. The current version is jMetalSP 2.0.

Current status

We are currently working on a redesign of the framework with the following ideas in mind:

Architecture

The architecture of the current development version of jMetalSP (Version 1.1) is depicted in the following UML class diagram:

jMetalSP architecture

A jMetalSPApplication is composed of:

The implementation of jMetalSP applies Java generics to ensure that all the componentes are compatible. The declaration of the jMetalSPApplication class and its main componentes is the following:

public class JMetalSPApplication<
        S extends Solution<?>,
        P extends DynamicProblem<S, ?>,
        A extends DynamicAlgorithm<?, ? extends ObservedData<?>>> {

  private List<StreamingDataSource<?>> streamingDataSourceList;
  private List<DataConsumer<?>> algorithmDataConsumerList;
  private StreamingRuntime streamingRuntime;

  private P problem;
  private A algorithm;
  ...
}

This way, by using generics the Java compiler can check that all the components fit together.

InDM2

InDM2 a new dynamic multi-objective optimization algorithm that allows the preferences of the decision maker (DM) to be incorporated into the search process. When solving a dynamic multi-objective optimization problem with InDM2, the DM can not only express her/his preferences by means of one or more reference points, which define the desired region of interest, but also those points can be also modified interactively.

Examples

The following example applications are included in the current development version:

How to execute the Spark example

In order to run the example DynamicContinuousApplicationWithSpark, it is necessary to run first the CounterProvider program whose goal is to store continuously in a directory files containing the data (i.e., the value of a counter) that will read by Spark in streaming.

Requirements

To run the examples that do not use Spark you need:

To execute the codes with Spark:

References