odtp-org / odtp

ODTP: A tool designed to manage, run, and design digital twins.
https://odtp-org.github.io/odtp-manuals/
GNU Affero General Public License v3.0
10 stars 1 forks source link

DAG basic compatibility based on Barfi #64

Open caviri opened 2 months ago

caviri commented 2 months ago

Context Currently only linear pipelines are compatible with odtp. In order to allow the evaluation and execution of DAG we need a solution for this in workflow.py.

Solution Barfi is a simple solution that provides a node-based interface and a runtime. It's implementation shouldn't take too much effort and it will provide a basic proof for the prototype.

Problem Barfi GUI is based in a Vue widget for Streamlit. This makes difficult to integrate this GUI into our NiceGUI solution.

If Barfi GUI is deployed independently the parameters could be passed using the domain and then store the JSON back on the database. Quite dirty solution.

Acceptance criteria

caviri commented 1 month ago

Code removed from codebase:

from barfi import st_barfi, barfi_schemas, Block

# TODO: Extract barfi examples for 1 component run, 2 components run, Confluence 3 components, Divergence 3 components

class BarfiManager:
    """This class check all the components available in the components database, and prepare the Barfi class"""
    def __init__(self):
        self.blocks = []
        pass

    def addBlock(self, name, inputs, outputs, options, dockerfunc):
        b = Block(name="name")
        for option in options:
            # Different types of blocks contains different options parameters
            if option["type"] == "display":
                b.add_option(name=option["name"],
                             type=option["type"],
                             value=option["value"])
            elif option["type"] == "select":
                b.add_option(name=option["name"],
                            type=option["type"],
                            value=option["value"],
                            items=option["items"])
            elif option["type"] == "input":
                b.add_option(name=option["name"],
                             type=option["type"],
                             value=option["value"])
            elif option["type"] == "integer":
                b.add_option(name=option["name"],
                             type=option["type"],
                             value=option["value"])
            elif option["type"] == "checkbox":
                b.add_option(name=option["name"],
                             type=option["type"],
                             value=option["value"])
            elif option["type"] == "slider":
                b.add_option(name=option["name"],
                             type=option["type"],
                             value=option["value"],
                             min=option["min"],
                             max=option["max"])

        for input in inputs:
            b.add_input(name=input["name"])

        for output in outputs:
            b.add_output(name=output["name"])

        def barfiFunc(self):
            # Here we need to build the docker method that will be send to run
            envStringList = []
            for option in options:
                optionValue = self.get_option(name=option["name"])
                envStringList.append("{}={}".format(option['name'], optionValue))

            # Actually Inputs/Outputs will not be managed by Barfi
            for input in inputs:
                inputValue = self.get_interface(name=input["name"])
                # Need to be copied in a folder. How this works on Barfi?

            for output in outputs:
                self.set_interface(name=output["name"], 
                                   value=output["value"])

            # Run the Component
            # 1. Copy input files
            # 2. Run component

        b.add_compute(barfiFunc)

        self.blocks.append(b)