s8sg / goflow

A Golang based high performance, scalable and distributed workflow framework
MIT License
1.06k stars 129 forks source link

[Question] How to run workflow without middleware independency #76

Open OhBonsai opened 11 months ago

OhBonsai commented 11 months ago

It's good for triggering workflow by api. But sometimes i wanna run my workflow in command mode. I wanna keep my workflow definition.

package amin

import (
    "fmt"
    flow "github.com/s8sg/goflow/flow/v1"
)

// Workload function
func node1(data []byte, option map[string][]string) ([]byte, error) {
    result := fmt.Sprintf("(Executing node 1 with data (%s))", string(data))
    fmt.Println(result)
    return []byte(result), nil
}

// DefineWorkflow Define provide definition of the workflow
func DefineWorkflow(workflow *flow.Workflow, context *flow.Context) error {
    dag := workflow.Dag()
    dag.Node("node1", node1)
    return nil
}

then run this workflow in main

workflow.Run("my-workflow")
s8sg commented 11 months ago

We already support running in worker mode. It doesn’t run the http server. If thats what you are asking for

On Mon, 3 Jul 2023 at 20:18, Bonsai @.***> wrote:

It's good for triggering workflow by api. But sometimes i wanna run my workflow in command mode. I wanna keep my workflow definition.

package amin

import ( "fmt" flow "github.com/s8sg/goflow/flow/v1" )

// Workload function func node1(data []byte, option map[string][]string) ([]byte, error) { result := fmt.Sprintf("(Executing node 1 with data (%s))", string(data)) fmt.Println(result) return []byte(result), nil }

// DefineWorkflow Define provide definition of the workflow func DefineWorkflow(workflow flow.Workflow, context flow.Context) error { dag := workflow.Dag() dag.Node("node1", node1) return nil }

then run this workflow in main

workflow.Run("my-workflow")

— Reply to this email directly, view it on GitHub https://github.com/s8sg/goflow/issues/76, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABYIJ2R2LADGHOPST7DXTY3XOLBBXANCNFSM6AAAAAAZ4OLWZQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

OhBonsai commented 11 months ago

We already support running in worker mode. It doesn’t run the http server. If thats what you are asking for

In worker mode, we need to connect to Redis to store and process task states and results. However, I hope to keep only the definition of workflows and not need to connect to Redis. Because my workerload is running in different network environments and the worker nodes can only connect to the server via HTTP to get tasks.

s8sg commented 11 months ago

Can you explain more like why you require only the definition?

OhBonsai commented 10 months ago

image

implement a SQLite state store, maybe a good idea