search

s8sg / goflow

A Golang based high performance, scalable and distributed workflow framework
MIT License
1.15k stars 139 forks source link

[GoFlow-4] Allow GoFlow workload to be completely independent to the flow #4

Open s8sg opened 4 years ago

s8sg commented 4 years ago

Currently Flow and Workload are tightly coupled as workload are defined inside of a flow definition This has a few drawbacks

  1. You can't scale only a specific workload
  2. Changing a workload will touch the whole flow definition
  3. Workload can't be shared across other flows

To solve this issue GoFlow will support a special kind of Operation called ApplyExternal() like Apply() It will take a unique ID of the workload that need to be called 

Node('n2').ApplyExternal('task1')

To define a workload user need to use the GoFlow Library and Register a Workload using the unique ID

fs.Register('task1', PerformTask1)
fs.StartWorkload(['myflow']) // start workload will take the list of flows

Workload will have the same signature

func PerformTask1(data []byte, option map[string][]string) ([]byte, error) {
  ...
}
lyphilip commented 2 years ago

Hi, any updates on this feature?