Open spolti opened 1 month ago
What is your opinion on generating the SDK code based on the json schema?https://github.com/serverlessworkflow/specification/blob/main/schema/workflow.yaml
@ribeiromiranda +1 if we can, although in my experience, the workflow can be really complex and hard to model using generators. We have much OneOf
and AnyOf
that makes everything harder to model in Go.
I understand, generating code can get very complex.
Another very interesting way to make a json schema interpreter with libraries: https://cuelang.org or https://github.com/xeipuuv/gojsonschema.
Is it simpler than generating the code and robust to changes in the specification.
+1, are you willing to give it a try?
The DSL is still changing, so I would wait till is more stable. Probably next week will be a good moment to start working on that. Relying completely in code generation is going to be complex because the presence of oneOf/anyOf/allOf, so I foresee following approaches.
map[string]interface{}
as in here) representing the workflow on memory@fjtirado despite the stability of the DSL, it's worth starting to explore our possibilities first.
Regarding your points, unfortunately, Go doesn't play well with inheritance. Let's see what we can do given @ribeiromiranda suggestions. I agree that in the Java SDK option 3 can be a good alternative for the xOf
approach.
oints, unfortunately, Go doesn't play well with inheritance. Let's see what we can do given @ribeiromiranda suggestions. I agree that in the Java SDK option 3 can be a good alternative for the
xOf
approach.
Yes, for option 3. in go, the option is to wrap the generated Pojo into another one https://www.geeksforgeeks.org/inheritance-in-golang/ (since GO allows calling the methods of the generated pojo within the container one, it will do the trick)
I think these features below can be using jsonschema validation:
Development the validation (e.g. map jsonschema into a graph):
Generate code based in jsonschema
I don't know if it has other features or I might be simplistic? I think there is no need to map jsonschema into struct. If everyone agrees, I can try to develop some simple cases.
A very simple validation example:
package main
import (
_ "embed"
"fmt"
"log"
"os"
"github.com/xeipuuv/gojsonschema"
"sigs.k8s.io/yaml"
)
//go:embed workflow.yaml
var yamlWorkflow []byte
func main() {
jsonWorkflow, err := yaml.YAMLToJSON(yamlWorkflow)
if err != nil {
log.Fatal(err.Error())
}
yamlExample, err := os.ReadFile("./example.yaml")
if err != nil {
log.Fatal(err.Error())
}
jsonExample, err := yaml.YAMLToJSON(yamlExample)
if err != nil {
log.Fatal(err.Error())
}
schemaLoader := gojsonschema.NewBytesLoader(jsonWorkflow)
documentLoader := gojsonschema.NewBytesLoader(jsonExample)
result, err := gojsonschema.Validate(schemaLoader, documentLoader)
if err != nil {
log.Fatal(err.Error())
}
if result.Valid() {
fmt.Printf("The document is valid\n")
} else {
fmt.Printf("The document is not valid. see errors :\n")
for _, desc := range result.Errors() {
fmt.Printf("- %s\n", desc)
}
}
}
oints, unfortunately, Go doesn't play well with inheritance. Let's see what we can do given @ribeiromiranda suggestions. I agree that in the Java SDK option 3 can be a good alternative for the
xOf
approach.Yes, for option 3. in go, the option is to wrap the generated Pojo into another one https://www.geeksforgeeks.org/inheritance-in-golang/ (since GO allows calling the methods of the generated pojo within the container one, it will do the trick)
I know how to do inheritance in Go, but that's not the point. The point is this technique is an anti-pattern in general we avoid.
@ribeiromiranda yup, that's the overall features we will provide for the SDK. Generating diagrams can be a nice to have, but not a priority since we do not have this feature atm anyway.
@ribeiromiranda I think that the input to the validator should be just the workflow definition stream (an io.reader) being validated (the loading of the validating schema should be hidden to the developer)
just a reminder about it:
Just provide validation that certain definition file is valid according to the schema (using https://github.com/xeipuuv/gojsonschema) . And maybe a shortcut call to load the generic Json object (load the map[string]interface{} as in here) representing the workflow on memory
using generics with the SDK is a not good approach due the kubernetes integration. we had some problem related to it in the past.
just a reminder about it:
Just provide validation that certain definition file is valid according to the schema (using https://github.com/xeipuuv/gojsonschema) . And maybe a shortcut call to load the generic Json object (load the map[string]interface{} as in here) representing the workflow on memory
using generics with the SDK is a not good approach due the kubernetes integration. we had some problem related to it in the past.
The internal representation as a graph for the resources:
I don't know how Kubernetes integration works, the need to represent it with a struct
Representation example:
package graph
import (
"bytes"
"encoding/json"
"fmt"
"strings"
)
type Node struct {
value interface{}
edgeds map[string]*Node
}
func (n *Node) UnmarshalJSON(data []byte) error {
data = bytes.TrimSpace(data)
if data[0] == '{' {
dataMap := map[string]json.RawMessage{}
err := json.Unmarshal(data, &dataMap)
if err != nil {
return err
}
for key, val := range dataMap {
node := NewNode()
err := json.Unmarshal(val, &node)
if err != nil {
return err
}
n.edgeds[key] = node
}
} else if data[0] == '[' {
dataMap := []json.RawMessage{}
err := json.Unmarshal(data, &dataMap)
if err != nil {
return err
}
for i, val := range dataMap {
key := fmt.Sprintf("%d", i)
node := NewNode()
err := json.Unmarshal(val, &node)
if err != nil {
return err
}
n.edgeds[key] = node
}
} else {
return json.Unmarshal(data, &n.value)
}
return nil
}
func (n *Node) Lookup(path string) (*Node, error) {
pathSplit := strings.Split(path, ".")
edge := n
walked := []string{}
for _, key := range pathSplit {
walked = append(walked, key)
if val, ok := edge.edgeds[key]; ok {
edge = val
} else {
return nil, fmt.Errorf("path not found: %s", strings.Join(walked, "."))
}
}
return edge, nil
}
func NewNode() *Node {
return &Node{
edgeds: map[string]*Node{},
}
}
I don't see an issue with this representation, @ribeiromiranda @spolti. And I agree that generics is a no-go. Too much compatibility issues with k8s libraries.
I created the repository with a proposal to implement the new DSL, if I follow this implementation I will create a PR.
This repository implemented a basic "validator" using jsonschema and "builder", in the file "example/example.go" there are some use cases.
What would you like to be added:
The DSL has been updated to a new and concise version.
Why is this needed: The SDK needs to support the new DSL, as found here: https://github.com/serverlessworkflow/specification/blob/main/dsl-reference.md
For more info: https://github.com/serverlessworkflow/specification/issues/843