FLPATH-597 - Move2Kube SWF

gabriel-farache commented 8 months ago

This PR add new SWF for Move2Kube.

Doc: https://docs.google.com/document/d/1Byy96FnE2_dyI0ZtT_KilRe7gVvWTyECMkbJ9x8-F9A/edit#heading=h.ds8q4xtkmu64

We will also need to setup proper CI/CD to test that the M2K workflow is working with real instances of Move2Kube and Github

gabriel-farache commented 8 months ago

cc @eloycoto @rgolangh @masayag

pkliczewski commented 8 months ago

@gabriel-farache do we want to continue work on this PR?

gabriel-farache commented 8 months ago

@gabriel-farache do we want to continue work on this PR?

@pkliczewski Yes, this can't be released until a new version of sonataflow is released

pkliczewski commented 7 months ago

@gabriel-farache now that we have prerelease available is this PR still blocked?

gabriel-farache commented 7 months ago

@gabriel-farache now that we have prerelease available is this PR still blocked?

@pkliczewski where do you see that?

pkliczewski commented 7 months ago

@gabriel-farache I tagged you in a slack thread

gabriel-farache commented 7 months ago

@gabriel-farache I tagged you in a slack thread

@pkliczewski so you are talking about https://redhat-internal.slack.com/archives/C059YMMB664/p1700145699121649?thread_ts=1700055474.502659&cid=C059YMMB664 which points to version 1.44.0.Final-redhat-00005 which is not there (https://repository.jboss.org/nexus/content/groups/public/org/kie/kogito/kogito-bom/) I could add this repo in the pom.xml to use this one but I am not sure I want to do that, it's just moving the problem for later and I'll still have to retest everything

pkliczewski commented 7 months ago

@gabriel-farache according to sonata team it should be there. It seems like it was not pushed to the registry. Let's revisit this when it is there.

gabriel-farache commented 7 months ago

@gabriel-farache according to sonata team it should be there. It seems like it was not pushed to the registry. Let's revisit this when it is there.

@pkliczewski or not... juggling with repo like that does not seems to be a good idea IMO, if we are to deliver something to customer, it shall be on the stable channel, not some back channel with unstable versions

pkliczewski commented 7 months ago

@gabriel-farache we can't wait on stable version because we won't be able to deliver anything. Once sonata team will fix the issue with missing RC version we can make progress and verify that it is working when it is released.

gabriel-farache commented 7 months ago

@pkliczewski What is the point of doing that? I know it's working with what on the main branch, and what/when do we plan to deliver to customer? IMO this is not the place for such discussion, it should be on the slack or agreed during a meeting so everyone is aware :)

pkliczewski commented 6 months ago

@masayag @rgolangh @RichardW98 @gciavarrini please review

dmartinol commented 6 months ago

overall, LGTM, but in order to move it to the final examples repo, we need to review these aspects IMO (the same probably applies to the escalation-eda application):

SWF should be reviewed using the flat layout instead of the quarkus one, this would remove a lot of overhead
SWF deployment must be consistent with the other examples in https://github.com/parodos-dev/serverless-workflows and use operator-based deployment instead of the kubernetes deployment
Generally speaking, the documentation should be simplified and focused on describing the implemented functionalities, the architecture and the deployment and user instructions (possibly using kn-workflow functions as much as possible)
Maybe some sections may be taken out from the workflow's README and moved to a common README
Still we need to design how the other software components (the Knative functions and eventing resources) can be managed by the common-purpose pipeline

Two more points added later:

in general all the SWFs should have a consistent error handling implementation, at least for the known errors that can come from openAPI services (this is the most common if not unique case where I'd expect some errors)
for the same reason, all the SWFs should implement a timeout expiration mechanism using the new TimedOut error introduced in the latest version of the platform.

@masayag @rgolangh

gabriel-farache commented 6 months ago

@dmartinol by

the final examples repo

You mean https://github.com/parodos-dev/serverless-workflows, right? On this point I agree, some changes will be needed (ie: removing the email sending)

I am still not sure if this repo (this one the PR, https://github.com/parodos-dev/serverless-workflow-examples) will be of any use once we get the other one ready and running, except probably for developer oriented usage (debugging when bugs, ...)

Using the operator will be nice too but for that we need a release with the changes from the 2.0.0-SNAPSHOT that we are using (ie: the fixes applied after bugs were found doing this workflow) otherwise it won't work

For the Docs/README, yes some part could be shared between all projects but we need also to provide for each projects, the steps to have it running locally and possibly how to debug it so I am not sure how we can reduce the amount of what we are writing. In my case, I have 3 README, I could reduce it to 1 in the main project but for the knative function, I want to provide as much information as possible for any one taking the project and looking to improve or fix things

dmartinol commented 6 months ago

@masayag can we LGTM now?

parodos-dev / serverless-workflow-examples

FLPATH-597 - Move2Kube SWF #8