Open brucearctor opened 3 years ago
kanterov
commented
3 years ago flytekit-java workflows and tasks aren't different from Python tasks once they are deployed. flytectl registration doesn't work for flytekit-java, because it doesn't have "serialize" command as flytekit-python does. jflyte can register workflows instead of using flytectl. Once tasks and workflows are registered, flytectl can be used in the same way as for flytekit-python.
brucearctor
commented
3 years ago Got it ... I'm wondering about the eventual abiility for flytectl to be the go-to for registering workflows no matter the language. Though, perhaps that is not feasible (which is OK, too).
kumare3
commented
3 years ago @kanterov we should add a serialize command, as we want to standardize this tooling and it helps with debugging, porting things over and sharing protobufs. It also decouples the logic of registration, various authentication support etc that jflyte would need to do otherwise.
Also I think this should be feasible
kanterov
commented
3 years ago Agree. I was working on refactoring to support dynamic workflow tasks and can add implementation for serialization for it. Due to dynamic tasks, we had to change how serialization is done because the execution of dynamic workflow tasks looks similar to registration.
kanterov
commented
3 years ago One detail, due to how registration is done, we have to stage jars during serialization. Otherwise, we have to put a lot of logic that is subject to change into flytectl.
kumare3
commented
3 years ago Also @kanterov with fast register- staging jars looks similar isn't it?
github-actions[bot]
commented
1 year ago Hello š, This issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will close the issue if we detect no activity in the next 7 days. Thank you for your contribution and understanding! š
github-actions[bot]
commented
1 year ago Hello š, This issue has been inactive for over 9 months and hasn't received any updates since it was marked as stale. We'll be closing this issue for now, but if you believe this issue is still relevant, please feel free to reopen it. Thank you for your contribution and understanding! š
github-actions[bot]
commented
2 months ago Hello š, this issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will engage on it to decide if it is still applicable. Thank you for your contribution and understanding! š
Motivation: Why do you think this is important? Flytectl is the to-be tool for interacting with Flyte. Flyte is to support tasks/workflows in multiple languages (notably python, which is well established and java/scala). FlyteCTL therefore should work with the various supported languages.
Goal: What should the final outcome look like, ideally? Flytectl allows the same [or similar] registration, running, etc of workflows for Java/Scala [or future languages] as it does for Python
Describe alternatives you've considered Using JFlyte ...
Caveats The lack of support but ability to work comes from a slack conversation: https://flyte-org.slack.com/archives/C014PL3HUTF/p1626473952005700 ... Perhaps this is more about bundling JFlyte. I imagine once workflows are registered (regardless of language) that Flytectl should be able to trigger/run them -- but I haven't tested that, yet. Anyways, seems I should clarity the extent of what specifically is possible and what doesn't work level-of-effort, etc -- and perhaps this warrants a design doc [ @brucearctor willing to take on if that is the case ] rather than a github issue ( will also make sure to connect with @kanterov ).