search

manetu / temporal-clojure-sdk

A Temporal SDK for Clojure
Apache License 2.0
71 stars 10 forks source link

Cross-language workflows #43

Open purcell opened 10 months ago

purcell commented 10 months ago

Hi, and thanks so much for sharing this SDK!

I'm evaluating Temporal to help with orchestration within a polyglot Clojure + Python system, so I'm particularly considering interop between different languages, and I recognise this might not be on your list of priorities at all.

From what I can tell, cross-language workflow interop is currently tricky with this SDK for a couple of reasons:

  1. Only activities defined in clojure with defactivity can be invoked, so an activity that may be implemented by a worker in another language is inaccessible from this SDK. (As a workaround one can declare a stub activity in Clojure that never gets executed, but this feels a bit hokey.)
  2. All activity params/results are encoded with nippy rather than JSON, so by default workflow data will be unreadable by SDKs in other languages. I guess this choice was made so that Clojure-to-Clojure activity interop works smoothly, with seamless serialisation of :keywords etc. For interop beyond Clojure, would you expect that a Data Converter could be written to convert nippy data to/from JSON or similar?

I'd rather not drop down to the Java SDK if I can avoid it, so I thought I'd check in here to see if you have any tips — thanks!

ghaskins commented 10 months ago

Hi,

Yeah, at the moment the SDK is fairly opinionated that its clojure-to-clojure for the reasons you highlighted.

Re (1), its probably a small lift to update the invoke api so that it takes an interop-friendly string, rather than a clojure symbol that was created by the defactivity macro.

Re (2), it's a little trickier, and I don't think a data converter will work. This is because we need to encode things before serializing them into Java types like strings/bytes, and the data encoder would sit too low in the stack.It's simply too late to capture what we need to capture there.

What can probably be done is add our own "data converter" type mechanism at the right point of the stack, such as a defprotocol. Then, the SDK can supply a nippy-based default implementation, or users may swap it out for their own.

kpassapk commented 5 months ago

I have been using Biff recently, and I like the author's approach of providing a very opinionated implementation but documenting alternatives. A defprotocol + how-to might work nicely here.