ClojureScript support possible?

[mdiin]

First of all, thanks for a great library! We are happy users of Carmine at my company. :-)

We have a situation where we use Redis as a message queue, via Carmine, and need to talk to it through a node.js backend as well. In the interest of keeping our back-ends similar across Clojure and ClojureScript, it would be nice to have ClojureScript supported by Carmine.

I haven't looked at the code in detail, so this is me throwing out a not-quite-thought-through question: Are there any major roadblocks for adding ClojureScript-support? Dependent libraries that lack support would be the primary problem I guess.

[mdiin]

Looking at project.clj the potentially problematic dependencies are:

• timbre nippy (EDITED)
• apache commons
• clojure.data.json

I don't know a ClojureScript alternative for any of them, but they must exist.

Assuming suitable alternatives for the above dependencies can be found, a quick browse through the code leads me to believe that these namespaces will require some work:

• taoensso.carmine.connections, due to a lot of Java interop code
• taoensso.carmine.tundra, due to URLEncoder and URLDecoder
  • Google closure library potentially has a good CLJS replacement for these

The Tundra-stuff could probably be left as a Clojure-only feature to begin with. Much of the code seems to be able to take a move to cljc-land without too much trouble though. What is your oppinion?

[ptaoussanis]

Hi Matthias,

First of all, thanks for a great library! We are happy users of Carmine at my company. :-)

You're very welcome, thanks for saying so!

[...] it would be nice to have ClojureScript supported by Carmine.

I'm not sure I fully understand your motivation here. So you have Carmine currently running on the JVM, and you also have a node backend. Instead of interacting with Redis through the JVM, you'd prefer to go through the node backend?

In the interest of keeping our back-ends similar across Clojure and ClojureScript

I think this is where I'm losing you. How would using Redis from ClojureScript (instead of Clojure) help keep your backends similar? (Haven't had my morning coffee yet, sorry if I'm missing something obvious).

As a general comment: Carmine's currently pretty heavily tied to the JVM for performance reasons. Things like connection handling and the read/write streams would need to be rewritten for a start. That'd be significant work, but not insurmountable if one were motivated.

Trickiest bit would be that you'd likely need/want to target a specific underlying platform for performance. I.e. you'd want a JVM-hosted Carmine or Node-hosted Carmine, not Clj/Cljs Carmine. Too much of the performance is tied up in platform interop. Nippy wouldn't work at all outside of the JVM for similar reasons (it's designed to leverage the JVM specifically).

I'm not familiar with Node, so can't say how difficult any of this would be to do well.

I do have plans to release a small subset of Carmine in the medium-term for use by a Disque client. While I'm doing that, I can keep on eye on trying to isolate platform specific stuff. That could then be something to build a minimal platform-specific port on.

Otherwise, first step would be understanding your particular motivations here. The JVM's a great platform for use with Redis, I'd be hesitant to give that up unless there's strong reasons to.

Cheers :-)

[mdiin]

I didn't explain my motivation well enough, sorry. Our main API is built on Clojure, and we want to keep it that way. However, we have run into a situation, where we need to use PhantomJS to run a small piece of Javascript that generates images from a WebGL canvas. Since that is potentially a very heavy thing to do, I would like to keep it separate from our main API. This is where nodejs becomes relevant, since it has good interop with PhantomJS. The Redis MQ would then be the path of communication between main API (Clojure) and the image generation server (nodejs). So the basic structure I am looking at is something like this:

I am not trying to replace Clojure with ClojureScript, just create a path of communication between the two. Since we are using Redis through Carmine already, using the message queue functionality makes sense. Furthermore, being able to use the same libraries for the Redis MQ on both accesspoints to the message queue makes for a simpler codebase

That said, I am not really familiar with Node myself, so I wouldn't know how to do the performance-critical Node-interop without some serious time-effort. It would be interesting to look into though. :-)

(On a, only partly related, side-note, I think I should look a little more at driving PhantomJS from Clojure/Java before I continue down the Node-path)

Thanks for your detailed answer. :-)

[ptaoussanis]

Furthermore, being able to use the same libraries for the Redis MQ on both accesspoints to the message queue makes for a simpler codebase

Okay, understood - thanks for the clarification.

My short/medium term recommendation would be to stick with a node Redis client for now. Longer term - as I mentioned - I have plans to split Carmine into a pair of libs including a core that can be shared with a Disque client.

I've made a note for myself to investigate a possible cross-platform core while I'm doing that (thank you for the suggestion). I.e if/when I get around to that, there might be a potential long-term path to multi-platform Carmine support.

I should warn: I've got interest in the Disque client myself, so I'll definitely be working on that when time allows. But I've got no personal interests in running Carmine on non-JVM platforms, so even if we end up with a cross-platform capable core, it's still going to need someone else's efforts to get a node extension.

Closing this for now, but have opened https://github.com/ptaoussanis/carmine/issues/144 which'd be a first step in the right direction.

Hope that helps? Sorry I couldn't offer a better short-term solution.

[mdiin]

That solution is definitely acceptable, and I appreciate your honesty on the matter. If it turns out that I fail to drive PhantomJS from Clojure I will continue down the Node path, and (if the Node solution ends up sticking around in our production systems) maybe look at porting Carmine using your portable core. Lots of _if_s there, but I don't feel I can commit to anything at the moment, since my primary interests in a port are tied to the priorities of my day-job.

[theronic]

@mdiin which NodeJs library are you using to interact with Redis from ClojureScript? I have the same situation.

[mdiin]

Hi @theronic,

Since I wrote the comment almost five years ago a lot has changed, but most importantly my then-employer never prioritized the feature that would have caused me work on this. Because of that, I never got to interact with Redis from Clojurescript.

Sorry for the late reply. I hope you find a solution.