REPL threading problem

[marceljanssens]

I am doing some redis experiments, and have a localhost redis server running for development and test. I am working inside the REPL for test and debug. The code below (I could pretty much simplify it down to the example code from the carmine README) works fine when run as a test program, but when trying to execute the same code in the REPL, the "publish" (last command in the trace below) does not end up being received by the subscription callbacks defined a few lines above.

A redis-cli running in another terminal, subscribed to the foobar channel, does receive the message.

As mentioned, It works fine when I turn this into a standalone program. It is also a little strange that the "subscribe" messages themselves are received fine. I assume this has something to do with the fact that I have the subscriber and publisher be in the same REPL and the task threading of the REPL, but it may also point to a threading issue in the library. I have googled possible REPL threading issues, but could not find anything.

------------------- terminal trace ------------------------

Run on MacBook Pro with El Capitan 10.11.2 using standard Apple terminal. Redis server running on localhost on standard 6379 port. Responses indented for readability. Comments indicated by <------.

lein repl nREPL server started on port 55405 on host 127.0.0.1 - nrepl://127.0.0.1:55405 REPL-y 0.3.7, nREPL 0.2.10 Clojure 1.7.0 Java HotSpot(TM) 64-Bit Server VM 1.8.0_60-b27 Docs: (doc function-name-here) (find-doc "part-of-name-here") Source: (source function-name-here) Javadoc: (javadoc java-object-or-class-here) Exit: Control+D or (exit) or (quit) Results: Stored in vars 1, 2, 3, an exception in e

user=> (require '[taoensso.carmine :as car :refer (wcar)]) nil

user=> (def local-pool {:pool {}

_=> :spec {:host "localhost"

_=> :port 6379

_=> :timeout-ms 1000}})

'user/local-pool

user=> (def listener

_=> (car/with-new-pubsub-listener (:spec local-pool)

_=> {"foobar" (fn f1 [msg](println "Channel match: " msg))

_=> "foo*" (fn f2 [msg](println "Pattern match: " msg))}

_=> (car/subscribe "foobar" "foobaz")

_=> (car/psubscribe "foo*")))

'user/listener

Channel match: [subscribe foobar 1] <------ subscribe messages arrive Pattern match: [psubscribe foo* 3]

user=> (defmacro wcar* [& body] `(car/wcar local-pool ~@body))

'user/wcar*

user=> (wcar* (car/publish "foobar" "Hello to foobar!")) 4 <-------- return value OK, but no messages arrive

user=>

[ptaoussanis]

Hi Marcel,

So a quick recap: you've got this code that seems to work fine in a stand-alone deployment but it seems like your pub/sub messages aren't getting delivered when running at the REPL?

How are you deciding whether a message has been delivered? Is it possible you're printing to std-out or similar and that your REPL environment is capturing your output?

If you're printing to show handler delivery, try do something else that doesn't depend on std-out (e.g. write to a file, write a Redis entry, etc.).

Does that help?

[marceljanssens]

Hi Peter,

thanks for commenting. Yes, my actual test code (I used the example because I didn't want to drag in all kinds of additional stuff) just swaps the result into an atom, which I then look at afterwards (ok for subscribe fail for publish). I have also run the test in the debugger, and again the subscribe message triggers the breakpoint on the first line of my specified message handler, but the publish message does not.

To me the strange part is that the subscribe message arrives, but the publish message does not (even though it does arrive at other redis-cli clients subscribed to the same subject). This would indicate to me that the library lost the reference to the callback somehow, or that the delivery mechanism used for the subscribe is different from that used for the publish (as far as threading is concerned).

Any insight would be appreciated.

[ptaoussanis]

I have also run the test in the debugger, and again the subscribe message triggers the breakpoint on the first line of my specified message handler, but the publish message does not.

Not sure what debugger you're talking about, but note that that kind of thing can also be sensitive to the thread you're running on; wouldn't consider that a reliable way of testing for delivery unless you understand your debugger well.

To me the strange part is that the subscribe message arrives, but the publish message does not (even though it does arrive at other redis-cli clients subscribed to the same subject)

The publish message does not? You mean the published message? Or are you looking for some kind of response on the publish call itself (there isn't any)?

Let's do this- can you create a reproducible repo somewhere that I can look at that clearly describes the behaviour you're expecting and what behaviour you're seeing that seems wrong? Might be easiest to proceed that way :-)

[marceljanssens]

Not sure what debugger you're talking about, but note that that kind of thing can also be sensitive to the thread you're running on; wouldn't consider that a reliable way of testing for delivery unless you understand your debugger well.

I was using the Intellij Cursive debugger, just to get another way of checking what is happening. I understand it is not quite reliable, but it is another indicator.

The publish message does not? You mean the published message? Or are you looking for some kind of response on the publish call itself (there isn't any)?

You are correct, I meant the message created by the publish call.

Let's do this- can you create a reproducible repo somewhere that I can look at that clearly describes the behaviour you're expecting and what behaviour you're seeing that seems wrong? Might be easiest to proceed that way :-)

I will think of the best way to do this and let you know where to find it.

[marceljanssens]

Wouldn't you know that when I created the new environment, things started to work :). At first I thought I had made a mistake in my code, but then I reduced my code down to the same bones as my new project, and it still wasn't working in mine, but was in the new project.

Anyway, I finally figured out the difference. In my project the connection pool spec was set to:

(def local-pool {:pool {} :spec {:host "localhost" :port 6379 :timeout-ms 1000}})

and in the new project it was set to:

(def local-pool {:pool {} :spec {:host "localhost" :port 6379}})

Only difference was the timeout. It works when you don't specify the timeout, and it doesn't work when you specify the timeout.

[marceljanssens]

We can close the issue as far as I'm concerned. It works fine normally, and this gives me a work-around for my REPL tests.

Kindly, Marcel

[ptaoussanis]

Great, happy you found a solution. BTW I wouldn't consider this a workaround; you must have the connection timeout disabled for Pub/Sub connections or the connection will timeout.

Best of luck, cheers! :-)