search

Bas-vE / LV-Redis

LabVIEW Redis library
MIT License
1 stars 0 forks source link

Good that I found you / discuss about LV-redis toolkits #1

Open EastEriq opened 3 weeks ago

EastEriq commented 3 weeks ago

Hi, I'm currently working out my way in what I thought were the only two public toolboxes for Redis, till I had the intuition of checking also github, and found yours. Since I see that you just began committing to it, I thought it may be interesting to exchange views.

I myself was aware of:

  1. the old set on NI forum, wrapped around redis-in-labview of Martjin Jasperse, from where you started as well. I forked it in https://github.com/EastEriq/redis-in-labview, merging it with Carlo Sandroni example of pub-sub.
  2. Nick Folse's https://github.com/tauterra/Redis-Client-for-LabVIEW, which I forked in https://github.com/EastEriq/Redis-Client-for-LabVIEW

Unfortunately for my use case I find both of them flawed, and I was considering where to plug in in order to adapt to my needs. My immediate goal would be a pipelined query of multiple HVALS from multiple servers, and both implementations fall short of it. This is for the moment my only yardstick, and I wonder how much I would be interested myself in planning ahead and expand a solution to a viable toolbox; I might end up putting together a snippet of code which just does what I need, and leave it there. On the other hand, if there is common interest we could also cooperate.

As for my critics:

  1. opens and closes the TCP connection for every transaction, and this is a complete no-brainer for pipelining, at least with the version of redis-server (7.4.0) which I'm using. A cursive look at your code convinced me that this is also your case. I was considering to take Iwac's code, remove all Open/Close TCP from his core Send_Command_Revf.vi and replace his session reference (merely a reference to a Host-Port cluster) with a reference to the current TCP session, opening and closing that session only at the beginning/end of the chain.

  2. is a bit unfinished, and left at the state of three years ago. Its author also confirmed that he is no more working on it. His toolbox carries around a class which accounts for # of responses pending when in pipeline mode, but: a) the mechanism does not work as it is for commands like HVALS which return an a priori unknown number of lines, b) it doesn't work for it's alternative HGETALL which returns a MAP response, which is not decoded. My least effort with this may be therefore to implement the treatment of the MAP case. Besides that, there were a couple of other flaws in his example (see https://github.com/tauterra/Redis-Client-for-LabVIEW/issues/1), but they could easily be fixed.

See also this LAVA thread: https://lavag.org/topic/23160-does-it-make-sense-to-use-redis-instead-of-shared-variables/#comment-150638.

What do you think about?

Enrico

Bas-vE commented 2 weeks ago

Hi,

I've been pretty busy the past few weeks and forgot I got a message on github. I started this project because I already needed to create a better version with authentication and continued to implement the rest of the API starting with the Lists and continuing with the Keys part.

I'm pretty new to REDIS but i think that providing a VIPM published package for redis and Valkey is giving the community easier access to these platforms. With that said, I have not been playing with the pipelining and could be a nice feature. And this would be the perfect time to implement that. unfortunately i've been pretty swamped with work and don't have the time to read on this topic and test code for this but if you have some ideas how pipelining could be implemented in my code I would like to hear about that.

EastEriq commented 1 week ago

Hi, I'm no expert of redis either, having landed on it quite recently. But picking up.... I finally moved on adopting my own fork of the original set, what I mentioned above in 1., for my project. I have since then not thought about how could I contribute to yours.

As for the comment on pipelining, I still think that the showstopper in your implementation as well in the others is the repeated openTCP-close for each elementary operation. Probably MULTI operations coming from "occasional" clients are rightly discarded by design in Redis, as the main purpose is of the construct is to provide monolithic evaluation of multiple commands. Up to you if you intend to support them or not. I haven't thought of possible workarounds in your implementation, which I also don't want to subvert. In my fork, I simply went for the open session once - do all session operation - close paradigm.

In fact, for my repeated HVALS, I even moved forward from MULTI-EXEC. I realized that each command given involves a transaction command-acknowledgement anyway, so I am now testing an implementation where I lump all the keys I want to know about in a single call to a lua script using EVALSHA. But this of course is particular of my needs, and not a general solution, and complicates the application code. (not to mention, that from Redis 7, SCRIPT is deprecated in favour of FUNCTION).

Best, Enrico