More idiomatic SCAN variants, as lazy-seqs of kv-pairs

[blak3mill3r]

I wonder if this is within the scope of this library or not.

I have at times needed to use SCAN, HSCAN etc with carmine, and pulled out something which is at least a little reusable, to avoid having to deal with the details of the CURSOR argument:

  (defn- redis-hash-scanner
    "Lazy sequence wrapper around Redis HSCAN command"
    ([key] (redis-hash-scanner key 0))
    ([key cursor]
     (let [[next results] (wcar* (car/hscan key cursor "COUNT" 1000))
           next (Integer/parseInt next)]
       (lazy-cat results
                 (when-not (= 0 next)
                   (redis-hash-scanner key next))))))

Then I (partition 2 (redis-hash-scanner k)) and I have a nice lazy-seq of [k v] pairs and don't have to touch cursors.

I could gussy this up and make a PR, I have a version for SCAN with a key-prefix as well.

I have a couple questions before I do that:

1. is this feature within the scope of carmine?
2. in the code above, I'm cheating and using a wcar* macro which refers to a particular carmine connection; is there a dynamic var or something it should use instead? ... perhaps taoensso.carmine.protocol/*context* ?
3. if it is within the scope of carmine, do you have any opinions about the code organization (new namespace?) or anything else that would help me make the PR one that you would like to merge?

Thanks for this wonderful library, it has been a pleasure to use.

[ptaoussanis]

@blak3mill3r Hi Blake, thanks for pinging.

That's a nice idea!

Just checking: are you aware of reduce-scan and reduce-hscan in Carmine? A reduce-based scan offers some advantages, incl. the flexibility of filtering (transducing in general - incl. deduplication), accumulation, early termination, etc.

Re: your questions-

1. Definitely w/in scope, but would want to understand if there's a specific motivation to add something besides the current reduce utils above. Not opposed in principle.

2. Would advocate for the fn to take an explicit conn-opts arg that can be passed to wcar.

3. Thanks for asking! Definitely not a new namespace, taoensso.carmine would be fine. (Otherwise it's a nuisance for consumers to have to import and remember a bunch of namespaces that offers little benefit). I'd focus first on addressing point 1 above, then can take a closer look at your implementation.

Again, much appreciated, cheers!

[blak3mill3r]

Thanks. I was not aware of reduce-scan and this fits my use-case just fine, and is better than the lazy-seq solution I was using. Since I cannot think of a compelling use-case for this, I'm closing this issue.

[ptaoussanis]

Great, thanks for letting me know!