Howto: cause short-circuiting selects to return collected (`putval`) values

[marick]

Summary: how can an implementation of select* get access to the collection collect-val builds?

---

ONLY is like ALL, except it is to complain whenever it descends into a non-singleton collection. Currently, it complains by throwing. Instead, I want it to return a special value, as that will allow a better error message (etc). Here's an implementation that doesn't work:

(extend-type OnlyVariantType
  sp/StructurePath
  (select* [this structure next-fn]
    (cond (not (coll? structure))
          (boom! "%s is not a collection" structure)

          (not= 1 (count structure)) ; TODO: make this work for infinite collections
          :return-some-value-that-indicates-truncation
          ; (->Truncation explain/err:only-wrong-count structure))

          :else
          (next-fn (first structure))))
  (transform* [& _] (no-transform!)))

This works as you'd expect for a simple select:

user=> (select [:a ONLY] {:a [3]})
[3]
user=> (select [:a ONLY] {:a []})
:return-some-value-that-indicates-truncation

... and for a successful traversal with a select that uses putval:

user=> (select [(putval "HI!") :a ONLY] {:a [3]})
[["HI!" 3]]

However, I was initially surprised when it didn't work for a short-circuited traversal:

user=> (select [(putval "HI!") :a ONLY] {:a []})
:return-some-value-that-indicates-truncation

The reason is pretty clear: whatever is the last step in a traversal has to conj the leaf/return value onto the collected values. But for the life of me, I can't find an accessor that a select* function can call. (It would also be convenient, though not necessary, for the select* to be able to "pass" a modified collection to the rest of the selector path.)

[nathanmarz]

It doesn't work because select is meant to only return what is fully navigated to – short-circuiting the navigation with your own results isn't a supported use case. Thinking through it, I'm not sure how that would even be implemented without greatly affecting the efficiency of all navigations.

There is a way to do this though, and that's to reformulate it so that non-singleton sequences navigate you to the error code, like this:

(defn only-or-complain [& only-path]
  (cond-path
             #(not (coll? %))
             (view (fn [_] :NOT-COLLECTION))

             [(view count) #(not= 1 %)]
             (view (fn [_] :NOT-SINGLETON))

             STAY
             [FIRST only-path]
             ))

user=> (select [:a (only-or-complain)] {:a [3]})
(3)
user=> (select [:a (only-or-complain)] {:a []})
(:NOT-SINGLETON)
user=> (select [:a (only-or-complain)] {:a 1})
(:NOT-COLLECTION)
^Muser=> (select [(putval "HI! :a (only-or-complain)] {:a [3]})
(["HI!" 3])
user=> (select [(putval "HI!") :a (only-or-complain)] {:a []})
(["HI!" :NOT-SINGLETON])
user=> (select [(putval "HI!") :a (only-or-complain)] {:a 1})
(["HI!" :NOT-COLLECTION])

The navigation path you want following successful navigation into the singleton is specified as the argument to only-or-complain. This formulation works for transforms as well as its just the composition of other navigators.

[marick]

Given that I've ended up writing my own StructurePath-extending types for everything except predicates, I've decided it makes most sense to skip the collect-val-style selector elements and do everything inside the select* functions. Those functions aren't passed values to traverse, but rather structures of my devising, like:

{:whole-value ...original structure...
:path []   ; grows as each element processes it
;leaf-value ..original-structure.. ; gets narrowed as `select*` processes elements; 
}

I was previously kludging around this by preprocessing selectors with code like this:

```clojure
(defn- surround-with-index-collector [elt]
  (vector (specter/view #(map-indexed vector %))
          elt
          (specter/collect-one specter/FIRST)
          specter/LAST))

That worked OK for a while, but I hill-climbed to a local maximum that didn't give me the control I needed.

I do intend to keep working on the book, so I will eventually query you about how out-of-band collecting works. I confess I can't figure it out.

[nathanmarz]

If value collection or late-bound parameterization is used, regular structure paths get lifted into a format that takes in more arguments. These "rich" paths pass along collected values as well as late-bound parameter info. See https://github.com/nathanmarz/specter/blob/master/src/clj/com/rpl/specter/impl.cljx#L299

Specter could expose that that interface directly, but I'm hesitant because it seems that should be kept as an implementation detail. I also have no use cases for that functionality to justify it.

Good to hear you will continue working on the book. Happy explain anything you need to know about Specter, so feel free to ping me. Since you started Specter got a lot of new capabilities.