Ability to track the sequence of conversions when using transducers

[velios]

Hi. First off all thanks for the amazing utility.

I'm not sure that the improvement I would like to see can be made, but I still want to ask a question. Suppose we have the following example.

((fn testy1 []
   (->> [-1 0 1 2 3 4 5]
        (into [] (comp (map dec)
                       (filter odd?)
                       (filter pos?))))))

I don't get any useful information for debugging when inspecting this example. All I see is a series of higher order functions being declared, but I don't see any interaction with the vector. Is there any way to modify this example to provide more useful information for debugging?

[jpmonettas]

Hi! Glad to hear FlowStorm it's being useful to you.

Yeah, that is an interesting example. The thing is there aren't any functions bodies. If you change it like this :

(defn testy1 []
  (->> [-1 0 1 2 3 4 5]
       (into [] (comp (map    #(dec %))
                      (filter #(odd? %))
                      (filter #(pos? %))))))

you can then step over all elements for each function.

[velios]

Yes, it really works and can be added for debugging time. It's funny that in The Clojure Style Guide we have recommendations not to do this. It seems that's why we can't leave such code in the real codebase and you need to switch back and forth between options. In any case, thank you that we have such a way. I hope someday a good idea will come up on how to track such situations. In fact, the function call is carried out.

[jpmonettas]

Yeah, I would have to think if it is possible to do any of this automatically. But I also think most of the time this isn't needed, because if the transducer functions are your project functions (in an instrumented namespace) then you will get tracing.

Your particular example uses transducers with clojure.core functions (which aren't instrumented) by name, which I think for real projects are more the exception than the rule. For those special cases changing them to #( ... %) notation is pretty quick.

[velios]

Simplifying track for transducers is definitely not a task of primary importance, but I will still write some details.

which I think for real projects are more the exception than the rule

Next, I’m just giving thoughts on why I would use this option instead of the common one thread-macro+map+filter approach, but since it turned out that this is more difficult to inspect via flowstorm, I’ll just stay with thread-macro+mapv+filterv option.

It might be interesting to mention that it was the flowstorm that got me thinking that writing this way might be a good idea. If you rewrite the example above into a more familiar version and start inspecting using flowstorm.

(defn my-fn [x]
  (* x x))
(defn testy3 []
  (->> [-1 0 1 2 3 4 5]
       (map my-fn)
       (filter odd?)
       (filter pos?)
       (vec)))

We'll see that we don't get into the my-fn until the vec forces the lazy sequence to be evaluated. But thanks to flowstorm, it provides sugar and we can select any section in the thread and flowstorm will calculate it for us and show the intermediate result. And for small functions this is more than enough for us.

It was always interesting why the lazy implementation of maps, filters and other functions was chosen by default. In 90% of cases in my programs I would like to immediately calculate the result of the lazy sequence. I also found confirmation in articles like Clojure's deadly sin that it is also faster. Therefore, functions such as mapv, filterv seem more convenient than their more common counterparts. And inspecting them using flowstorms is also more consistent and logical. And the option with info is an even more generalized approach, because not all functions have analogues out of the box, like keepv and removev.

[jpmonettas]

Yeah, in my case I use the threading+mapv+filterv etc option unless I need to optimize, in which case I will try transducers after measuring. But again, even in the case of transducers they are ok with FlowStorm if the involved functions are instrumented ones.