Metadata handling by `postwalk` in some cases

[andrewboltachev]

In one of the projects I want to do a thing as complicated as transforming a structure based on it's value (with children already being processed) and it's metadata. And I want to return a brand new structure based on that (structure with children already processed and metadata).

This means that I should use postwalk, 'cause when I would return a brand new strucutre, I don't want a program to enter it anymore (and prewalk would enter).

My interest is the function f, which I expect to receive the value (with children already processed) and metadata. So I constructed a function which acts like identity (as I don't care about actual transformation yet) and check, what it's been called with:

(defn print-with-meta-and-return [value]
  (binding [*print-meta* true]
    (prn value)
    (newline))
  value)

And when I use it like that:

(potemkin.walk/postwalk
  print-with-meta-and-return
  '[^:foo (x)])

I've got such output (from prn and newline):

x

(x)

[^{:line 1, :column 54, :foo true} (x)]

But I've expected the 2nd element to receive metadata, i.e. to be:

^{:line 1, :column 54, :foo true} (x)

(This additional line/column meta-information might be another question, although I ignore it for a moment. This is prehaps just how Clojure reader works on lists)

I later realized that on vectors it behaves well:

user=> (potemkin.walk/postwalk print-with-meta-and-return '[^:foo [x]])
x

^{:foo true} [x]

[^{:foo true} [x]]

[[x]]

(the last line here is the result)

---

Long story short, if we look at the code, we see that in case of postwalk the f would be outer function (in walk).

And in case of list it receives the result of (apply list ...): https://github.com/ztellman/potemkin/blob/f1a13fd07e294413610a09d4eade92a46102bfa7/src/potemkin/walk.clj#L9

but e.g. in case of a vector it would receive (into (empty form) ...), which rather preserves metadata: https://github.com/ztellman/potemkin/blob/f1a13fd07e294413610a09d4eade92a46102bfa7/src/potemkin/walk.clj#L14

---

Going on, it turns out that results are similar for both potemkin.walk and clojure.walk:

tracer.core=> (binding[*print-meta* true]
         #_=>   (println "potemkin on list:")
         #_=>   (prn (potemkin.walk/postwalk print-with-meta-and-return '[^:foo (x)]))
         #_=>   (println "potemkin on vector:")
         #_=>   (prn (potemkin.walk/postwalk print-with-meta-and-return '[^:foo [x]]))
         #_=>   (println "clojure on list:")
         #_=>   (prn (clojure.walk/postwalk  print-with-meta-and-return '[^:foo (x)]))
         #_=>   (println "clojure on vector:")
         #_=>   (prn (clojure.walk/postwalk  print-with-meta-and-return '[^:foo [x]])))
potemkin on list:
x

(x)

[^{:line 3, :column 61, :foo true} (x)]

[^{:line 3, :column 61, :foo true} (x)]
potemkin on vector:
x

^{:foo true} [x]

[^{:foo true} [x]]

[^{:foo true} [x]]
clojure on list:
x

(x)

[(x)]

[(x)]
clojure on vector:
x

^{:foo true} [x]

[^{:foo true} [x]]

[^{:foo true} [x]]

I.e. when handling list, the 2nd node is metadata-less ((x)), but in case of vector the 2nd node has metadata (^{:foo true} [x]). And the reason is that (apply list ...) as opposed to (into (empty form) ...).

---

So, speaking of metadata preservation, the main question is — how the actual metadata perservation should happen?

1. Is potemkin.walk intended to pass structure's metadata into f (as it does now for e.g. vector and other structures but not for list)?
2. Is it intended rather to restore metadata after f has returned (as it does now)?

On that, consider such “wrapper” use case:

user=> (binding [*print-meta* true]
              (prn (clojure.walk/postwalk (fn [x] (if (vector? x) {:value x} x))
^{:a 111} [1 ^{:a 222} [2 3] 4])))

{:value ^{:a 111} [1 {:value ^{:a 222} [2 3]} 4]}
nil

user=> (binding [*print-meta* true]
              (prn (potemkin.walk/postwalk (fn [x] (if (vector? x) {:value x} x))
^{:a 111} [1 ^{:a 222} [2 3] 4])))

^{:a 111} {:value ^{:a 111} [1 ^{:a 222} {:value ^{:a 222} [2 3]} 4]}

i.e. in f here I'm destroying the actual structure, and potemkin.walk adds metadata once again.