How to force blank lines between vector elements?

[ghost]

Hi, I’m using zprint as a library to write out in-memory values to .edn files. I’ve got a lot of vectors containing multiple maps, and I’d like to force a blank line between each map. Is there a way to do this?

In other words:

given the input

{:foods [{:name "broccoli"} {:name "cauliflower"} {:name "kohlrabi"}]}

(as an in-memory data structure, not as a string)

I’d like my call to zprint to write out:

{:foods [{:name "broccoli"}

         {:name "cauliflower"}

         {:name "kohlrabi"}]}

Is this possible?

Thanks!

[ghost]

Also, I think in some cases I’d like to force two blank lines between vector elements… so I’d love to hear if that’s possible as well. Thank you!

[kkinnear]

I am pretty sure this is possible. That said, it isn't as easy as doing it is lists -- where I added a new feature to do just that (i.e., add blank lines between elements) just recently. Vectors don't current support that, so I can't give you a quick way to get what you want in vectors.

However, I expect I have another way to make this happen in vectors that isn't too complex and will work with the currently shipping version of zprint, but I need to try it out first. I'm tied up just now, so it may be a few days before I can figure out solution to this, but I will do so. Thanks for asking!

[ghost]

Thanks so much!

[kkinnear]

I have a way for you to do this, and it is relatively simple. That said, it is a total hack that depends on exactly how the code is written in a way that isn't long term supportable. I think what you want to do is perfectly reasonable, and I want to build a supported (and supportable) way to do it, but that's not going to happen until the release after next, I expect.

However, here is a way for you to do what you want with the currently available code:

zprint.core=> (czprint i283 {:parse-string? true :vector-fn {:nl-count 2 :wrap-coll? false :indent 1} :vector {:fn-format :wrap :force-nl? true} :fn-force-nl #{:wrap}})
{:foods [{:name "broccoli"}

         {:name "cauliflower"}

         {:name "kohlrabi"}]}

This has several caveats:

1. It affects all vectors. If you have vectors inside your internal maps, let me know, and I'll fix it so that it only affects the first vector encountered.
2. It depends on there being a collection (e.g., a map) inside of the vector where you want blank lines. As long as there is at least one collection inside of the vector, you will get the blank lines.

You can control the number of blank lines with :nl-count n, where 'n' is the number of newlines. So n = 2 gives you one blank line, n = 3 gives you two blank lines, etc.

In case you were interested, here is what is happening: the :vector {:fn-format :wrap ...} sends the vector code off to be formatted like a list. It turns out that the function type :wrap is the only one that goes through code that doesn't treat the first element is a list as a function, and it happens to have two arms -- one of which goes through a section of code that supports :nl-count when there is a collection in the list (or vector, in this case) and when you say that you don't want wrapping when there is a collection -- the :wrap-call? false. The :vector-fn {...} is where the configuration for vectors that are being printed as lists happens to reside.

Hopefully this will work for your use case. If it doesn't, let me know, and I have another approach which is a little more complex which will. I have no plans to change how this works, so you can keep using this for a long while. I will, however, be building a way that I will document and support long term that you might want to move to later, though that probably won't be required.

[ghost]

Gave this a try, and I’m using it in my project for now. It’s not perfect but it’s pretty close! Thank you!

[ghost]

(Not sure if you want this to stay open to track a more formal solution; feel free to close of course. Thanks!)

[kkinnear]

I'm glad it helps, but if it isn't perfect I would appreciate knowing what perfect would look like. When I make a more formal solution to this issue, it is going to work just like this one, so if it isn't right -- that isn't good. If you give me an example that isn't perfect now, and show me what you want it to look like (that is, what it would look like if it were perfect), I'll see what I can do to make that possible now and take that into account in a future "real" solution. Thanks!

[ghost]

Here’s an example of a file I just formatted wherein some of the vectors have blank lines between their elements, and some don’t:

{:foo/bar
 [{:foo/bar :activities
   :foo/bar [{:foo/bar :foo/bar.sigmoidopexy
              :name "pseudowhorl"
              :estimated-duration-business-days 5
              :teams [{:foo/bar :foo/baz}]}

             {:foo/bar :foo/bar.nonpolitical
              :name "cardiosclerosis"
              :estimated-duration-business-days 5
              :teams [{:foo/bar :foo/baz}]}

             {:foo/bar :foo/bar.kingwood
              :name "predisclosure"
              :estimated-duration-business-days 5
              :teams [{:foo/bar :foo/baz}]}]}
  {:foo/bar :programs
   :foo/bar [{:foo/bar :foo/baz
              :name "bardel"
              :activities [{:foo/bar :foo/bar.cosmic
                            :start-offset-days 0}

                           {:foo/bar :foo/bar.objectivism
                            :start-offset-days 0}

                           {:foo/bar :foo/bar.nonperforming
                            :start-offset-days 0}

                           {:foo/bar :foo/bar.splenodynia
                            :start-offset-days 0}

                           {:foo/bar :foo/bar.Cashmere
                            :start-offset-days 5}

                           {:foo/bar :foo/bar.undelectably
                            :start-offset-days 5}

                           {:foo/bar :foo/bar.theosopheme
                            :start-offset-days 20}]}
             {:foo/bar :foo/baz
              :name "irresonant"
              :activities [{:foo/bar :foo/bar.phytogenetical
                            :start-offset-days 0}

                           {:foo/bar :foo/bar.oriental
                            :start-offset-days 5}]}]}]}

and here’s another file, same settings, that for some reason doesn’t have blank lines between any vector elements:

{:foo/semiyearlys
 [{:foo.cadinene/name :indiscussible
   :foo.sacculated/hyenanchins
   [{:foo.passe/id :bicursal/company
     :centrolepidaceous {:foo.walletful/idref :taurolatry/manganic}
     :infraprotein [{:foo.acalephoid/idref :symptomatography/definitively-review
                     :planned-start-date "2023-01-30"
                     :activity-overrides [{:foo.unaccomplishedness/idref :foo/bar
                                           :start-offset-days 5}]}
                    {:foo.runcinate/idref :foo/bar
                     :planned-start-date "2023-03-20"}]}]}]}

Here are the settings I’m using:

{:configured? true
 :style [:community]

 ; This was provided by the maintainer of zprint here:
 ;   https://github.com/kkinnear/zprint/issues/283#issuecomment-1371322037
 :vector-fn {:nl-count 2 :wrap-coll? false :indent 1} :vector {:fn-format :wrap :force-nl? true} :fn-force-nl #{:wrap}

 :map {:comma? false
       :force-nl? true}
 :width 120
 :output {:real-le? true}}

Thanks for the help!

[kkinnear]

Ah, yes. Sorry about that. The thing I gave you only works for vectors with more than two elements, sigh. Here is one that works better, and should handle vectors of any length.

(czprint i283e {:parse-string? true :vector {:option-fn (fn ([] "vector-lines") ([options len sexpr] {:guide (into [] (->> (repeat (count sexpr) :element) (interpose [:newline :newline]) flatten))}))}})
{:foo/bar
   [{:foo/bar :activities,
     :foo/bar [{:estimated-duration-business-days 5,
                :foo/bar :foo/bar.sigmoidopexy,
                :name "pseudowhorl",
                :teams [{:foo/bar :foo/baz}]}

               {:estimated-duration-business-days 5,
                :foo/bar :foo/bar.nonpolitical,
                :name "cardiosclerosis",
                :teams [{:foo/bar :foo/baz}]}

               {:estimated-duration-business-days 5,
                :foo/bar :foo/bar.kingwood,
                :name "predisclosure",
                :teams [{:foo/bar :foo/baz}]}]}

    {:foo/bar :programs,
     :foo/bar
       [{:activities [{:foo/bar :foo/bar.cosmic, :start-offset-days 0}

                      {:foo/bar :foo/bar.objectivism, :start-offset-days 0}

                      {:foo/bar :foo/bar.nonperforming, :start-offset-days 0}

                      {:foo/bar :foo/bar.splenodynia, :start-offset-days 0}

                      {:foo/bar :foo/bar.Cashmere, :start-offset-days 5}

                      {:foo/bar :foo/bar.undelectably, :start-offset-days 5}

                      {:foo/bar :foo/bar.theosopheme, :start-offset-days 20}],
         :foo/bar :foo/baz,
         :name "bardel"}

        {:activities [{:foo/bar :foo/bar.phytogenetical, :start-offset-days 0}

                      {:foo/bar :foo/bar.oriental, :start-offset-days 5}],
         :foo/bar :foo/baz,
         :name "irresonant"}]}]}

I added just the :option-fn for clarity -- it should work ok with the rest of your configuration as well (at least it did when I tried it). You will want to take out all of the previous thing I gave you -- this replaces all of it.

If you want more than one blank line, increase the number of :newline elements in the interpose.

Thanks for getting back to me on this, when I do this "for real", it will work better because of your testing!

[ghost]

My pleasure, and thanks again for the thorough and detailed assistance!

I just tried to integrate that into my config but I’m running into a problem.

I updated my config with this entry:

:vector {:option-fn (fn
                                         ([] "vector-lines")
                                         ([options len sexpr]
                                          {:guide (into [] (->> (repeat (count sexpr) :element)
                                                             (interpose [:newline :newline])
                                                             flatten))}))}

but when I call set-options! I get this error:

Exception: set-options! for repl or api call 5 found these errors: In repl or api call 5, The value of the key-sequence [:vector :option-fn] -> (fn ([] "vector-lines") ([options len sexpr] {:guide (into [] (->> (repeat (count sexpr) :element) (interpose [:newline :newline]) flatten))})) was not a clojure.core/fn?

I’m a little confused by this. I guess the problem is that I’m storing my config in an .edn file, and at runtime I’m parsing it with clojure.edn/read-string so the value for :option-fn is indeed not a function; it’s a list. I didn’t realize that set-options! expected the config to have been compiled, i.e. fed into the Clojure reader.

Assuming I’ve understood the problem correctly, is there a quick/easy way to feed the Clojure map that was parsed from the .edn file into the Clojure reader? Or do I need to use, say, pr to convert the map back into a string first?

Thanks!

[kkinnear]

At least the error message was helpful...

This is a tough one. You could convert it back into a string first, but then you'd have to read it with something that would turn it into a real function, which you probably don't really want to do, because it is a security hole of significant size. The zprint binaries get around this by reading the config from files using sci, which "sandboxes" the execution of the functions so that they can't do anything bad. But there is not, at present, any way to get the sci built inside of zprint to accept a string as a configuration input. Which is an interesting idea, but not one that is going to do you any good today.

Depending on how much you want to hack things up temporarily to make this all work, one approach would be to simply put the function into your code somewhere (which is what I did when testing it originally). Then you could read in the config like you do now from the .edn file -- without the :vector {:option-fn ...} and do your normal set-options. Then do another set-options with the :vector {:option-fn your-fn-name} in it to set up the multi-line stuff. Then, when zprint does the multi-line vectors "for real", you can remove that code. Not optimal, but a lot safer than reading in the .edn file to something that will allow someone to do functions. I think that is what I would do, unpretty as it is.

That seems to strike a balance between safety and getting the output you want.

[ghost]

At least the error message was helpful...

Oh yeah, this is no small thing!

…which you probably don't really want to do, because it is a security hole of significant size.

Actually, in my case, it’s fine, as this .edn file is in my private repo and it’s trusted. So I’m just passing the map to eval and that seems to be working just fine; the function is compiled. So this problem is now solved. Thanks so much for your generous and thorough help!

Now that I’m actually running your most recently suggested settings for :vector though, I’m getting a new error. (It seemed to work at first, an hour ago, but I just tried it again and now it’s not working? 🤷)

The error is:

>>> Exception: Unknown values: guide-seq: '' 
cur-zloc:nil

And here’s most of the stack trace:

most of the stack trace

```text zprint.core/zprint core.cljc :961 clojure.core/apply core.clj :673 zprint.core/zprint-str-internal core.cljc :792 zprint.core/zprint* core.cljc :555 zprint.core/fzprint-style core.cljc :425 zprint.sutil/sredef-call sutil.cljc :278 zprint.redef/redef-vars redef.cljc :106 zprint.sutil/sredef-call/fn--22431 sutil.cljc :354 clojure.core/partial/fn--5912 core.clj :2654 zprint.zprint/fzprint zprint.cljc :7904 zprint.zprint/fzprint* zprint.cljc :7768 zprint.zprint/fzprint-map zprint.cljc :7166 zprint.zprint/fzprint-map* zprint.cljc :7119 zprint.zprint/fzprint-map-two-up zprint.cljc :1577 zprint.zprint/fzprint-two-up zprint.cljc :1310 zprint.zprint/fzprint* zprint.cljc :7767 zprint.zprint/fzprint-vec zprint.cljc :6643 zprint.zprint/fzprint-vec* zprint.cljc :6515 zprint.zprint/fzprint-guide zprint.cljc :6358 zprint.zprint/guided-output zprint.cljc :5250 zprint.zprint/fzprint* zprint.cljc :7768 zprint.zprint/fzprint-map zprint.cljc :7166 zprint.zprint/fzprint-map* zprint.cljc :7119 zprint.zprint/fzprint-map-two-up zprint.cljc :1577 zprint.zprint/fzprint-two-up zprint.cljc :1254 zprint.zprint/fzprint* zprint.cljc :7767 zprint.zprint/fzprint-vec zprint.cljc :6643 zprint.zprint/fzprint-vec* zprint.cljc :6515 zprint.zprint/fzprint-guide zprint.cljc :6358 zprint.zprint/guided-output zprint.cljc :5250 zprint.zprint/fzprint* zprint.cljc :7768 zprint.zprint/fzprint-map zprint.cljc :7166 zprint.zprint/fzprint-map* zprint.cljc :7119 zprint.zprint/fzprint-map-two-up zprint.cljc :1560 zprint.zprint/contains-nil? zprint.cljc :487 clojure.core/some core.clj :2709 clojure.core/seq--5467 core.clj :139 clojure.lang.RT.seq RT.java :535 clojure.lang.LazySeq.seq LazySeq.java :51 clojure.lang.LazySeq.sval LazySeq.java :42 clojure.core/map/fn--5935 core.clj :2770 clojure.core/partial/fn--5914 core.clj :2660 clojure.core/apply core.clj :673 zprint.zprint/fzprint-two-up zprint.cljc :1254 zprint.zprint/fzprint* zprint.cljc :7767 zprint.zprint/fzprint-vec zprint.cljc :6643 zprint.zprint/fzprint-vec* zprint.cljc :6515 zprint.zprint/fzprint-guide zprint.cljc :6434 >>> Exception: Unknown values: guide-seq: '' cur-zloc:nil ```

Any idea what’s happening here?

Thanks again!

[kkinnear]

Yes, I think this is an(other) unfortunate oversight on my part. I didn't account for empty vectors and didn't test with them either. I don't know for sure that this is what you are encountering, but when I put in an empty vector I get similar exception output as you are getting. Try this :option-fn:

(czprint i283i {:parse-string? true :vector {:option-fn (fn ([] "vector-lines") ([options len sexpr] (when (not (empty? sexpr)) {:guide (into [] (->> (repeat (count sexpr) :element) (interpose [:newline :newline]) flatten))})))} :style :community :map {:comma? false :force-nl? true} :width 120 :output {:real-le? true}})
{:foo/bar
 [{:foo/bar :activities
   :foo/bar []}

  {:foo/bar :programs
   :foo/bar [{:activities [{:foo/bar :foo/bar.cosmic
                            :start-offset-days 0}

                           {:foo/bar :foo/bar.objectivism
                            :start-offset-days 0}

                           {:foo/bar :foo/bar.nonperforming
                            :start-offset-days 0}

                           {:foo/bar :foo/bar.splenodynia
                            :start-offset-days 0}

                           {:foo/bar :foo/bar.Cashmere
                            :start-offset-days 5}

                           {:foo/bar :foo/bar.undelectably
                            :start-offset-days 5}

                           {:foo/bar :foo/bar.theosopheme
                            :start-offset-days 20}]
              :foo/bar :foo/baz
              :name "bardel"}

             {:activities [{:foo/bar :foo/bar.phytogenetical
                            :start-offset-days 0}

                           {:foo/bar :foo/bar.oriental
                            :start-offset-days 5}]
              :foo/bar :foo/baz
              :name "irresonant"}]}]}

This is the :option-fn:

(fn ([] "vector-lines")
    ([options len sexpr]
     (when (not (empty? sexpr))
       {:guide (into []
                     (->> (repeat (count sexpr) :element)
                          (interpose [:newline :newline])
                          flatten))})))

I don't test these one-off things very well, and usually that doesn't cause any problems. I'm sorry you seem to be running into so many issues that I've missed.

If this doesn't work, maybe you could send me the input that triggers it?

[ghost]

That’s working well for me — thank you!

---

Next question: is it possible to only insert the blank lines when the values in the vectors are multi-line values?

For example, the current config results in this:

{:foo [{:db.record/id :staff/captain
        :teams [{:db.record/idref :teams/phyllopodium}

                {:db.record/idref :teams/shavese}]}

       {:db.record/id :staff/centrale
        :teams [{:db.record/idref :teams/sithens}

                {:db.record/idref :teams/imaginous}]}]}

but I’d prefer this:

{:foo [{:db.record/id :staff/captain
        :teams [{:db.record/idref :teams/phyllopodium}
                {:db.record/idref :teams/shavese}]}

       {:db.record/id :staff/centrale
        :teams [{:db.record/idref :teams/sithens}
                {:db.record/idref :teams/imaginous}]}]}

Thank you!

[kkinnear]

I glad the previous stuff worked. Finally. The "blank line after multi-line elements" is actually a lot harder.

I'm still thinking it over, and I certainly don't see any way to do that with the current code. It is looking moderately challenging even with making code modifications, but I'm still mulling that over. It might not be too bad, but as I'm very late with 1.2.5, it isn't going to be in this upcoming release. I might get it into the next release if it comes together well. I'll put it on the "todo" list for 1.2.6.

One thing I could do now is give you blank lines between the elements of the first vector encountered, and normal line breaks for all embedded vectors. That probably isn't going to solve your problem -- though it would make your example work. I suspect that the output isn't regular enough for that to be interesting, but I thought I'd at least ask. That is pretty straightforward, I think.

Something else that is not hard would be to give you blank lines between elements of a vector which has certain data in it. It isn't hard to look at the data and decide on blanks lines or not -- what is harder is to know how the data is actually going to format in advance of formatting it. That is doable, but not with the code currently in the field.

[kkinnear]

So, I've implemented what you were looking for -- blank lines after things that take more than one line on output. It is a moderately complex series of configuration options to get this. Here is the example:

(czprint i283o {:parse-string? true :vector-fn {:nl-count 2 :indent 1 :nl-separator? true} :vector {:fn-format :list} :map {:force-nl? true}})
{:foo [{:db.record/id :staff/captain,
        :teams [{:db.record/idref :teams/phyllopodium}
                {:db.record/idref :teams/shavese}]}

       {:db.record/id :staff/centrale,
        :teams [{:db.record/idref :teams/sithens}
                {:db.record/idref :teams/imaginous}]}]}

To explain... This says to format vectors like lists, and to use the :list function type (which says to format lists like lists, instead of like things with a function in the first element). It then says to add a blank line after everything that takes more than one line, by using :vector-fn {:nl-separator? true}. If all you want is a blank line, you can leave out the :vector-fn {:nl-count 2}, but if you want more than one blank line, you can have as many newlines as you want by changing the value of :nl-count to be the number of newlines you want after every list element that has more than one line. A blank line is, to be pedantic, 2 newlines. The :map {:force-nl? true} gets you maps that put each pair on a new line, which seems to be what you like, but if not, don't use it.

The good news is that this is, I believe, exactly what you want. The bad news is that it will be in release 1.2.6, which I have just started working on. So, please be patient. I'm writing it up now because it is slightly complex and, having just done it, I have it all reasonably figured out. If I wait a few weeks, I'll have to figure it out to some degree again, so you get this explanation now. I'll update this issue when I release 1.2.6.

[kkinnear]

Another new thing in the upcoming 1.2.6 is that you can now give set-options! a string value for an options map, and set-options will use the Small Clojure Interpreter (sci) to 'compile' it into something safe to execute (in the event that it contains functions). Thus, you will be able to read in the zprint config from your .edn file and then use pr-str to turn it back into a string, and then give it to set-options!, which will correctly handle it. This only works for set-options! at present, but that should solve your issue (not that you didn't already solve it by using eval since you control the file yourself). This too will be coming when I release 1.2.6.

[ghost]

Thanks so much for all the super helpful thoughts and information!

In the meantime until I can try 1.2.6, I think I’d like to give this a try:

Something else that is not hard would be to give you blank lines between elements of a vector which has certain data in it.

Can you point me to a section in the docs that shows/explains how to do this? And/or maybe throw in an example here, if you’ve got one in your fingertips?

Thanks again!

[ghost]

Whoops — never mind, I found the page — I’ll be giving that a try. Thank you!

[kkinnear]

Just released 1.2.6. It has all of the capabilities I mentioned above in it. Sorry it took so long. There was a lot to do after I implemented the features you requested.