Translating CIDER/clojure-mode indent specs to zprint config

[vemv]

clojure-mode and CIDER have the notion of "indent specs" by which macros can declare a specific indentation. Some valid values under its spec are 0, 1, :defn, or more complex values like [2 nil nil [:defn]] (n.b I almost never use such complex values)

Quick examples: https://github.com/clojure-emacs/clojure-mode/blob/c339353f9e649b3af084f1bb6ce759e614a2f243/clojure-mode.el#L1651

More info: https://docs.cider.mx/cider/indent_spec.html

I wonder if zprint has a similar notion of mapping macro names to a specific style, and if you've ever faced the problem of translating one syntax into the other?

(This is not necessarily a feature request or such - I'm simply trying to get a full picture)

---

...With this information I could implement a zprint integration for formatting-stack. Currently it already translates "cider indent spec" to "cljfmt indent". This automatic translation is pretty essential for us since we really like :style/indent metadata attached to .clj defmacros.

That way one defines intent once, agnostically, and gets similar formatting with a number of tools (clojure-mode, cider, cljfmt, now zprint)

Cheers - V

[kkinnear]

This is a complex question, with many ramifications. To start off, you have referenced "macros", above, but my thinking is that you are referring to anything that shows up right after a left parenthesis. Sometimes these are things defined by defmacro, and sometimes by defn. I tend to refer to those things as "functions", which also includes those things which are actually defined by defmacro. Not to be pedantic, but just to confirm we are really talking about the same things.

That said, while I have seen the CIDER "indent specs" here and there, as I am not a CIDER user I haven't spent time investigating them, nor have I had many (any?) requests about zprint interpreting them over the years. That doesn't mean that I'm not willing to explore the subject now.

Certainly zprint classifies (what I call) functions, using the :fn-map in the options map. Basically this is a map of function names (as strings) to function types. The function types are documented here. Over time, the :fn-map has been enhanced to accept not only a function type, but also a function type associated with an options map enclosed in a vector. You can see the default :fn-map of zprint if you have it in a REPL by executing (czprint nil :explain) which will output the current options map, which isn't small. It should be alphabetized, and the :fn-map is there. Here is a bit of it:

    ":require" :force-nl-body,
    "=" :hang,
    "alt" :pair-fn,
    "and" :hang,
    "apply" :arg1,
    "are" [:guided {:style :areguide}],
    "as->" :arg2,
    "assert-args" :pair-fn,
    "assoc" :arg1-pair,
    "assoc-in" :arg1,
    "binding" :binding,
    "case" :arg1-pair-body,
    "cat" :force-nl,
    "catch" :arg2,
    "comment" :flow-body,
    "cond" :pair-fn,
    "cond->" :arg1-pair-body,
    "cond-let" :pair-fn,
    "condp" :arg2-pair,
    "def" :arg1-body,
    "defc" :arg1-mixin,
    "defcc" :arg1-mixin,
    "defcs" :arg1-mixin,
    "defexpect" [:arg1-body
                 {:list {:respect-nl? true},
                  :map {:respect-nl? true},
                  :next-inner-restore [[:list :respect-nl?] [:map :respect-nl?]
                                       [:vector :respect-nl?]
                                       [:set :respect-nl?]],
                  :set {:respect-nl? true},
                  :vector {:respect-nl? true}}],
    "defmacro" :arg1-body,
    "defmethod" :arg2,
    "defmulti" :arg1-body,

You can see that for the most part, the :fn-map simply maps function names to function types. However, in some cases more complex operations happen. Note that are uses a "guide", which is specific code for are that justifies some of the elements. Note also that I have defined some complex capabilities for defexpect, as I use expectations for my more than 1000 tests, and this formats my testing files readably.

There are two issues with CIDER "indent specs" about which I'm not clear and which have a large effect on how they might fit with zprint:

1. It isn't clear to me whether CIDER (using these "indent specs") will change what line something is on. Will it pull things together onto the same line that are on two lines? Will it move something from the line it is on to a subsequent line if it doesn't "fit"? Is there even a concept of "fit", i.e., is there a width that the code is supposed to not go beyond? Classic zprint will place things on the lines it thinks makes sense, regardless of where the source started out. There are two (well, three) additional styles for zprint regarding lines: :indent-only which will never change lines (and which ignores function types because of that), :respect-nl which keeps all of the newlines in the source, but might add some if things get wider than allowed by the :width, and :respect-bl which is "respect blank lines", where any existing blank lines are kept but otherwise newlines are ignored in favor what what zprint think makes sense.
2. It looks to me like the "indent specs" are held as metadata on the name of the function. Which is certainly convenient and makes a lot of sense, but not something that zprint usually has access to when formatting source. On the other hand, you say that you have translated CIDER "indent specs" to cljfmt "indents", and zprint and cljfmt share a common source parser, so how did you do that?

There are several ways that CIDER "indent specs" could be mapped to zprint function types.

1. One of the things that can be in an options map associated with a function type (or in the options map for every list encountered) is {:list {:option-fn (fn [opts n exprs] ...)}}. The option-fn is called with the elements of the list, and it can return any options map it pleases (or nil). The returned options map can specify a function type. So if the option-fn could determine the "indent spec" it could access a mapping (or execute arbitrary code to determine a mapping) from "indent spec" to "zprint function type" and then return that.
2. I could write a "guide" which would accept the CIDER "indent" spec (again, assuming zprint could get access to it), and essentially "do it" in the context of zprint. I invented "guides" in no small part to implement the "rules of defn". This would not actually be translating "indent specs" to zprint function types, but rather just "doing it" sort of by hand by creating a guide based on the "indent spec" and the actual thing being formatted. That might actually be easier than trying to translate it (as in #1).

There is plenty of plumbing in zprint to get it to do different things. I could always add more.

To finish this up for now, I am interested in exploring how to enhance zprint to use CIDER "indent specs" to format functions. Whether that is translation or just outright implementation, I don't know. Given zprint could get access to the information, I think that there is probably a pretty good chance that I could figure out how to get zprint do most if not all of the "indent specs", though until I got into it I wouldn't know for sure. The chances of success would certainly be increased if you would be willing to answer the occasional question and try out various alpha releases I could make available to you.

[vemv]

To start off, you have referenced "macros", above, but my thinking is that you are referring to anything that shows up right after a left parenthesis

They can be functions, but it is not customary in clojure-mode/cider to define custom indentation rules for defns. In the end, anything that is not a macro does not define syntax, and therefore generally is not expected to alter a conventional, homogeneous style.

It isn't clear to me whether CIDER (using these "indent specs") will change what line something is on.

No, never. clojure-mode is only concerned with indentation.

However, if a given argument was placed in a non-customary position, it will be indented differently. This informs the user that his choice was odd.

You can see in red the non-customary user choice, and in green the customary user choice for if:

-(if
-    1
-  2
-  2)
+(if 1
+  2
+  3)

In other words, for a macro like if which indent spec is 1, it is expected that the 1 first arguments will be in the same line as if, otherwise those misplaced arguments get that extra indentation.

n.b., precisely because this indentation makes odd choices ugly, people will rarely choose those. So for simplicity, zprint could, instead of generating this "double indentation", ensure that all 1 first arguments are in the same line as if.

So, for the question It isn't clear to me whether CIDER (using these "indent specs") will change what line something is on, my answer would be "it doesn't, but it'd welcome if zprint did it"

Will it move something from the line it is on to a subsequent line if it doesn't "fit"? Is there even a concept of "fit", i.e., is there a width that the code is supposed to not go beyond?

(no)

It looks to me like the "indent specs" are held as metadata on the name of the function. Which is certainly convenient and makes a lot of sense, but not something that zprint usually has access to when formatting source. On the other hand, you say that you have translated CIDER "indent specs" to cljfmt "indents", and zprint and cljfmt share a common source parser, so how did you do that?

I invoke cljfmt from the same JVM that runs my main code. So I can easily translate var metadata to cljfmt config at runtime. I envision doing the same for zprint.

The project that does that is called formatting-stack. Another project that invokes cljfmt over the user's jvm is cider-nrepl (see its format ns).

There is plenty of plumbing in zprint to get it to do different things. I could always add more.

Appreciated! From my side I don't have a specific rush. It certainly will be useful medium/long-term and could easily mean that the mentioned projects would feature a zprint integration.

But my priority short-term would be e.g. https://github.com/kkinnear/zprint/issues/223 and other possible follow-up :rod issues so I wouldn't like to distract you or ask more than I need.

Cheers - V

[kkinnear]

Thank you for both answering my questions, and clarifying what you care most about. I really appreciate it.

223 is straightforward, and will be in 1.2.3 when it comes out. I have a plan for #226, but I haven't started on it yet as I'm still working on another issue. Once I finish #226 I'm expecting to generate the next release, 1.2.3.

I also really appreciate the test you created for #226. It will make testing that aspect of the fix much easier.