Clojure support?

[cap10morgan]

Based on my reading of the documentation, it sounds like you all have to do some work to support a new language before an extension can any new capabilities to it. Is this the right place to let you know that I'd like to try writing an LSP extension for Clojure? Let me know if there is anything else I can do to help move that forward. Thanks!

[mattmassicotte]

Hello! Yeah, you are right. You can make extensions outside of the Chime main application and they will work.

However, there are a few bits that aren't possible externally. The biggest task is getting tree-sitter support working. I have done this approximately 1 bazillion times, so I can take care of that here:

https://github.com/sogaiu/tree-sitter-clojure

I'll also update the app internally to recognize clojure file types. One thing that would be helpful is having someone more familiar with Clojure define the file extensions. I found .clj, .cljs, .cljc, .edn. I think it's better if these be defined as semantically as possible via appropriate UTIs. Clojure is already listed in the UTI map here, but more may be needed.

https://github.com/ChimeHQ/ChimeKit/blob/6e80898da48c7d4c4fb78bbad7700c7065b9e241/Sources/ChimeExtensionInterface/UTType%2BLanguages.swift#L11

To correctly interface with LSP, Clojure needs to be added here: https://github.com/ChimeHQ/ChimeKit/blob/main/Sources/ChimeLSPAdapter/DocumentContext%2BLSP.swift

I also took the liberty of creating an initial repo. However, I'm not trying to assert ownership. If you are excited about moving this forward a) that would be awesome and b) you can get commit permissions. Zero pressure though.

https://github.com/ChimeHQ/chime-clojure

[cap10morgan]

Cool, thanks! Yeah I would love to give it a shot. Having a native Mac Clojure editor would be 🙌🏻!

That looks like the right file name extension list. There was an older, third-party cross-platform format that used .cljx but I doubt that sees much use anymore as .cljc replaces it has been built-in to the language for many years now.

I'll start exploring and see where I get.

[cap10morgan]

Added Clojure languageId: https://github.com/ChimeHQ/ChimeKit/pull/3

[mattmassicotte]

Moving fast, I love it!

I've finished up and tested the tree-sitter changes. Official PR is here, but I do not need it to be merged to ship the changes.

Unfortunately, I just released a beta of 2.2.1 today. I'd like to give it a little longer. However, the next big task is to investigate the language server options for Clojure. Any ideas?

Having a native Mac Clojure editor would be 🙌🏻!

I'm just curious, have you looked into any others? I'd very surprised if Nova didn't support it.

[cap10morgan]

However, the next big task is to investigate the language server options for Clojure. Any ideas?

Yep! https://clojure-lsp.io

I'm just curious, have you looked into any others? I'd very surprised if Nova didn't support it.

Haha, yeah. This is me: https://extensions.panic.com/extensions/com.timetraveltoaster/com.timetraveltoaster.clojure/

...but I couldn't get it to do the things I wanted it to and eventually gave up. I wasn't super motivated to keep working on it since Nova is closed source. It's a nice editor and I generally love Panic's software, but hitting the closed source boundary where I couldn't implement features and/or fixes was frustrating.

[mattmassicotte]

Ok so the maintainers for the Clojure tree-sitter repo aren't inclined to add Swift bindings. That's a shame, but I both understand and respect their reasoning. I'll just maintain my fork for now.

Yep! https://clojure-lsp.io

Ok great! I've filled in a skeleton for it in the extension repo. It could be sufficient, but does not handle any server-specific configuration/quirks. This may or may not even be necessary.

Next week, approximately, I'll do another release of Chime that includes the syntax support and a bundled version of that extension.

[cap10morgan]

OK, sounds good. Is there anything else I can work on in the meantime, or should I wait until the release?

[mattmassicotte]

You are more or less blocked until I make those core changes available. However, it would be useful to make sure we understand clojure-lsp's functionality well enough so that it can be started and run easily.

I'd also like know what features you are most interested in. You know, to make sure that Chime can actually do what you want 😬

[mattmassicotte]

2.2.2 beta is now available and includes the minimal set of changes needed for Clojure syntax support. Next up is to test the extension.

[cap10morgan]

I'd also like know what features you are most interested in. You know, to make sure that Chime can actually do what you want 😬

I typically use IntelliJ with a non-free (in either sense) plugin called Cursive that gives me static analysis (which I mostly use for finding usages of symbols, going to definitions, auto-importing namespaces, etc.), automatic code formatting, and a Lisp-specific feature called parinfer which sets your parens' (and other paired brackets') nesting level automatically based on the indentation of your code. It's basically a requirement to use either this or an older, more explicit approach called paredit to hack on Lisp for Real Work™.

So I think I can get ~80-90% of what I want from the LSP support alone. The stretch goal will be implementing parinfer. There are libraries out there that do it, but I haven't done too much with them yet. It usually seems to come down to how granular an editor's plugin hooks are since it needs to move text around, override keystrokes (e.g. it doesn't let you backspace away a closing paren/bracket, you have to delete its opening pair), etc. But we can cross that bridge when we come to it.

2.2.2 beta is now available and includes the minimal set of changes needed for Clojure syntax support. Next up is to test the extension.

Cool! I'll grab it and start exploring!

[mattmassicotte]

So, Chime's current level of LSP support can do some of what you've listed. Auto-import could possibly need tweaking depends on how it works, as that is a little funny in the LSP. Jump to definition is supported, as is formatting.

Finding symbol usage is not implemented. That would require ChimeKit work (API, LSP machinery) and then integrating that into the UI. Without knowing anything about it, my guess is the ChimeKit side wouldn't be terribly involved. However, given the app is in this half open-source state it isn't an ideal time to make significant UI changes.

Chime's extension interface is strictly asynchronous, because extensions run out-of-process. The current design does not allow anything that involves text mutation during an edit to be done by an extension to prevent blocking the UI. However! Chime's live typing system is actually an independent project, called TextFormation. And it may support everything you'd need to do. And if not, we can expand it! It's also used by a few other macOS editors, so the additional capabilities would be welcome I'm sure.

[cap10morgan]

I'm seeing some basic syntax highlighting in 2.2.2 in a .clj file. Cool!

Is there a way for me to add the Clojure extension and start kicking the tires w/ clojure-lsp, etc.? Or do you need to do that?

[mattmassicotte]

Great! I'm not familiar enough with Clojure to evaluate the quality of the highlighting. But, typically, the included tree-sitter highlights.scm file is poorly maintained. So that could potentially something that needs to be looked at if you notice highlighting that isn't present but should be.

I will typically bundle extensions directly inside the app for easier use. But, it can be done by you, it's just a bit involved. The Swift extension includes the needed component: a hosting application. This app can do nothing, but ExtensionKit will only make extension discoverable when published this way.

[cap10morgan]

Actually the syntax highlighting might be close to what tree-sitter can do w/o LSP info too. The only glaring omission I see is highlighting the first symbol of an unquoted list (i.e. the symbol in function call position), but I'm not sure if tree-sitter's highlighting can do that or not. I personally also really like rainbow parens when working with Lisps, but I'm not sure if the tree-sitter syntax highlighter is the right place to implement that either. Might be better to base such things off of LSP metadata? And rainbow parens are useful in lots of languages, so maybe better as its own extension or built-in (optional) feature?

Anyway, let me know if there's something more I can be doing right now. I'm happy to try building the app w/ the Clojure extension bundled if that seems like a good next step for me (probably something I'll want to get working anyway I guess).

[cap10morgan]

Yeah, looks like the author of that tree-sitter-clojure implementation thinks additional features should come from tools that understand Clojure at a higher level than a static language grammar: https://github.com/sogaiu/tree-sitter-clojure/blob/master/doc/scope.md

[mattmassicotte]

Great to hear about the highlighting. Tree-sitter's query system probably can target that leading element, but I'd have to look more deeply at how the grammar is set up to be sure.

Chime supports LSP's semantic tokens, so that should work as well.

I think the next thing to do here is build that container app project in the clojure extension repo. At this point, I should probably make a template...

[mattmassicotte]

I've added a hosting app project to the clojure repo, and included instructions for building and running. This should be sufficient to experiment with the plug-in!

[cap10morgan]

@mattmassicotte Sorry this fell off my radar for awhile. I tried building the chime-clojure project in Xcode following the directions in the README but got a "No such module 'chime_clojure'" error under "ChimeClojureTests". Any pointers on what might be going on?

[mattmassicotte]

Never need to apologize!

Ok so there actually were a few problems. The main package itself was a little borked, and that's what was causing this issue you saw. That's now fixed.

But, you actually need to open the embedded Xcode project, not the top-level package. Instructions were not clear at all about that. Also fixed.

However, the code-signing information, which is unfortunately necessary, wasn't correctly being applied. I'm pretty sure that is fixed as well.

Whew. At this point, it now should actually be possible to build/run. But, I wouldn't be stunned to see some other issues crop up. Please reach out again if there are problems!

[cap10morgan]

OK it builds now! Thanks! But it doesn't show up as an extension in Chime. Is there something I need to do to make Chime aware that it exists?

[mattmassicotte]

Ah. You must run the "Extension Container" application target at least one time to register it. Is that what you did?

[cap10morgan]

Hmm... I guess not. I'm not sure I know how to do that. Product -> Run is greyed out. Does that mean I need to configure something else? I setup my Apple ID and told it to "Sign to Run Locally" but I'm not sure if there's something else that's missing.

[mattmassicotte]

Do you have "Extension Container" selected here?

[cap10morgan]

I only have these options in that menu:

• chime-swift
• Clojure
• Semaphore

[cap10morgan]

Oh, I figured that out. I had opened the folder when I needed to open the .xcodeproj "file"

[cap10morgan]

OK I got it to show up and enabled it! How do I monitor / debug what it's doing? I want to see what it's sending and getting back from clojure-lsp, for example.

[mattmassicotte]

Ok great question. I'm going to try to get better about documenting this stuff, so it is more approachable for the future. I will write up a document, and then link it from here.

Also, I wanted to share with you a link to a discord server, just in case that is your kind of thing. It is a great place for live help, but I'm also completely fine continuing here. https://discord.gg/mnqa6tC8

[mattmassicotte]

Here it is: https://github.com/ChimeHQ/ChimeKit/blob/main/Sources/ChimeKit/Documentation.docc/Building.md

Eventually, this will also be published here, but it takes a little while for that to get picked up.