Question: How can I share config / docs between Solargraph and Yard ?

[Fire-]

As the title states, I'd like to share configuration, extensions, cache, and document-store between Solargraph (and its internal YARD) as it's used in the VSCode extension, and Yard as it's run from the command line.

The goals are effectively to avoid some duplication of doc generation, let YARD CLI understand Solargraph's custom tags , and have Solargraph utilize any YARD extensions I currently have set up -- something it does not appear to do currently with designations inside .yardopts in a project's root

I can get part of the way there by adding a require "yard-solargraph" to my .yardopts file to help YARD understand tags, but that's about it.

[castwide]

You can set the cache directory with the SOLARGRAPH_CACHE environment variable. I just tested by setting it to a network share and was able to access it from both a Windows machine and a Linux machine.

There's currently no way to define a shared configuration file, but we could probably add support for it. I'm open to suggestions on how to implement it.

YARD extensions modify yardocs at generation time, so the modifications they add to gem documentation will be visible when Solargraph loads them. Unfortunately, Solargraph has limited ability to utilize them on the current workspace at runtime. Adding yard-solargraph to .yardopts is useful for suppressing warnings about unknown tags while yardoc is running, but has little to no effect on the documentation itself.

[jspawar]

@castwide apologies, but would you mind clarifying your response a bit more please? Is there no support then for Solargraph to load YARD plugins via a .yardopts file or such at all? Not fully clear to me if there is or not and if there is support, is there anything I need to do to enable it beyond having a .yardopts in my project root?

Unfortunately, Solargraph has limited ability to utilize them on the current workspace at runtime.

Also, assuming that YARD plugins are supported, what exactly do you mean by this? Say I had a plugin which injected new methods or classes into the YARD docs, will Solargraph not honor them?

[castwide]

YARD extensions modify the documentation they generate, but Solargraph generally can't make use of them in an active workspace. If you reference a gem that used a YARD extension to modify its documentation, the modifications should be available in a workspace that includes that gem as a dependency. If you're working on a project that includes YARD extensions that would modify the generated documentation when published, those changes probably aren't visible to Solargraph yet.

In short, the changes introduced by the YARD extension are visible to Solargraph if they're a gem dependency in your current workspace, but not if your current workspace uses the YARD extension to generate those changes to its own documentation. It's an unfortunate limitation, but it's not easy to fix.

Edit: Also, just to clarify, all of that means that extensions defined in a .yardopts file are generally ignored by Solargraph. I've worked on support for YARD extensions, but it's a decidedly non-trivial problem.

[grncdr]

I've worked on support for YARD extensions, but it's a decidedly non-trivial problem.

I'm interested in this problem 😉

Could you tell me more about what the main issues are? Like the previous commenter, I'm interested in having dynamic attributes/methods added by YARD handlers available to Solargraph.

My approach would be "use the workspace .yardopts when generating YARD map for the workspace". If you can warn me about what issues I'm going to encounter I'd appreciate it.

[grncdr]

Ok, so I've looked into this a bit today and it seems I might have fundamentally misunderstood how Solargraph and YARD relate to each other. 😬

What I thought then: Solargraph is using the YARD database to understand external libs and workspace source files.

What I think now: Solargraph only uses YARD for Gems. Source files in the workspace itself are never parsed into YARD code objects, only Solargraphs internal Solargraph::Source representation (which does use YARD for parsing comments).

If I have that right, it might mean that trying to support YARD extensions in Solargraph is harder than simply duplicating them as Solargraph extensions. In the interest of sharing knowledge and maybe getting some feedback, I think it could work if Solargraph:

1. also parsed source files with YARD.
2. Queried the YARD registry for matching code-objects when mapping "namespace" types (Module/Class) from source AST -> pins.
3. Added pins for any dynamic members of the namespace.

Unfortunately, my impression is that YARD is pretty "batch-oriented" when it comes to processing documentation. It relies on a global singleton Registry that expects to read/write from disk with pessimistic (global) locks. Not a very good fit for an interactive live tool. Not to mention the overhead of parsing every file twice.

---

An alternative approach I'm considering is to write a small abstraction layer that can provide both a YARD code object handler and a Solargraph plugin/extension. For that I think plugins (#307) need a way to hook into the creation of a Solargraph::SourceMap to create pins for dynamically defined methods. I haven't read enough of the source to figure out if that's already possible.

[castwide]

@grncdr Your conclusions about the process and the reasoning behind it are largely correct. Solargraph uses its own mechanisms to parse and analyze the active workspace, and uses standard YARD tags to augment its data. It only uses yardocs for external gems. Earlier versions of Solargraph used yardocs for the active workspace, but 1) it was too slow and resource-intensive; 2) it didn't provide information about variables; and 3) it didn't allow for type inference.

You might be able to use conventions to implement a plugin. See lib/solargraph/convention for the base class and a few examples. Unfortuntately, I'm not sure it's possible to implement a single plugin that can reliably handle arbitrary YARD extensions. Each extension might require its own Solargraph implementation. If you decide to give it a try, I'll provide whatever help I can.

[grncdr]

Thanks for the confirmation @castwide. It's especially helpful to know that YARD was too resource intensive.

Unfortuntately, I'm not sure it's possible to implement a single plugin that can reliably handle arbitrary YARD extensions. Each extension might require its own Solargraph implementation.

This seems very likely to me, and is not a path I want to take. I suspect that the 80% use case (and certainly my use case) for YARD extensions is "I have a custom DSL that defines methods dynamically". I will see if I can accomplish something with conventions and open/comment on more relevant issues as needed.