Investigate creating a mechanism for managing module dependencies

[bhardesty]

In the Modular Documentation Reference guide, a module is defined as "independent" and something that "does not depend on other documents" (https://redhat-documentation.github.io/modular-docs/#modular-docs-terms-definitions). In addition, the guidelines we've been given for the Module Library indicate that modules cannot have any external dependencies.

I think this guidance is a bit misleading and unrealistic. Rather than guiding writers to avoid all external dependencies, I think we need to realistically consider all of the external dependencies that naturally exist, and then investigate the possibility of enabling writers to enable/manage them in a way that does not threaten the independence and reusability of the module.

Consider the following potential dependencies:

• Attributes - Modules use attributes that are typically defined in an externally-referenced attributes file. In many cases (such as working upstream first), it is not possible to include in the module the reference to the attributes file. Even if it is possible, it's still an external dependency on a document outside of the module.
• Images - Any module that includes a diagram, screenshot, or any other type of image has an external dependency on that file.
• Includes - While modules should not include other modules, and while fragments (aka, snippets) are best avoided whenever possible, they are nonetheless sometimes necessary. When working upstream-first, it is sometimes necessary to use snippets to include upstream-only content that, when brought downstream, can be replaced with a downstream-only snippet containing the downstream-only text. Without using snippets in these cases, the modules would not be reusable between the upstream and downstream. (Note also that any prohibition on using includes in modules also, by definition, prohibits us from using images in modules.)
• Links - for cross-references, using links instead of xrefs still introduces an implicit external dependency. AsciiDoc link syntax does not provide a mechanism for transcluding the title of the module being cross-referenced, which means we must manually enter the title of the module that is the link target. This adds a dependency on the target module's title; if that title should ever change, the link text in the cross-reference will be incorrect.

I marked this issue with the "Discussion" label in order to start the discussion. Perhaps a configuration file could be implemented to manage all of the above dependencies externally from the modules. Or maybe we would need a different type of mechanism altogether. Or perhaps I've misread things, and some of the above dependencies are not issues at all. In any case, I'd like to learn which dependencies we actually have in our modules, and how we might address them.

[adahms]

@bhardesty - Great discussion!

It's true that while modules are intended to be completely independent by nature, they still need a few inputs and resources to build correctly.

We're coming up to a milestone on the tooling side where we'll be starting to look at how this will work, so this is a great time for us to figure out what we would like to see. The system can probably grab any images a module includes and take care of that for us, but we recognize that a more elegant solution might be needed to supply the values of attributes.

The question about includes and links introduces a problem of a different kind, but something I am hoping we can nonetheless address in our tooling to some degree. For example, we know that assemblies will need to include modules from other products using some kind of special link, in which case, can these be resolved to either an xref if the module exists in the assembly, or a link if the module has been independently published?

Curious to get your thoughts.

[VladimirSlavik]

Asciidoctor can resolve any dependencies that are realized as files (includes, images), so a well made system should be able to just do the same: No need to enumerate these manually again - you already did so in the markup.

For links, any hypothetical future xref-like mechanism (give target id, get a link with text) would be machine-resolved same as the files, thus hopefully again nothing to explicitly state. I also imagine it would be easier to use than bare url links (just like normal xref), so that would rule these out.

As to input attributes. Right now, in my mind a module is defined as a "file". That suggests to me two paths:

a) If a module will have the file and some sort of entity in a database, then that database could be a place to have input attribute definitions? One can also imagine having some configuration (text) file to import the module files, so that's another place to have these definitions.

b) Stay strictly with the module = file model and have the default values for inputs defined in some extension of the format. Eg. a line that says // default :productname: somevalue

[adahms]

Thanks for the input @VladimirSlavik!

I like the idea of defining modules as individual files (which they are) and resolving dependencies based on that definition.

You make some good points about images and other resources, and your point about defining module attributes at the file level rather than later on gets me thinking as well. Earlier designs of the module library suggested users might need to supply attributes when they use a module, but maybe modules should have default values defined and resolvable in the files themselves.

[jboxman]

I haven't much time to follow these discussions in any depth, but it's worth mentioning that Asciidoctor has an extension mechanism, so it's possible define custom directives. A few examples:

• pageRef handler
• include handler
• block that renders live JavaScript

[jboxman]

Also this for the actual Asciidoctor extension instructions.

[bobjohnsrh]

I agree that blanket guidance against external dependencies is unrealistic. Blanket guidance against dependencies is overly broad and elides the difference between content dependencies (dependencies that are themselves part of the content, or are essential to building the content deliverable), and remote dependencies (such as links from one module to another, or links to third-party content). As a content author, attributes, graphics and transcluded content are part of the content. Graphics in particular appear to be external dependencies, especially from the tool and publishing perspective, but that is a side-effect of our text-based source. As author, graphics are as much part of the content as the words.

The root of this problem is that the guidance conflates the module as source and the module as deliverable. They are not, in fact, the same thing. The source has to go through a transformation to become the deliverable. As part of that process content dependencies (such as external content transcluded into the published result) are normalized into the content. Normalized content should not be considered an external dependency. Not that all content can be normalized; graphics included in content delivered in a text-based presentation format such as HTML cannot be normalized, and must remain a dependency.

The deliverable module is a different matter. In the deliverable, published object, all content dependencies have been resolved and normalized, resulting in a complete unit. But any remote dependencies, usually links to remote content, cannot, by definition be normalized. These links will remain as links, with the resultant risk of breakage. Minimizing these dependencies is appropriate.

I would recommend guidance and tooling support that allows dependencies that are ultimately normalized to create the final deliverable, while restricting dependencies on remote resources that may result in broken links.

[adahms]

@bhardesty @VladimirSlavik @jboxman @bobjohnsrh - Thanks for the ongoing discussion on this issue.

Some great points have been raised here, but I am starting to feel like this is something we can handle as part of the planning and testing process for the system that will manage and render modules and assemblies.

Unless there is any guidance we can add to the reference guide itself, I might close this issue here and help take up the discussion and keep you involved on the system side instead.

Is there anything you'd like to see added to our guidance in its current state?

[bhardesty]

@adahms Agreed: I don't have anything that needs to be added to the current reference guide.

[VladimirSlavik]

I don't think I have anything, either. Current practice and advice regarding dependencies is sane as far as I can tell, namely discouraging xrefs and not much prescribed beyond that. That looks good to me, given the overall state of things. I mean, for this, dialog with or input from tooling would help more than us writers speculating on what they could give us...

[bobjohnsrh]

Andrew,

Can you explain how planning and testing process will address content dependencies versus remote dependencies? And how will the process address the difference between module as source and module as deliverable? I still think these are fundamental questions that need to be addressed. If we do not address them, we will likely create an overly-constrained system that will be difficult for authors to work with.

[VladimirSlavik]

I'll just second the concern about overly constrained systems.

[adahms]

Thanks for the comments, all, and for the question @bobjohnsrh!

@bobjohnsrh - I completely agree with and really like the distinction you've called out regarding modules as source and modules as deliverables, and you're right that there are two types of dependencies that need to be looked at from their own standpoint.

My argument with regards to this issue here in GitHub is that unless it results in some change being made to the guidance documentation, we should close it and find the best place to raise these concerns and considerations so that they can be taken into account as part of the planning for the system itself.

In that regard, questions around attributes, images, and includes are all valid and good use cases for the system to handle, and there has been and will be more discussion around this. If it's something close to heart, I can send out information about them and let you all know where to provide input and feedback.

Includes and links as part of assemblies are a trickier thing, but there could be a use case here for the system to be aware of modules included in a document and to resolve links to their actual, published link, for example. That wouldn't solve all our concerns - some links will point to external resources we don't control, and such an approach might simply not be possible - but it could go a ways to addressing it.

Long story short, my question here is - can we close this issue in its potential role as a tracker for changes to the reference guide and involve you all in the planning of these use cases, or is there anything we can add to the reference guide as an outcome of this discussion?

[emmurphy1]

I'm closing this issue for the reasons outlined by @adahms.

[emmurphy1]

Or I would if I had permission :-) @adahms would you please close this issue?