Enhancement - allow include to point to Git URL for mkdocs.yml & docs in other repos

[ryandaugherty]

Oooh wouldn't this be so cool... :)

So you mention the ability to use git submodules to include the mkdocs.yml + docs in other repositories. If these sub-repos (let's call them) are standalone mkdocs directories without any developer code, this works.

However, let's say you have a bunch of dev code in a sub-repo, but you also want to have the documentation live there in a 'docs' folder. And then you have several of these sub-repos with separate docs folders. This sort of distributed system is great because the docs live alongside the code (docs-as-code), and developers can make changes to their code and documentation with a single commit, but still have a central mkdocs build for ALL the documentation.

The problem with git submodules is they inherently bring over the entire repository, not a single directory. In the scenario above using submodules, all the code would be brought over alongside any documentation. This is quite a bit of bloat, and you would have to then solve for excluding all the code from your docs build. Submodules are also a bit messy anyways, particularly for tech writers or folks not familiar.

What would be ideal is to have separate "docs" directories in each sub-repo, and only bring over that specific folder into the primary docs build at the time of the build. I'm currently using a Git overlay tool called Gilt in the mkdocs build to solve for this very problem.

It would be nice to have this functionality included in your tool - allow the mkdocs.yml include function point to a Git URL of a mkdocs.yml + docs in another repo (making sure to point to the right folder and branch). Then everything can be brought together for the primary mkdocs build.

I betcha this would help your own documentation flow too ;)

[LeonardAukea]

@ryandaugherty Thanks for making me aware of Gilt. I was currently looking for this specific functionality. And I agree with you that it would make a great addition to the tool.

[jdoiro3]

I just started a plugin to try and implement this feature. more here.

[kirannhegde]

Hello @ryandaugherty We are dealing with the documentation for the source code here. Our source code is split into different repositories. Does this plugin support searching in the source code of different git repositories?

The documentation states that it is possible to provide the url to another git repository. However, it isn't clear as to where the url needs to be specified. Any assistance would be very much appreciated.

I have also described my problem in detail at: https://github.com/mkdocs/mkdocs/discussions/2766

Regards, Kiran Hegde

[ryandaugherty]

The only real way to solve this right now (I believe) is to have a build script that grabs docs folders from the various repos and assembles them. I think I just used a simple make file for the build, and in that file I assembled the docs folders from the different repos using gilt, and then used monorepo to point to those different directories. I might be able to find details if you wanted.

[kirannhegde]

Hello @ryandaugherty Thank you very much for responding back. Sure, it would be great if you could find the details. When you state that you grab docs folders from other repositories and then use monorepo to point to those different repositories, are you implying that these docs folders were already processed in their respective repositories using the mkdocs build command?

We make use of a plugin called mkdocstrings to generate documentation for the source code. This plugin loads the different Python libraries mentioned in the source code during the documentation generation process. As a result of this, there are Python dependency resolution issues that we encounter when using the monrepo plugin in a root repository . If the mkdocs.yaml file in the root repository references multiple mkdocs.yaml files from different repositories, we run into dependency resolution errors as there are conflicting dependency requirements for the various libraries. To overcome this, we were thinking that the docs folder in each of the different repositories should be processed independently and the output of the mkdocs build command should be copied to some place in the root repository and then have mkdocs somehow assemble the documentation. Is this what you ended up doing?

Kindly let me know.

Thanks, Kiran Hegde

[ryandaugherty]

No, the order I did it in was grab other repos, bring into the central one, then mkdocs build on all files using monorepo. I’m not sure how you’d accomplish what are trying to do, and I’m not sure monorepo is intended to solve it. There might be a way of building in the separate repos, gathering the html output together, and somehow taking the output of each search indexer and concatenating it together to have a unified search. Not sure. You might need a platform engineer to help.

[ryandaugherty]

https://github.com/olivernn/lunr.js/issues/76

[kirannhegde]

Hello @ryandaugherty So i have been trying to accomplish what you have tried. I gathered the output of different mkdocs builds. Each of those outputs had the individual search indexes created in the search folder. I then put all of the builds into a centralized folder, and in my main repo, i had a mkdocs.yml file which includes the monorepo plugin. This main repo has its own mkdocs.yaml which references the other child repository folders. I then ran mkdocs --serve on this main repo. mkdocs seems to create a folder under /tmp/mkdocs, from where it serves the files. Unfortunately, when i look at the search index under the search at /tmp/mkdocs, i dont see references to the other folders being included.

What am i missing here? Do you have a reference that i could take a look at? Something which shows how you did this. Ideally, this should work but is not.

Thanks, Kiran Hegde

[jamesone]

Is there any work being done on this? I think it's a feature that many orgs would want to use.

[jamesone]

@ryandaugherty how did you approach this? Were you able to implement a solution for it that works with Backstage?