Use Git for Content Storage

[tuaris]

Desired Feature:

Users who prefer a "git-based editorial gate" will dismiss Bookstack as an option. They view the workflow offered by tools like Gitbook/Grav to be more in-line with CI/CD practices that some companies operate on.

For Bookstack to capture this audience, Git should be an option to use as a content storage backend instead of the database. The closest existing implementation of this I can find is wiki.js.

Expected Behavior

For users with the admin role, creating a new "Book" should present the option to select the backend storage type. Either "SQL Database" or "Remote Git Repository". This can not be changed once selected.

Selecting "Remote Git Repository" will further prompt the admin user to enter information on how to access the Git repository such as the repo URL and the name of the master branch. There will be an optional field set to enter a user name and password or a drop down field to select an SSH key.

The SSH keys are managed by one of the Bookstack admins, I won't go into detail but ideally there will be an admin UI to allow either the upload of a key, or generation of one.

When saved, Bookstack should perform a git clone of the repository to local temporary file storage. This feature request is only asking for the ability to render Markdown from an existing Git repository. A separate feature request was filed (#777) to support editing of content when using a Git repository backend.

This feature request is dependent on the implementation of #458 on a book level. Bookstack should define a content/file/directory structure on how the files should be organized. Something compatible to Read The Docs or Gitbook.

A git pull on the repository can be triggered from the admin UI manually or by a webhook/API call.

[ssddanbrown]

Hello again @tuaris, Thank you for your suggestion.

My initial thoughts are that as a developer, and a big fan of Git, I can see the appeal.

This would be a big feature to maintain though. Especially so with the given options and required UI around that. We'd really need to understand the benefits of such a feature.

I know there are benefits but from your feature request above the only real benefit listed is to capture a bigger audience. While nice I don't think this should ever be a driving point for how time and energy is spent on this project. I'd much rather focus on keeping and building to the core project definition.

[timeandtimeago]

I can say that at first the fact that the app used MySql as the data storage was a conn for my organization. We were trying to find a good wiki that used .md files for storage. We see it as a longevity thing. .md files will be around a long time and no mater what happens to the app we are using, our info remains in a user readable form that can be accessed through a git repo.

This said, after evaluating BookStack we decided that the features offered far out weighed the conn of not using Git.

If this is a major sticking point for a lot of people, perhaps a feature could be developed to simply back-populate a git repository instead of actually using it as the database/storage engine for the app. Basically maintain a git repo with .md files in parallel to the MYSQL database.

[thomasjsn]

What about an markdown export command? An artisan CLI command that produced a file structure based on shelves, books and chapters and .md files with page content.

[derek-shnosh]

After evaluating BookStack for some time now and actively using it for my current project tasking; I have almost, but not quite, concluded that we (i.e. my team) need git-based storage.

This isn't meant to be antagonistic, I just wanted to throw my vote in for this function.

[ssddanbrown]

@derek-shnosh If you could share the exact reasoning as to why your team would need git-based storage that would really help. (In terms of the exact functionality or process that it would enable for you)

[derek-shnosh]

Need is subjective obviously, but to answer I'd likely just refer to the key points from @tuaris's original statements in issues #776 and #777, especially points pertaining to backend storage and version control/diffs.

An added bonus would allow document editors to use an MD editor of their choice, given that the current MD editor (in BookStack) is a bit lacking in terms of features; e.g. Toolbar, table formatting, callouts, etc.

Edit: Hopefully the 'lacking' comment doesn't strike the wrong chord. I think BookStack is developed very well and appreciate your simplistic philosophy; our use-case/preference is just calling for a bit, more.

[ssddanbrown]

@derek-shnosh Thanks, and I totally understand.

If i was looking for a documentation platform to share with a group of developers, or at least technically confident people, I'd also look for something that I could edit in my own text editor that's git controlled.

That said, my primary target audience for BookStack is for more mixed environments where ease-of-use and low barrier-to-entry take priority (More WYSIWYG focused). While i'd love to support both audiences I feel those two things pull in two different directions and the focus of BookStack would be diluted thus efforts would be spread too thin.

I've thought about moving the page content out to static files which you could then check into git but even then, that comes with it's own set of challenges (indexing, caching, meta data editing, high-availability setups) and we'd still need the DB for other bits.

[derek-shnosh]

@ssddanbrown, understood.

Quite honestly, integrating a WYSIWYG MD-sourced editor like ToastUI for BookStack would be a very welcome addition, and possibly carry a hefty persuasion factor for those not super hung up on using git as their backend.

[ChristiaanDev]

Sorry for the bump but I agree, this would be a great feature. Would love to see this being implemented in BookStack. I think this would greatly benefit BookStack as it could appeal to a whole new audience (software dev teams).

[djagoo]

I also have to agree that this would be an awesome feature because the mysql database backend is the only conn I can see so far. I was looking for a solution I can throw in my existing IT documentation (md files) so my team can access it from everywhere. But I still need to be able to maintain those files in my editor.

I'm still not sure if all BookStack offers out weights this conn for me. But after testing it I'm really impressed.

Thanks and keep up the good work.

[Habi-Thapa]

Hello again @tuaris, Thank you for your suggestion.

My initial thoughts are that as a developer, and a big fan of Git, I can see the appeal.

This would be a big feature to maintain though. Especially so with the given options and required UI around that. We'd really need to understand the benefits of such a feature.

I know there are benefits but from your feature request above the only real benefit listed is to capture a bigger audience. While nice I don't think this should ever be a driving point for how time and energy is spent on this project. I'd much rather focus on keeping and building to the core project definition.

Git would be really helpful. With Git and Weblate, translation workflow would be much smoother.

[tuaris]

Revisiting this almost 3 years later... I realize that the only benefit of having a Git storage backend would be peer review. BookStack has since gained features such as revision history and a change log (I think). Also my views on Git have changed. It's lost it's appeal and I no longer believe Git to be a good solution for hosting documentation (but that's just my opinion).

I think a proper peer review feature would satisfy most people. I'm not sure if there is an open feature request for that or if it is already implemented.

[Habi-Thapa]

Revisiting this almost 3 years later... I realize that the only benefit of having a Git storage backend would be peer review. BookStack has since gained features such as revision history and a change log (I think). Also my views on Git have changed. It's lost it's appeal and I no longer believe Git to be a good solution for hosting documentation (but that's just my opinion).

I think a proper peer review feature would satisfy most people. I'm not sure if there is an open feature request for that or if it is already implemented.

Me neither. I'm also not a fan of git, as a platform to host documentation. But, I use Weblate for translating documents, which require 'rst' or 'md' files to be in git to automate the translation workflow.

I want to use Weblate with the Bookstack. Wiki.js works but I think Bookstack UI is cleaner.

Is there any way, I can integrate Bookstack with Weblate?

Thanks.

[leaf-node]

Having a Git back end can be useful if you're a sysadmin and your network goes offline. You'll still have your local Git repo to refer to any documentation you need. This has saved me many times in the past.

I don't know if there is a better way to handle this, but ikiwiki shows a second edit page with conflicting text if there is a merge conflict while editing in the browser. I don't know how complicated it would be, but it's probably not enough to just do a "git commit; git push" to a remote repo after every edit from a web browser.

[ssddanbrown]

@leaf-node If desired purely for a backup standpoint, I have an example API script here to export all books: https://github.com/BookStackApp/api-scripts/tree/main/php-export-all-books

You could tack on a couple of git commands to that script, then add it to a daily cron.

[helmut72]

Git storage would be fantastic. We can't get used to webbased text editors. They are all ok for casual writing, but everyone here prefers his own favourite (vim, vscode, bbedit...). Bookstack would be great as a webinterface, for reading and for small changes on the go.

[HerrDerb]

How does it look for this feature? It would be pretty much the enabler for us to use bookstack :)

[ssddanbrown]

@HerrDerb It's unlikely git will ever become a first-class citizen in BookStack, and thus be built-in.

If it's just required for backup, You could likely quite easily build something using our logical theme system. Within the 22.07 release post I displayed an example of saving page content out to a file on save, you could extend this customization with a few git commands to commit and push changes. Would slow down page saves though unless you maybe just only commit, then run the pushes externally via cron on a daily basis or something.

[A9G-Data-Droid]

A document management system needs version control. You need to be able to see previous versions, the difference between versions, and who did what. If your pages are markdown then git is the obvious choice. The main repo contains all published official pages. All edits happen on a draft repo. When someone with approve permission approves a draft it gets pushed to the main repo, satisfying #473.

[skylord123]

Wanted to come here and state that my company is evaluating using this product and this question was one of the ones that came up. Being able to store a doc in git and have bookstack keep a clone of it would be really handy. I would prefer people modify documents via git so we have a versioned history and can still easily get to the files if BookStack goes down. Plus we can then do content moderation by having PR requests have to be reviewed and approved by someone else that didn't write the content which would solve #473 for us.

So personally I think adding this feature would be extremely handy and be the final nail that makes this the perfect tool for us.

[otherjoel]

Seems like this kind of thing could be done by a combo of GitHub actions and the Bookstack API without gumming up the regular Bookstack interface. There are also already lots of other tools for people who prefer publishing using a more complicated git-first workflow.

[skylord123]

Seems like this kind of thing could be done by a combo of GitHub actions and the Bookstack API without gumming up the regular Bookstack interface. There are also already lots of other tools for people who prefer publishing using a more complicated git-first workflow.

Hmm this is a good point. I'll have to look into this. Could use something like Node-RED to automate this from the API.