search

kovetskiy / mark

Sync your markdown files with Confluence pages.
https://samizdat.dev
Other
972 stars 143 forks source link
confluence git markdown

readme

Mark

All Contributors

Mark — a tool for syncing your markdown documentation with Atlassian Confluence pages.

Read the blog post discussing the tool — https://samizdat.dev/use-markdown-for-confluence/

This is very useful if you store documentation to your software in a Git repository and don't want to do an extra job of updating Confluence page using a tinymce wysiwyg enterprise core editor which always breaks everything.

Mark does the same but in a different way. Mark reads your markdown file, creates a Confluence page if it's not found by its name, uploads attachments, translates Markdown into HTML and updates the contents of the page via REST API. It's like you don't even need to create sections/pages in your Confluence anymore, just use them in your Markdown documentation.

Mark uses an extended file format, which, still being valid markdown, contains several HTML-ish metadata headers, which can be used to locate page inside Confluence instance and update it accordingly.

File in the extended format should follow the specification:

<!-- Space: <space key> -->
<!-- Parent: <parent 1> -->
<!-- Parent: <parent 2> -->
<!-- Title: <title> -->
<!-- Attachment: <local path> -->
<!-- Label: <label 1> -->
<!-- Label: <label 2> -->

<page contents>

There can be any number of Parent headers, if Mark can't find specified parent by title, Mark creates it.

Also, optional following headers are supported:

<!-- Layout: (article|plain) -->
<!-- Type: (page|blogpost) -->
<!-- Content-Appearance: (full-width|fixed) -->
<!-- Sidebar: <h2>Test</h2> -->

Setting the sidebar creates a column on the right side. You're able to add any valid HTML content. Adding this property sets the layout to article.

Mark supports Go templates, which can be included into article by using path to the template relative to current working dir, e.g.:

<!-- Include: <path> -->

If the template cannot be found relative to the current directory, a fallback directory can be defined via --include-path. This way it is possible to have global include files while local ones will still take precedence.

Optionally the delimiters can be defined:

<!-- Include: <path>
     Delims: "<<", ">>"
     -->

Or they can be switched off to disable processing:

<!-- Include: <path>
     Delims: none
     -->

Note: Switching delimiters off really simply changes them to ASCII characters "\x00" and "\x01" which, usually should not occure in a template.

Templates can accept configuration data in YAML format which immediately follows the Include and Delims tag, if present:

<!-- Include: <path>
     <yaml-data> -->

Mark also supports attachments. The standard way involves declaring an Attachment along with the other items in the header, then have any links with the same path:

<!-- Attachment: <path-to-image> -->

<beginning of page content>

An attached link is [here](<path-to-image>)

NOTE: Be careful with Attachment! If your path string is a subset of another longer string or referenced in text, you may get undesired behavior.

Mark also supports macro definitions, which are defined as regexps which will be replaced with specified template:

<!-- Macro: <regexp>
     Template: <path>
     <yaml-data> -->

NOTE: Make sure to define your macros after your metadata (Title/Space), mark will stop processing metadata if it hits a Macro.

Capture groups can be defined in the macro's which can be later referenced in the <yaml-data> using ${<number>} syntax, where <number> is number of a capture group in regexp (${0} is used for entire regexp match), for example:

  <!-- Macro: MYJIRA-\d+
       Template: ac:jira:ticket
       Ticket: ${0} -->

Macros can also use inline templates. Inline templates are templates where the template content is described in the <yaml-data>. The Template value starts with a #, followed by the key used in the <yaml-data>. The key's value must be a string which defines the template's content.

  <!-- Macro: <tblbox\s+(.*?)\s*>
       Template: #inline
       title: ${1}
       inline: |
           <table>
           <thead><tr><th>{{ .title }}</th></tr></thead>
           <tbody><tr><td>
        -->
  <!-- Macro: </tblbox>
       Template: #also_inline
       also_inline: |
           </td></tr></tbody></table>
        -->
  <tblbox with a title>
  and some
  content
  </tblbox>

Customizing the page layout

If you set the Layout to plain, the page layout can be customized using HTML comments inside the markdown:

<!-- Layout: plain -->
<!-- ac:layout -->

<!-- ac:layout-section type:three_with_sidebars -->
<!-- ac:layout-cell -->
More Content
<!-- ac:layout-cell end -->
<!-- ac:layout-cell -->
More Content
<!-- ac:layout-cell end -->
<!-- ac:layout-cell -->
Even More Content
<!-- ac:layout-cell end -->
<!-- ac:layout-section end -->

<!-- ac:layout-section type:single -->
<!-- ac:layout-cell -->
Still More Content
<!-- ac:layout-cell end -->
<!-- ac:layout-section end -->

<!-- ac:layout end -->

Please be aware that mark does not validate the layout, so it's your responsibility to create a valid layout.

Code Blocks

If you have long code blocks, you can make them collapsible with the Code Block Macro:

...
some long bash code block
...

And you can also add a title:

...
some long bash code block
...

Or linenumbers, by giving the first number

...
some long bash code block
...

And even themes

...
some long bash code block
...

Please note that, if you want to have a code block without a language use - as the first character, if you want to have the other goodies

...
some long code block
...

Block Quotes

Block Quotes are converted to Confluence Info/Warn/Note box when the following conditions are met

  1. The BlockQuote is on the root level of the document (not nested)
  2. The first line of the BlockQuote contains one of the following patterns Info/Warn/Note

In any other case the default behaviour will be resumed and html <blockquote> tag will be used

Template & Macros

By default, mark provides several built-in templates and macros:

Template & Macros Usecases

Insert Disclaimer

This should be in disclaimer.md.

**NOTE**: this document is generated, do not edit manually.

Add this to your article.md.

<!-- Space: TEST -->
<!-- Title: My Article -->

<!-- Include: disclaimer.md -->

This is my article.

Insert Status Badge

<!-- Space: TEST -->
<!-- Title: TODO List -->

<!-- Macro: :done:
     Template: ac:status
     Title: DONE
     Color: Green -->

<!-- Macro: :todo:
     Template: ac:status
     Title: TODO
     Color: Blue -->

* :done: Write Article
* :todo: Publish Article

Insert Colored Text Box

<!-- Space: TEST -->
<!-- Title: Announcement -->

<!-- Macro: :box:([^:]+):([^:]*):(.+):
     Template: ac:box
     Icon: true
     Name: ${1}
     Title: ${2}
     Body: ${3} -->

:box:info::Foobar:
:box:tip:Tip of day:Foobar:
:box:note::Foobar:
:box:warning:Alert!:Foobar:

Insert Table of Contents

<!-- Include: ac:toc -->

If default TOC looks don't find a way to your heart, try parametrizing it, for example:

<!-- Macro: :toc:
     Template: ac:toc
     Printable: 'false'
     MinLevel: 2 -->

# This is my nice title

:toc:

You can call the Macro as you like but the Template field must have the ac:toc value. Also, note the single quotes around 'false'.

See Confluence TOC Macro for the list of parameters - keep in mind that here they start with capital letters. Every skipped field will have the default value, so feel free to include only the ones that you require.

Insert PageTree

# My First Heading
<!-- Include: ac:pagetree -->

The pagetree macro works almost the same as the TOC above, but the tree behavior is more desirable for creating placeholder pages above collections of SOPs.

The default pagetree macro behavior is to insert a tree rooted @self.

The following parameters can be used to alter your default configuration with parameters described more in depth here:Confluence Pagetree Macro.

Parameters:

E.G.

<!-- Macro: :pagetree:
     Template: ac:pagetree
     Reverse: 'true'
     ExpandCollapseAll: 'true'
     StartDepth: 2 -->

# My First Heading

:pagetree:

Insert Children Display

To include Children Display (TOC displaying children pages) use following macro:

<!-- Macro: :children:
     Template: ac:children
-->

# This is my nicer title

:children:

You can use various parameters to modify Children Display:

<!-- Macro: :children:
     Template: ac:children
     Sort: title
     Style: h3
     Excerpt: simple
     First: 10
     Page: Space:Page title
     Depth: 2
     Reverse: false
     All: false -->

# This is my nicest title

:children:

Insert Jira Ticket

<!-- Space: TEST -->
<!-- Title: TODO List -->

<!-- Macro: MYJIRA-\d+
     Template: ac:jira:ticket
     Ticket: ${0} -->

See task MYJIRA-123.

Insert link to existing confluence page by title

This is a [link to an existing confluence page](ac:Pagetitle)

And this is how to link when the linktext is the same as the [Pagetitle](ac:)

Link to a [page title containing spaces](<ac:With Multiple Words>)

Upload and included inline images

![Example](../images/examples.png)

will automatically upload the inlined image as an attachment and inline the image using the ac:image template.

If the file is not found, it will inline the image using the ac:image template and link to the image.

Add width for an image

Use the following macro:

<!-- Macro: \!\[.*\]\((.+)\)\<\!\-\- width=(.*) \-\-\>
     Template: ac:image
     Attachment: ${1}
     Width: ${2} -->

And attach any image with the following

![Example](../images/example.png)<!-- width=300 -->

The width will be the commented html after the image (in this case 300px).

Currently this is not compatible with the automated upload of inline images.

Render Mermaid Diagram

Confluence doesn't provide mermaid.js support natively. Mark provides a convenient way to enable the feature like Github does. As long as you have a code block and are marked as "mermaid", the mark will automatically render it as a PNG image and insert into before the code block.

graph TD;
A-->B;

In order to properly render mermaid, you can choose between the following mermaid providers:

Installation

Homebrew

brew tap kovetskiy/mark
brew install mark

Go Install / Go Get

go install github.com/kovetskiy/mark@latest

For older versions

go get -v github.com/kovetskiy/mark

Releases

Download a release from the Releases page

Docker

docker run --rm -i kovetskiy/mark:latest mark <params>

Compile and install using docker-compose

Mostly useful when you intend to enhance mark.

# Create the binary
$ docker-compose run markbuilder
# "install" the binary
$ cp mark /usr/local/bin

Usage

NAME:
   mark - A tool for updating Atlassian Confluence pages from markdown.

USAGE:
   mark [global options] 

VERSION:
   9.12.0

DESCRIPTION:
   Mark is a tool to update Atlassian Confluence pages from markdown. Documentation is available here: https://github.com/kovetskiy/mark

GLOBAL OPTIONS:
   --files value, -f value                       use specified markdown file(s) for converting to html. Supports file globbing patterns (needs to be quoted). [$MARK_FILES]
   --compile-only                                show resulting HTML and don't update Confluence page content. (default: false) [$MARK_COMPILE_ONLY]
   --dry-run                                     resolve page and ancestry, show resulting HTML and exit. (default: false) [$MARK_DRY_RUN]
   --edit-lock, -k                               lock page editing to current user only to prevent accidental manual edits over Confluence Web UI. (default: false) [$MARK_EDIT_LOCK]
   --drop-h1, --h1_drop                          don't include the first H1 heading in Confluence output. (default: false) [$MARK_H1_DROP]
   --strip-linebreaks, -L                        remove linebreaks inside of tags, to accomodate non-standard Confluence behavior (default: false) [$MARK_STRIP_LINEBREAK]
   --title-from-h1, --h1_title                   extract page title from a leading H1 heading. If no H1 heading on a page exists, then title must be set in the page metadata. (default: false) [$MARK_H1_TITLE]
   --minor-edit                                  don't send notifications while updating Confluence page. (default: false) [$MARK_MINOR_EDIT]
   --version-message value                       add a message to the page version, to explain the edit (default: "") [$MARK_VERSION_MESSAGE]
   --color value                                 display logs in color. Possible values: auto, never. (default: "auto") [$MARK_COLOR]
   --debug                                       enable debug logs. (default: false) [$MARK_DEBUG]
   --trace                                       enable trace logs. (default: false) [$MARK_TRACE]
   --username value, -u value                    use specified username for updating Confluence page. [$MARK_USERNAME]
   --password value, -p value                    use specified token for updating Confluence page. Specify - as password to read password from stdin, or your Personal access token. Username is not mandatory if personal access token is provided. For more info please see: https://developer.atlassian.com/server/confluence/confluence-server-rest-api/#authentication. [$MARK_PASSWORD]
   --target-url value, -l value                  edit specified Confluence page. If -l is not specified, file should contain metadata (see above). [$MARK_TARGET_URL]
   --base-url value, -b value, --base_url value  base URL for Confluence. Alternative option for base_url config field. [$MARK_BASE_URL]
   --config value, -c value                      use the specified configuration file. (default: System specific) [$MARK_CONFIG]
   --ci                                          run on CI mode. It won't fail if files are not found. (default: false) [$MARK_CI]
   --space value                                 use specified space key. If the space key is not specified, it must be set in the page metadata. [$MARK_SPACE]
   --parents value                               A list containing the parents of the document separated by parents-delimiter (default: '/'). These will be prepended to the ones defined in the document itself. [$MARK_PARENTS]
   --parents-delimiter value                     The delimiter used for the parents list (default: "/") [$MARK_PARENTS_DELIMITER]
   --mermaid-provider value                      defines the mermaid provider to use. Supported options are: cloudscript, mermaid-go. (default: "cloudscript") [$MARK_MERMAID_PROVIDER]
   --mermaid-scale value                         defines the scaling factor for mermaid renderings. (default: 1) [$MARK_MERMAID_SCALE]
   --include-path value                          Path for shared includes, used as a fallback if the include doesn't exist in the current directory. [$MARK_INCLUDE_PATH]
   --help, -h                                    show help
   --version, -v                                 print the version

You can store user credentials in the configuration file, which should be located in a system specific directory (or specified via -c --config <path>) with the following format (TOML):

username = "your-email"
password = "password-or-api-key-for-confluence-cloud"
# If you are using Confluence Cloud add the /wiki suffix to base_url
base-url = "http://confluence.local"
title-from-h1 = true
drop-h1 = true

NOTE: Labels aren't supported when using minor-edit!

NOTE: The system specific locations are described in here: https://pkg.go.dev/os#UserConfigDir. Currently these are: On Unix systems, it returns $XDG_CONFIG_HOME as specified by https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html if non-empty, else $HOME/.config. On Darwin, it returns $HOME/Library/Application Support. On Windows, it returns %AppData%. On Plan 9, it returns $home/lib.

Tricks

Continuous Integration

It's quite trivial to integrate Mark into a CI/CD system, here is an example with Snake CI in case of self-hosted Bitbucket Server / Data Center.

stages:
  - sync

Sync documentation:
  stage: sync
  only:
    branches:
      - main
  image: kovetskiy/mark
  commands:
    - for file in $(find -type f -name '*.md'); do
        echo "> Sync $file";
        mark -u $MARK_USER -p $MARK_PASS -b $MARK_URL -f $file || exit 1;
        echo;
      done

In this example, I'm using the kovetskiy/mark image for creating a job container where the repository with documentation will be cloned to. The following command finds all *.md files and runs mark against them one by one:

for file in $(find -type f -name '*.md'); do
    echo "> Sync $file";
    mark -u $MARK_USER -p $MARK_PASS -b $MARK_URL -f $file || exit 1;
    echo;
done

The following directive tells the CI to run this particular job only if the changes are pushed into the main branch. It means you can safely push your changes into feature branches without being afraid that they automatically shown in Confluence, then go through the reviewal process and automatically deploy them when PR got merged.

only:
  branches:
    - main

File Globbing

Rather than running mark multiple times, or looping through a list of files from find, you can use file globbing (i.e. wildcard patterns) to match files in subdirectories. For example:

mark -f "helpful_cmds/*.md"

You can also use ** to get all files recursively.

mark -f "**/docs/*.md"

Linting markdown

We recommend to lint your markdown files with markdownlint-cli2 before publishing them to confluence to catch any conversion errors early.

Issues, Bugs & Contributions

I've started the project to solve my own problem and open sourced the solution so anyone who has a problem like me can solve it too. I have no profits/sponsors from this projects which means I don't really prioritize working on this project in my free time. I still check the issues and do code reviews for Pull Requests which means if you encounter a bug in the program, you should not expect me to fix it as soon as possible, but I'll be very glad to merge your own contributions into the project and release the new version.

I try to label all new issues so it's easy to find a bug or a feature request to fix/implement, if you are willing to help with the project, you can use the following labels to find issues, just make sure to reply in the issue to let everyone know you took the issue:

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Manuel Rüger
Manuel Rüger
🚧 💻		 Egor Kovetskiy
Egor Kovetskiy
🚧 💻		 Nick Klauer
Nick Klauer
💻		 Rolf Ahrenberg
Rolf Ahrenberg
💻		 Charles Southerland
Charles Southerland
💻		 Šarūnas Nejus
Šarūnas Nejus
💻		 Alexey Baranov
Alexey Baranov
💻
Anthony Barbieri
Anthony Barbieri
💻		 Devin Auclair
Devin Auclair
💻		 Gezim Sejdiu
Gezim Sejdiu
💻		 Josip Ćavar
Josip Ćavar
💻		 Juho Saarinen
Juho Saarinen
💻		 Luke Fritz
Luke Fritz
💻		 Matt Radford
Matt Radford
💻
Planktonette
Planktonette
💻		 Stefano Teodorani
Stefano Teodorani
💻		 Tim Schrumpf
Tim Schrumpf
💻		 Tyler Cole
Tyler Cole
💻		 elgreco247
elgreco247
💻		 emead-indeed
emead-indeed
💻		 Will Hegedus
Will Hegedus
💻
Leandro Carneiro
Leandro Carneiro
💻		 beeme1mr
beeme1mr
💻		 Taldrain
Taldrain
💻		 Hugo Cisneiros
Hugo Cisneiros
💻		 jevfok
jevfok
💻		 Mateus Miranda
Mateus Miranda
💻		 Stephan Hradek
Stephan Hradek
💻
Dreampuf
Dreampuf
💻		 Joel Andritsch
Joel Andritsch
💻		 guoweis-outreach
guoweis-outreach
💻		 klysunkin
klysunkin
💻		 Florent Monbillard
Florent Monbillard
💻		 Joey Freeland
Joey Freeland
💻		 Noam Asor
Noam Asor
💻
Philipp
Philipp
💻		 Pommier Vincent
Pommier Vincent
💻		 Toru Kawaguchi
Toru Kawaguchi
💻		 Will Gorman
Will Gorman
💻		 Zackery Griesinger
Zackery Griesinger
💻		 cc-chris
cc-chris
💻		 datsickkunt
datsickkunt
💻
recrtl
recrtl
💻		 Stanislav Seletskiy
Stanislav Seletskiy
💻

This project follows the all-contributors specification. Contributions of any kind welcome!