• wiki-migration-test Repo for testing migration of wiki (and maybe other stuff) ** Plan
• This is a test-bed for conversion of the trac wiki to a github wiki

*** DONE Stage 1: Dump the pages from Trac wiki to files CLOSED: [2019-10-16 Wed 19:58]

• Main difficulty was getting access to the server, but now sorted
• Also ~trac-admin~ only seems to work when run from inside the ~webapps/trac/bin~ directory

• Commands to dump the wiki files and tar them up

  +begin_src sh

  cd webapps/trac/bin ./trac-admin ../trac wiki dump ~/will-trac-wiki-dump cd ~ tar czf will-trac-wiki-dump.tar.gz will-trac-wiki-dump

  +end_src

  *** TODO Stage 2: Import wiki pages into github repo There are two options here:

  1. Leave files as in current format
     • It appears that trac wiki format is similar to moinmoin and mediawiki, but not quite the same as either
     • Github offers the following formats
     • AsciiDoc
     • Creole
     • Markdown
     • This is the only one with full support from the online editor
     • This is default, and probably most commonly used on github
     • MediaWiki
     • Zero support from online editor
     • OrgMode
     • 90% support from online editor
     • Does have some features that Markdown lacks, such as tables
     • Potential clash between org-mode link syntax and wikilinks
     • Pod
     • RDoc
     • Textile
     • RestructuredText
     • I will try copy a file verbatim and adding a mediawiki extension
     • [X] Try with ~CompilingCloudyC17~
     • This works partially
     • Good
       • Auto-generates TOC
       • Headers, bold font, and external links work fine
     • Bad
       • All the verbatim code, like ~{{{whatever}}}~, is just dropped. Not surprising since this is not actually mediawiki syntax

  2. Convert to markdown

     • There are various scripts that claim to do this
     • pandoc

       +begin_src sh

       pandoc --from mediawiki --to markdown -o WikiStart.md WikiStart

       +end_src

     • This was my first attempt, using ~WikiStart~
       • Good
       • Headers (but see below) and links and bold seems OK
       • Bad
       • Plain text ~CamelCaseLinks~ are not honored
       • Explicit links like ~[wiki:NewC17]~ are not converted either
       • Strange target-ids appear after headers, like ~{#getting_started_with_cloudy}~
     • https://gitlab.xiph.org/ePirat/trac-to-gitlab has a Trac2Down.py
     • Tracboat also has a trac2down.py
     • They both seem to be descended from the same code (Eric van der Vlist, 2013), but this one is more full-featured
     • There is a function ~trac2down.convert(TEXT, BASEPATH, **kwds)~
       • ~TEXT~ is text of wiki or ticket (or other ... ?)
       • ~BASEPATH~ is a required argument and seems to be related to the source code tree, which I will ignore for now (set to ".")

     • Make a simple wrapper [[file:test-trac2down.py][test-track2down.py]] to convert all the dumped wiki files:

       +begin_src python :eval no :tangle test-trac2down.py

       import sys import re from pathlib import Path sys.path.append("../../../Source/tracboat/src/tracboat") import trac2down

       Paths for reading input and saving output

       try: input_path = Path(sys.argv[1]) save_path = Path(sys.argv[2]) except (IndexError, ValueError, TypeError): sys.exit(f"Usage: {sys.argv[0]} INPUT_PATH SAVE_PATH")

       Get the names so that we can convert links to gollum format

       "CamelCaseName" -> "[[CamelCaseName]]"

       wikinames = [p.name for p in input_path.glob("*")]

       def fixup_wikilinks(text, targets): """Add double [[...]] around wikilinks (given as targets)

       Most prosaic method possible - simply loop over the explicit list of targets. Tries to be careful in avoiding code blocks, but will still be fooled by inline code """ lines = [] is_code_block = False for line in text.split('\n'):

       not blockquote?

          if not line.startswith('    '):
              if line.startswith("````"):
                  is_code_block = not is_code_block
              if not is_code_block:
                  for target in targets:
                      line = line.replace(target, f"[[{target}]]")
              # line = re.sub(r'\!(([A-Z][a-z0-9]+){2,})', r'[[\1]]', line)
          lines.append(line)

       return "\n".join(lines)

       Process all the files in the input folder

       for p in input_path.glob("*"):

       Open file from the dump of the trac database and convert it

       with open(p) as f: text = trac2down.convert(f.read(), ".") text = fixup_wikilinks(text, wikinames)

       Save the converted file with a ".md" extension to output folder

       The 3rd, 4th, 5th arguments to this function are unused

       trac2down.save_file(text, p.name, None, None, None, save_path)

       +end_src

     • Try it out

       +begin_src sh :results silent

       python test-trac2down.py ../will-trac-wiki-dump ../wiki-migration-test.wiki

       +end_src

     • Maybe modify one or write my own
     • Dealing with the ~CamelCaseLinks~
       • We can get a list of all the current pages names, and then replace all mentions of these with a suitably formatted link: ~[[WikiLink]]~
       • This is now what I am doing, but need to improve it a bit
       • I noticed that there are false positives that are inside URLs
       • Could be fixed by not allowing any letters either side - so need to go back to regexps
• Other useful-looking stuff
• [ ] Should find a better home for this section ** Documentation of gollum - the github wiki rendering engine
• https://github.com/gollum/gollum/wiki
• Can also be run standalone
• It has its own wiki tag syntax, ~[[STUFF]]~ which supports
  • internal wiki links
  • external links
  • images
  • include of other files but I think this is not supported in github wikis
• Also has macros - but is that really supported on github?
• Quote from https://help.github.com/en/articles/editing-wiki-content : For security and performance reasons, some syntaxes are unsupported. : : Transclusion : Definition lists : Indentation : Table of contents

** Allowing math equations

• Github doesn't support embedded javascript, so no mathjax
• Seem to be two main approaches
  1. Call out to some web service to render equation on-the-fly
     • Seems very clunky
  2. Replace equations with image or SVG
     • would be fine if efficiently automated
• Best bet (method 2) seems to be [[https://github.com/apps/texify][github-texify]]
  • Main repo https://github.com/agurodriguez/github-texify
  • Githup app install https://github.com/apps/texify
  • Adds a commit hook that transforms all files with names ~FOO.tex.md~ to ~FOO.md~ with all latex math replaced with inline SVG images (only supports dollar signs, it seems)
  • There are complaints it does not support wikis
  • See https://github.com/agurodriguez/github-texify/issues/10
  • However, I think maybe the complainants aren't recognizing that the wiki is itself a repo - so possibly it could still work

** Continuous integration, etc with Github Actions

• Currently this is a public beta that you have to ask to join
• https://github.com/features/actions
• It looks very powerful - workflows can run on windows, linux, or macos hosts, triggered by pushes, releases, etc.
• Pricing is free for public repos, 3000 free minutes/month for private repos after beta period ends