find a way to create static html dumps

[ThomasWaldmann]

Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).

---

moin 1.9 had "moin dump" to create static html dumps, but moin2 has no such feature yet.

Problem: if one wants to load such html directly from the filesystem into a browser (without using a web server), the URLs in the html need to be relative. also, the filenames need to get some appropriate extension, because there is no web server to give a content-type header.

I tried Frozen-Flask a while ago, but was unable to create relative URLs. So we could use this just for html dumps that are served at some specific URL.

[ThomasWaldmann]

Original comment by RogerHaase (Bitbucket: RogerHaase, GitHub: RogerHaase).

---

fixed by 7842b1a (bb) and 082d9e8 (bb)

[ThomasWaldmann]

Original comment by RogerHaase (Bitbucket: RogerHaase, GitHub: RogerHaase).

---

added cms theme https://codereview.appspot.com/257550043/

https://codereview.appspot.com/261900043/ has an updated TODO list

pushed to my repo 082d9e807148 (bb) and 7842b1aff804 (bb)

[ThomasWaldmann]

Original comment by RogerHaase (Bitbucket: RogerHaase, GitHub: RogerHaase).

---

https://codereview.appspot.com/261900043/

• needs a CMS-like theme and a way to override current theme
  • app.cfg.theme_default = u'modernized' has no effect
  • copy theme.css
• should all raw data be copied to +get directory, or only images, zip, targz, ...?
• append ".html" to all html files and fix resulting broken links
• fix links to subitems
  • becomes
  • but do not change
• the "svg" image does not display, but works if renamed to svg.svg and link is changed accordingly

Original comment by RogerHaase (Bitbucket: RogerHaase, GitHub: RogerHaase).

---

I gave Frozen-Flask a try on Windows 4 years after the above comments. Short answer is Frozen-Flask looks hopeless.

Last release is Version 0.11 on 2013-06-13. There are a few patches added on github and 10 open issues: https://github.com/SimonSapin/Frozen-Flask.

Problems:

1. On moin 1.9, the "export dump" command created .html files for wiki pages. Frozen-Flask hooks into the url_for method and tries to create pages for everything, including the +admin, +misc, etc resulting in unhandled 403 return codes. I had to add code to Frozen-Flask to ignore the 403's. There is no option to ignore files matching a regex. The files created do not have a .html suffix.

2. Seems poorly tested on Windows, yielding some permission denied errors trying to write a file. A guess is an attempt was made to open the file twice.

3. On the few occasion I somehow managed to process the Home, moin, creole files, a directory named Home, moin, creole was created with an index.html file within each.

4. Links to any URL with an embedded + are broken.

To get Frozen-Flask "working", I did the following:

To make.py, add this near line 378:

def cmd_dump(self, *args):
    """create a directory of html files for each item within the wiki"""
    if wiki_exists():
        command = '{0}moin dump'.format(ACTIVATE)
        with open(BACKUPWIKI, 'w') as messages:
            result = subprocess.call(command, shell=True, stderr=messages, stdout=messages)
        if result == 0:
            print 'Success: wiki was dumped to {0}'.format(filename)
        else:
            print 'Important messages from {0} are shown below. Do "{1} log backup" to see complete log.'.format(BACKUPWIKI, M)
            search_for_phrase(BACKUPWIKI)
            print '\nError: attempt to backup wiki failed.'
    else:
        print 'Error: cannot dump wiki because it has not been created.'

To /script/init.py, add this near line 38:

from MoinMoin.script.maint import moindump
manager.add_command("dump", moindump.Dump())

In /script/maint/ add a new file moindump.py:

"""
MoinMoin - use Frozen-Flask to create a static html dump of wiki items
"""

import sys
import os

from flask.ext.frozen import Freezer
from flask import current_app as app
from flask.ext.script import Command, Option

from MoinMoin.app import create_app

wiki_config = os.path.dirname(os.path.abspath(__file__)) + '/../../../wikiconfig_local.py'
app = create_app(wiki_config)

freezer = Freezer(app)

@freezer.register_generator
def product_url_generator():
    # URLs as strings
    yield '/Home/'
    yield '/creole/'
    yield '/moin/'

from MoinMoin import log
logging = log.getLogger(__name__)

class Dump(Command):

    def run(self):
        freezer.freeze()

In wikiconfig_editme.py, add a few config parameters:

FREEZER_BASE_URL = 'http://localhost:8080'
FREEZER_DESTINATION_IGNORE = []
FREEZER_DESTINATION = '../FreezerQQQ'
FREEZER_REMOVE_EXTRA_FILES = True
FREEZER_IGNORE_404_NOT_FOUND = True
FREEZER_RELATIVE_URLS = False

Run it by doing ./m dump