search

scragg0x / realms-wiki

Git based wiki inspired by Gollum
http://realms.io
GNU General Public License v2.0
833 stars 91 forks source link

persistent data #218

Closed nFrechen closed 6 years ago

nFrechen commented 6 years ago

I try to use realms with docker and store the persistent data on the host.

If I do it following your privided docker-compose.yml file like the following:

version: "3"

services:

  realms:
    container_name: realms
    image: realms/realms-wiki
    volumes:
      - ./realms-data:/home/wiki/data
    ports:
      - "5001:5000"

The folder /home/wiki/data stays empty. I found the .md files inside the container in /tmp/wiki.

Then I tried to provide a realms-wiki.json to set the file path to /home/wiki/data:

version: "3"

services:

  realms:
    container_name: realms
    image: realms/realms-wiki
    volumes:
      - ./realms-data:/home/wiki/data
      - ./realms-wiki.json:/home/wiki/realms-wiki/realms-wiki.json
    ports:
      - "5001:5000"

with a json file like this: 

{
    "ALLOW_ANON": true,
    "BASE_URL": "http://localhost",
    "CACHE_TYPE": "simple",
    "DB_URI": "sqlite:////home/wiki/data/wiki.db",
    "PORT": 5000,
    "REGISTRATION_ENABLED": true,
    "SECRET_KEY": "pSD4IZzBEPRFuztt8NynbkdtVoujoLdgNOfbNmfwFFgeJ5o4wMlAQrEYXlFWRYil",
    "SITE_TITLE": "REALMS",
    "WIKI_PATH": "/home/wiki/data"
}

This time the container doesn't start properly becasue Worker failed to boot.:

$ docker-compose up
Creating network "test_default" with the default driver
Creating realms ... done
Attaching to realms
realms    | [2018-03-11 14:55:00 +0000] [5] [INFO] Starting gunicorn 19.3.0
realms    | [2018-03-11 14:55:00 +0000] [5] [INFO] Listening at: http://0.0.0.0:5000 (5)
realms    | [2018-03-11 14:55:00 +0000] [5] [INFO] Using worker: gevent
realms    | [2018-03-11 14:55:00 +0000] [10] [INFO] Booting worker with pid: 10
realms    | [2018-03-11 14:55:00 +0000] [11] [INFO] Booting worker with pid: 11
realms    | [2018-03-11 14:55:00 +0000] [12] [INFO] Booting worker with pid: 12
realms    | /home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/flask_cache/jinja2ext.py:33: ExtDeprecationWarning: Importing flask.ext.cache is deprecated, use flask_cache instead.
realms    |   from flask.ext.cache import make_template_fragment_key
realms    | /home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/flask_cache/jinja2ext.py:33: ExtDeprecationWarning: Importing flask.ext.cache is deprecated, use flask_cache instead.
realms    |   from flask.ext.cache import make_template_fragment_key
realms    | [2018-03-11 14:55:00 +0000] [11] [ERROR] Exception in worker process:
realms    | Traceback (most recent call last):
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 507, in spawn_worker
realms    |     worker.init_process()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 192, in init_process
realms    |     super(GeventWorker, self).init_process()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 118, in init_process
realms    |     self.wsgi = self.app.wsgi()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
realms    |     self.callable = self.load()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
realms    |     return self.load_wsgiapp()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
realms    |     return util.import_app(self.app_uri)
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/util.py", line 366, in import_app
realms    |     app = eval(obj, mod.__dict__)
realms    |   File "<string>", line 1, in <module>
realms    |   File "/home/wiki/realms-wiki/realms/__init__.py", line 218, in create_app
realms    |     app.discover()
realms    |   File "/home/wiki/realms-wiki/realms/__init__.py", line 83, in discover
realms    |     sources.init(self)
realms    |   File "/home/wiki/realms-wiki/realms/modules/wiki/__init__.py", line 11, in init
realms    |     Wiki(app.config['WIKI_PATH'])
realms    |   File "/home/wiki/realms-wiki/realms/lib/hook.py", line 15, in wrapper
realms    |     rv = fn(self, *args, **kwargs)
realms    |   File "/home/wiki/realms-wiki/realms/modules/wiki/models.py", line 35, in __init__
realms    |     self.repo = Repo.init(path, mkdir=True)
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/dulwich/repo.py", line 1006, in init
realms    |     os.mkdir(path)
realms    | OSError: [Errno 17] File exists: '/home/wiki/data'
realms    | Traceback (most recent call last):
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 507, in spawn_worker
realms    |     worker.init_process()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 192, in init_process
realms    |     super(GeventWorker, self).init_process()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 118, in init_process
realms    |     self.wsgi = self.app.wsgi()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
realms    |     self.callable = self.load()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
realms    |     return self.load_wsgiapp()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
realms    |     return util.import_app(self.app_uri)
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/util.py", line 366, in import_app
realms    |     app = eval(obj, mod.__dict__)
realms    |   File "<string>", line 1, in <module>
realms    |   File "/home/wiki/realms-wiki/realms/__init__.py", line 218, in create_app
realms    |     app.discover()
realms    |   File "/home/wiki/realms-wiki/realms/__init__.py", line 83, in discover
realms    |     sources.init(self)
realms    |   File "/home/wiki/realms-wiki/realms/modules/wiki/__init__.py", line 11, in init
realms    |     Wiki(app.config['WIKI_PATH'])
realms    |   File "/home/wiki/realms-wiki/realms/lib/hook.py", line 15, in wrapper
realms    |     rv = fn(self, *args, **kwargs)
realms    |   File "/home/wiki/realms-wiki/realms/modules/wiki/models.py", line 35, in __init__
realms    |     self.repo = Repo.init(path, mkdir=True)
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/dulwich/repo.py", line 1006, in init
realms    |     os.mkdir(path)
realms    | OSError: [Errno 17] File exists: '/home/wiki/data'
realms    | [2018-03-11 14:55:00 +0000] [11] [INFO] Worker exiting (pid: 11)
realms    | [2018-03-11 14:55:00 +0000] [10] [ERROR] Exception in worker process:
realms    | Traceback (most recent call last):
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 507, in spawn_worker
realms    |     worker.init_process()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 192, in init_process
realms    |     super(GeventWorker, self).init_process()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 118, in init_process
realms    |     self.wsgi = self.app.wsgi()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
realms    |     self.callable = self.load()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
realms    |     return self.load_wsgiapp()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
realms    |     return util.import_app(self.app_uri)
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/util.py", line 366, in import_app
realms    |     app = eval(obj, mod.__dict__)
realms    |   File "<string>", line 1, in <module>
realms    |   File "/home/wiki/realms-wiki/realms/__init__.py", line 218, in create_app
realms    |     app.discover()
realms    |   File "/home/wiki/realms-wiki/realms/__init__.py", line 83, in discover
realms    |     sources.init(self)
realms    |   File "/home/wiki/realms-wiki/realms/modules/wiki/__init__.py", line 11, in init
realms    |     Wiki(app.config['WIKI_PATH'])
realms    |   File "/home/wiki/realms-wiki/realms/lib/hook.py", line 15, in wrapper
realms    |     rv = fn(self, *args, **kwargs)
realms    |   File "/home/wiki/realms-wiki/realms/modules/wiki/models.py", line 35, in __init__
realms    |     self.repo = Repo.init(path, mkdir=True)
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/dulwich/repo.py", line 1006, in init
realms    |     os.mkdir(path)
realms    | OSError: [Errno 17] File exists: '/home/wiki/data'
realms    | Traceback (most recent call last):
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 507, in spawn_worker
realms    |     worker.init_process()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 192, in init_process
realms    |     super(GeventWorker, self).init_process()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 118, in init_process
realms    |     self.wsgi = self.app.wsgi()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
realms    |     self.callable = self.load()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
realms    |     return self.load_wsgiapp()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
realms    |     return util.import_app(self.app_uri)
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/util.py", line 366, in import_app
realms    |     app = eval(obj, mod.__dict__)
realms    |   File "<string>", line 1, in <module>
realms    |   File "/home/wiki/realms-wiki/realms/__init__.py", line 218, in create_app
realms    |     app.discover()
realms    |   File "/home/wiki/realms-wiki/realms/__init__.py", line 83, in discover
realms    |     sources.init(self)
realms    |   File "/home/wiki/realms-wiki/realms/modules/wiki/__init__.py", line 11, in init
realms    |     Wiki(app.config['WIKI_PATH'])
realms    |   File "/home/wiki/realms-wiki/realms/lib/hook.py", line 15, in wrapper
realms    |     rv = fn(self, *args, **kwargs)
realms    |   File "/home/wiki/realms-wiki/realms/modules/wiki/models.py", line 35, in __init__
realms    |     self.repo = Repo.init(path, mkdir=True)
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/dulwich/repo.py", line 1006, in init
realms    |     os.mkdir(path)
realms    | OSError: [Errno 17] File exists: '/home/wiki/data'
realms    | [2018-03-11 14:55:00 +0000] [10] [INFO] Worker exiting (pid: 10)
realms    | Traceback (most recent call last):
realms    |   File "/home/wiki/realms-wiki/.venv/bin/gunicorn", line 11, in <module>
realms    |     load_entry_point('gunicorn==19.3.0', 'console_scripts', 'gunicorn')()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 74, in run
realms    |     WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 189, in run
realms    |     super(Application, self).run()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 72, in run
realms    |     Arbiter(self).run()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 201, in run
realms    |     self.halt(reason=inst.reason, exit_status=inst.exit_status)
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 297, in halt
realms    |     self.stop()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 214, in handle_chld
realms    |     self.reap_workers()
realms    |   File "/home/wiki/realms-wiki/.venv/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 459, in reap_workers
realms    |     raise HaltServer(reason, self.WORKER_BOOT_ERROR)
realms    | gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
realms exited with code 1

Apparently it doesn't work, because File exists: '/home/wiki/data'. I tried to use different folders (for example /home/wiki/wikidata) that ar not created in the Dockerfile. This works and I find the files inside this folder inside the container. But once I define them as volumes in docker-compose.yml I get the File exists: '/home/wiki/wikidata' Error again.

How do I do it the right way?

nFrechen commented 6 years ago

Ok, it works if I put the WIKI_PATH inside the persistent folder like so:

"WIKI_PATH": "/home/wiki/data/repo"

This way docker can create the folder /home/wiki/data and the realms worker can create /home/wiki/data/repo.

The data folder only needs to have the right permissions. Ich changed it this way:

chown 1000:1000 ./realms-data

Since 1000 is the userid of wiki inside the container this works. It might display a different user name on the host.