Docker container fails to start failing to find coffee-script module

Andrei-Paul commented 6 years ago

Steps to reproduce:

> git clone https://github.com/fedwiki/wiki.git $SOMEPATH
> cd $SOMEPATH
> docker-compose up -d --build

Expected Result: Have a container of _wikiweb image running.

Actual result: Container exited. Logs:

> docker logs $CONTAINERID
module.js:478
    throw err;
    ^

Error: Cannot find module 'coffee-script'
    at Function.Module._resolveFilename (module.js:476:15)
    at Function.Module._load (module.js:424:25)
    at Module.require (module.js:504:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/usr/src/app/index.js:3:1)
    at Module._compile (module.js:577:32)
    at Object.Module._extensions..js (module.js:586:10)
    at Module.load (module.js:494:32)
    at tryModuleLoad (module.js:453:12)
    at Function.Module._load (module.js:445:3)

Additional information: I receive the following during the build process:

npm WARN deprecated coffee-script@1.12.7: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)

WardCunningham commented 6 years ago

I wonder if it makes sense for this repo to even suggest one way to run wiki inside of docker. The docker ecosystem is much larger than our project and has developed its own standards and conventions. I would like to see various production ready configurations developed and containerized as separate projects. TLS is a good example of something looming over us for which there are many solutions. Login and persistence are two more variables. Backups and logging come to mind too.

Andrei-Paul commented 6 years ago

My background: I set up one of these pre-docker, 2 years ago, modified the welcome page and called it a ... 2 years. Now I want to have it as my main website, in a container, along with anything else that I want to experiment with. So I may really be missing the real point for why this project exists and it's design philosophy.

My path from here: The persistence part I would have solved for myself by mounting a volume, at worst by adding a few more command scripting when starting the container and some environment variables. For TLS I would have used another container (nginx) that took care of that, and at worst even more command scripting and env variables, maybe bundling nginx in. If any contribution I can bring on this side is useful, I'd be happy to provide, but it may just as well look like an abomination from outside my path.

My issue: Javascript (and whatever disguise it wears) and I really don't mix, to put it mildly (sorry?). I tried removing the '-' from coffeescript from packages.json, but I still got the warning and stuff still didn't work, maybe because of coffee-trace? I really have no actual idea what I'm doing with this part and I would invoke the philosophy that I should not need to know ASM/C/C++/Python/Perl/Whatever to install mysql/windows/whatever :)

WardCunningham commented 6 years ago

We are slowly moving toward more configuration being done from within wiki rather than by issuing shell commands before launch. There are a few things that must be known to make this work but these would be expected as environment variables as I understand is a docker best practice. One important environment variable would be the identity of the administrator who is granted configuration powers.

I do not speak from experience here so consider this speculation. I can imagine at least three off the shelf docker configurations.

small, single site install for one individual
large, wiki farm with reverse proxy and other community support tooling
app, building block instance to run within a fully managed environment

I would expect each of these to be available in docker image repositories and built from projects of their own with their own user communities. This could reasonably be part of the fedwiki github org but need not be.

I would expect this org to maintain key repos, wiki, wiki-server, wiki-client, as required to support inclusion in docker builds downstream. This would be a separation between coding in javascript and administration with command line tools captured in docker files.

I would not expect this org to be experts about the specific constraints of docker and the evolving best practices about working within them. We are interested, but not expert.

WardCunningham commented 6 years ago

Regarding coffeescript. I have a vague memory of installing this on my computer using instructions I found on the coffeescript site. It was something like npm install -g coffeescript. I'm not inclined to experiment here because I don't want to corrupt my own install. I understand this fear is addressed by docker once you have one that works and is maintained.

try installing coffeescript before building the docker image
try installing coffeescript from within the docker file
try adding coffeescript to the package.json

Perhaps someone watching will tell us the preferred approach.

Andrei-Paul commented 6 years ago

Found the issue: @line 12-13 in docker-compose.yml

  volumes:
- .:/usr/src/app

That basically mounts the current host directory over the app directory from within the image as specified at build time. That way, the node_modules directory created by the build process becomes hidden by the mount and index.js can't find the coffee-script module.

Andrei-Paul commented 6 years ago

Regarding possible future docker images, I would recommend, as a user seeking straight forward components with a reasonable learning curve, not an expert, the following:

A single image, 'flavors' separated by tags, as are versions:

> docker pull fedwiki/wiki # defaults to latest, and:
> docker pull fedwiki/wiki:latest | docker pull fedwiki/wiki:1.1 | docker pull fedwiki/wiki:simple | docker pull fedwiki/wiki:proxied # for whatever needs arise.

Containers should start with nothing specified, even if this only creates a 'demo' experience

> docker run fedwiki/wiki # Now I can go to http://localhost:3000 and see something. A 'something' can be improved, a 'nothing' cannot.

There should be a clean separation between application essentials and data If you mount your data dir, it should not hide the node_modules and break everything. 🤕 I guess this could also be addressed by clear instructions, i.e. "mount a config file to /usr/src/app/config.json and your data at /usr/src/app/data, if your config file wants ./data"

In my case, using a configuration mount and a data mount is enough to have my old welcome page and the owner set, so I have my old 'site' in docker now. 👍

Ideally, parameters should be exposed for any and all aspects of configuration and consistent precedence should be established (env vars > file config or the reverse)

WardCunningham commented 6 years ago

Well, you see that my advice is not of much value when it comes to docker. My own computers are insufficient to even start it. Do you think it possible that this configuration has ever run?

paul90 commented 6 years ago

The docker configuration here was added back in 2014, in d29177aed5fce54c68130ed067a7b516c0b4bdf3, to "create reproducible development environments and scenarios".

I am minded to simply remove the docker element from this repo, it has caused needless confusion on more than one occasion, and have a separate repo for docker. In fact several already exist, for example https://github.com/dobbs/wiki-tls .

dobbs commented 6 years ago

@Andrei-Paul, the docker ecosystem has definitely exploded in the past 4 years. There are a lot of lessons that have been learned since these first steps were taken in fedwiki/wiki.

I have been building https://github.com/dobbs/wiki-tls (as @paul90 already pointed to) with the benefit of more recent docker experience. That's got fairly recent docs in wiki over here:

https://local-farm.wiki.dbbs.co

I use the same code with different docker-compose.yml to host my own wiki. The differences between local and public are fairly big and not documented. But if you can read bash I think it's not insurmountable: https://github.com/dobbs/wiki.dbbs.co

In @WardCunningham's categorization this thing is in-between the first two bullets:

small, single site install for one individual
large, wiki farm with reverse proxy and other community support tooling
app, building block instance to run within a fully managed environment

It is bigger than a single wiki. It is a wiki farm. It includes a reverse proxy and TLS. But it is built for a single author and lacks tooling for supporting a community.

I'm gradually building roads toward more complete path to production, some of which will find their way into our "official" docs soon.

WardCunningham commented 6 years ago

medium, a collection of sites truly owned by one fully engaged individual.

almereyda commented 6 years ago

@paul90 I would second your idea to remove any docker specifics from the repository. Indeed it is at times convenient, if developers provide Dockerfiles to their projects for development. Yet these are also often mistaken for production ready environments, which they are not.

The diversity in Docker-based environments, starting with plain Docker, moving past docker-compose and looking at hybrids that build and orchestrate such containers, as CI/CD pipelines or "PaaS" like Dokku, allows us to anticipate the diverse set of requirements we are facing here. Together with the perceived user bases, we may give in and not support this at all in the core.

Coming from here the community could coordinate around increasing documentation around how to set up a build environment for development or production cases and provide exemplary configurations for those, given different user groups needs (proxy, or not, farm, or not, transporters, or not, customised security, or not, plugins, or not, ...).

As an example, the Dokku application which receives https://github.com/federated-wiki/dokku-wiki-distribution/blob/master/Dockerfile is configured to pass a selection of wildcard domains in nginx' special .sld.tld notation, and has a volume configured for /root/.wiki/, which persists the pages. This volume is initialised as a git repository and lazily commited and pushed to a remote from within the container.

This setup itself is based on a choice of plugins https://github.com/federated-wiki/docker-wiki-distribution/blob/master/Dockerfile, which are provided to extend an assumed standard way of setting up a wiki core https://github.com/federated-wiki/docker-wiki-core/blob/master/Dockerfile

@Andrei-Paul If you are comfortable with maintaining build pipelines and image maintenance, I would like to invite you to collaborate on reproducible and documented example wiki configurations for development and production use. These cases could both target Dokku and Docker-Compose as platforms each.

The core and distribution from above surface as automatically built images, syncing to the tags of the source repositories, at https://hub.docker.com/r/federatedwiki/base/ and https://hub.docker.com/r/federatedwiki/distribution/tags/

almereyda commented 6 months ago

In continuation of #111 and the availability of automatically built container artifacts at

it appears safe to close here.

Read the story about the build pipeline(s) in #142

fedwiki / wiki

Docker container fails to start failing to find coffee-script module #110