search

remipichon / manifmaker

MIT License
1 stars 1 forks source link

readme

PROJECT IS DEAD BECAUSE OF REASONS

(mostly because it's annoying to work 'alone', if you are interested to take ownership of it, use it, develop on it, let me know)

ManifMaker

What is for ?

ManifMaker is a single page web app aimed to plan and organize events where volunteers take a great part.

In a few words, organizers create tasks describing the job to be done, add time slot defining when the task has to be done and specify people needs to explicit how many and what kind of volunteers are needed to perform the task.

Volunteers register on the app to add a few availabilities and skills to detail when they want to work and what they can do.

Once the tasks and their needs are validated, organizers assign volunteers to tasks according to :

Live Demo

You can find a live demo here.

Continuous Deployment build status

branch deploy : Build Status

Table of Contents

This TOC has been generated using 

docker run --rm  -v $(pwd)/:/root/ meedan/base gh-md-toc /root/README.md

Installation

The project relies on Meteor, a full stack single page app framework with real time capabilities.

Windows User : you need to install Git if you don't already have it

Windows User : if Meteor is not a known command, add meteor to your path. Meteor binary can be found here C:\Users\YOU\AppData\Local.meteor

Edit Windows path

Dev tools

Dev tools that are already installed and available to be used when implementing cool features.

Quality

Auto generated Doc

JSDoc is used to generate doc from annotations on code. The generated doc is available as Markdown in the repo /doc/markdown or as HTML in the stagging machine (NOT DEPLOYED ANYMORE).

The HTML doc is automatically build and deployed (DISABLED), see Continuous Deployment section. The Markdown doc has to be build and commit/push when it's relevant.

HTML

JSDoc Github

npm install jsdoc -g
npm run doc:html

If you don't have npm globally installed, you can use the one provided by meteor. Add 'meteor' before npm command to do so.

Open doc/html/index.html in a browser.

Markdown

doc jsdoc-to-markdown

npm install jsdoc-to-markdown --save-dev
npm run doc:md

If you don't have npm globally installed, you can use the one provided by meteor. Add 'meteor' before npm command to do so.

Generated in /doc/markdown

Design and UI tools

Material design icon

<i class="mdi mdi-home"></i>

Icon definition can be found here : https://materialdesignicons.com/.

CSS Classes

Some useful classes implemented in css :

.clickable                  cursor is a hand over this element ;
.hide-on-small-devices      the element is only displayed on large devices
.hide-on-small-devices      the element is only displayed on large devices

Alert

User friendly alerting use s-alert. You basically only need

Error
sAlert.error('Your message');
Warning
sAlert.warning('Your message');
Info
sAlert.info('Your message');
Success
sAlert.success('Your message');
Error

Alert box will be displayed 2.5 seconds, if 'Your message' if too long to be read in 2.5 seconds you can override it with (in ms) :

sAlert.error('Your message',{ timeout : 60000 });

Confirm and Prompt

BootBox has to be used to display a confirmation box or a prompt box. 

 bootbox.confirm("Are you sure ?", function(result){
                if(result){
                    //user was sure
                }
            });

Do not use alert or custom dialog features as S-Alert is the preferred way.

CustomSelect

A powerfull custom selector is available. It is largely inspired by Github selector and provides following features :

You can refer to the auto-generated doc select-component.md and the live demo (need to be logged in): /demo-select, or localhost:3000/demo-select.

Data management

Add a reference collection

What is a reference collection ?

A Reference collection is used when the user as a choice between a set of editable values. Typically, you will need a reference collection with form field using a Custom Select (Teams, Places or Group Roles can be dynamically edited while being available in a select). All reference collection are editable in a page (/conf-maker) linked to a role CONFMAKER.

Each reference collection provides a set of features :

The good news is that a few configuration is needed to get all those features out of the box.

define a schema

Add the schema to /both/collection/schema/CollectionReference.js. It will create Schema and Mongo Collection and generate every needed routes)

See Schemas.references.Teams for a minimal collection reference example.

Let's say you want a collection (eg: Equipment) to reference another reference collection (eg: EquipmentCategory) to allow a link between the two (eg : Equipement refers to a EquipmentCategory)

CollectionName_Id  : eg  EquipmentCategories_Id

It will display the "name" field of the reference collection (eg: EquipmentCategory) in the insert/uptate form (eg:Equipment).

Basic references field with custom verification that the _id actually exists and autoform to generate the dropdown

# eg: EquipmentCategory contains a list of Equipment
EquipmentCategories_Id: {
        type: SimpleSchema.RegEx.Id,
        label: "Equipment Category",
        custom: function () {
            if (!EquipmentCategories.findOne(this.value))
                return "unknownId";
            return 1
        },
        autoform: {
            afFieldInput: {
                options: Schemas.helpers.allEquipmentCategoriesOptions
            }
        },
    },

some magic conf

Please note that you need to add the following fields to have the "update" button working (sorry...)

baseUrl: { 
        type: String,
        label: "Team base URL",
        defaultValue: "team"
}

Please note that you need to add the following fields to have the "remove" button working (sorry...)

 type: { 
        type: String,
        label: "Teams type",
        defaultValue: "Teams"
    },

configure routes

Add the newly created Mongo Collection to the AllCollections array in /client/routes/config/route-collection-references.js.

some templates

Add your specific template in /client/templates/references/ (just copy/paste and update the existing templates to your needs. Be careful with singular and plural to have everything correctly generated)

publish and subscribe

and add your new Collection to publish/subscribe policy

You should follow the current populate/clean policy

tests

If everything went well :

Keep on eye on the consoles (server and client) and fix all errors.

Troubleshootings

Carefully check singular and plural and that your naming is similar to the existing (use Team as an example).

Data test

In dev mode (when app is run using meteor run), you can inject data by using the URL :

Details regarding authentication data can be found here :

Super Admin user

A super admin user (superadmin/superadmin in dev mode) is created at startup no matter what. This user has all existing roles, it can't be updated or removed and doesn't have to be used for anything else that injecting data (dev mode, stagging) or create user with roles (production, at least one admin user with ROLE role to add roles to other users).

Security

Access Right Security uses alanning:roles.

Thoses following verifications are done (and every new features should uses all these verifications) :

Data integrity

Following is a long pamphlet about data, you don't normally want to read it

Data validation is done inside the schemas (both/collection/model). Simple Schema provides common validations like date, string, int. Other validation can be done in custom methods with custom code to validate dates overlapping, fields updates according to validation state...

If a single operation/action needs several atomic database update, all validation has to be done explicitly BEFORE updating the data. If the validation is done one times (in the operation/action), no need to put it in the schema, just do it along with the action/operation. If not and you have to put the validation in the schema and use pre validation from Simple Schema BEFORE updating any data. This way you ensure that the data updates will not fail.

That is because of one thing : Meteor+MongoDB is not transactional. Indeed, if one of the database update fails because of a custom control that throws an error, the previous database updates will not be reverted and the following could occurs if you don't use a callback or manage the update return by yourself.

Lets take two example : update an assignment term and perform an assignment :

Updating a assignment term uses ONE atomic database. An update on AssignmentTerms collection. The controls can be performed on the related schema without any side effects. If the requirements are not satisfied, the update will just fail and the error can be displayed to the user.

Performing an assignment used THREE atomic database update (actually SIX but let's simplify the example) :

(Let's cut it short : it could have been prevented by another data design where the assignment information is only store in one place (Assignments) instead of having the data copied in the Task and the User.)

From here, two philosophies to take into account whether you agree or not to :

"it's easier to ask forgiveness than it is to get permission" which can be explained by "try to do it and if it fails, repair it"

When assigning, either you firstly check everything (user is available, task is ready, people need specs matches the user) and if it's ok, you perform the assignment or you perform the assignment and revert it if something failed. When choosing what to do you have to keep in mind that Meteor is real time, if you update something on the DB, it will be broadcasted to everyone subscribed. If you update something and revert it right away, you will unefficiently use DDP, the clients will compute the data and probably display something for a short amount of time before the sytem reverts the changes. It can lead the GUI to flickr. That is why it is probably better to check everything BEFORE database operations if you need more than one database update to perform one operation/action).

JWT

Javascript Web Token can be used to sign Json payload into a string token that can be sent/shared and can only be read by the app (like to create a one time login token)

Json Payload (not generic yet)

PDF Export

A simple Docker image with wkhtmltopdf installed

Cmd t tests

OUTPUTL_FOLDER=/Users/remi/sandbox;
PDF_FILE=output.pdf;
IN=192.168.192.4:3000/jwt/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0YXJnZXQiOiJodHRwOi8vbG9jYWxob3N0OjMwMDAvdXNlci9uRHd3UnlQYnVDWlo4UTZwQS9leHBvcnQiLCJ0eXBlIjoidXJsIiwiaWF0IjoxNDg5NTI4NTgzfQ.DF98Qq7jqWK_qYcPL5JU0wrY97soU2JRb22S2_b-q7M
docker run --rm -v $OUTPUTL_FOLDER:/root/out/ --env IN=$IN --env OUT=/root/out/$PDF_FILE assomaker/wkhtmltopdf

Node PDF Export

Export Node PDF docker pull assomaker/wkhtmltopdf image at startup

Cmd to test (from repo root folder)

OUTPUTDIR=$(pwd)/exported-pdf
# build node app
docker build -t assomaker/export_pdf production/export-pdf-node
# use node app in dev mode (with code in shared volume)
docker rm -f nodeexport; docker run --env OUTPUTDIR=$OUTPUTDIR --network host -v /var/run/docker.sock:/var/run/docker.sock --name nodeexport -p 3030:3030 -d -v $(pwd)/production/export-pdf-node:/root --entrypoint="" assomaker/export_pdf tail -f /dev/null; docker exec -ti nodeexport sh
$ cd /root/app/; npm install; node app.js
# use node app in normal mode (code in image)
docker rm -f nodeexport; docker run --env OUTPUTDIR=$OUTPUTDIR -v /var/run/docker.sock:/var/run/docker.sock --name nodeexport -p 3030:3030 -d  assomaker/export_pdf; docker logs -f nodeexport
# Nginx to serve file
docker rm -fv nginx; docker run --name nginx -p 8080:80 -d -v $OUTPUTDIR:/usr/share/nginx/html/pdf nginx

JSON Export

An endpoint is available to retrieve data as Json: api/:resource/:action.

Available resources and actions can be found at ApiResourceActionService.js#L4

Add/Update JSON Export

note that newKey can use the dot notation

Ops tools

Environment variable

For Dev/Preprod:

DATA_INJECT_ONCE

Whatever data will be added only once, even if ManifMaker app is restarted.

DATA_INJECT_EVERYTIME

Whatever data will wiped out for each anifMaker app restart.

DATA_INJECT_CLASS_ALL

Which of the defined InjectDataServerService to use to inject various data.

INJECT_MINIMUM_ACCESS_RIGHT

Inject Roles define in Roles enum, add a superadmin group roles (not updatable) and a superadmin user (not updatable).

Superadmin user has "superadmin" password in Development and a random one in Production. Superadmin password can be found in the app log when starting.

For Production:

SKIP_INIT_ACCESS_RIGHT

Skip injected init access right (superadmin user and roles) even if it has never been injected. Either the roles and superadmin users are already there or you will seed your database differently (not recommended).

IT_IS_NOT_PRODUCTION_IT_IS_OK_TO_DELETE_DATA

Force isDevelopment mode to allow wiping out and injected data. It is only valid at startup, 'isProduction' remains true for the app lifecycle. Value should be 'truetrue', just because we don't want to easily allow deleting data in production.

IT_IS_REALLY_NOT_PRODUCTION

Value should be 'iknowwhatiamdoing' to enable usage of IT_IS_NOT_PRODUCTION_IT_IS_OK_TO_DELETE_DATA. The usage for both env is not being explained in the logs.

MAILGUN_PASSWORD
MAIL_URL

If MAILGUN_PASSWORD, MAIL_URL will be set

JWT_PRIVATE_KEY

Key use by JWT to sign a Json payload. If JWT_PUBLIC_KEY is not set, a random secret will be generated and used to both sign and verify (way less secure)

JWT_PUBLIC_KEY

Key use by JWT to verify a Json payload. If JWT_PRIVATE_KEY is not set, a random secret will be generated (way less secure)

EXPORT_PDF_ENDPOINT

Where the Node app can be reached. the ManifMaker app is POSTing to this endpoint when PDF need to be generated.

NGINX_ENDPOINT

Where the Nginx serving the PDF can be reached. Currently no authentication whatsoever.

MANIFMAKER_ENDPOINT

Where the ManifMaker can be reached. It used by assomaker/wkhtmltopdf to load the HTML page that needs to be exported as PDF

GOOGLE_CLIENTID and GOOGLE_SECRET

See https://console.developers.google.com/apis/credentials

FACEBOOK_APPID and FACEBOOK_SECRET

https://developers.facebook.com/docs/apps/register/

Setup environment (production and preproduction available)

https://docs.docker.com/engine/installation

https://docs.docker.com/compose/install/

__777 on ~/manifmaker_images seems to be required by Fs Collection to store image, it didn't even work with 666. It can be a security flaw as we are giving exec access to a volume shared in a Docker__

Build specific version

  "version": "0.3.0",

You need to have matching version number between docker-compose-preproduction.yml and package.json to deploy the version you just built.

Update deployed version

Current update policy provokes a service interruption as there is only one ManifMaker node. Following steps should get easier one day.

Make sure the version you are updating to is available on the Docker hub.

Backup data

A Mongo level backup is run everyday at midnight, it backups all /manifmaker database.

Restore from a backup

See the list of backups, you can run:

   docker exec mongodb_backup ls /backup

To restore database from a certain backup, simply run:

docker exec mongodb_backup /restore.sh /backup/2015.08.06.171901

It will delete everything (--drop) and restore whole /manifmaker database.

Force a backup by hand

docker exec mongodb_backup /backup.sh

Continuous Deployment

Travis CI is used to achieve Continuous Deployment.

When a push occurs on branch deploy :

When a push occurs on branch production :

Stagging old conf to build Doc

Project Management

We are using the Github issues enhanced with Zenhub product which I recommend to install.

Our specs and manuals tests are written in a GDoc, ask me if you want access to it.