NVM is recommended for managing NodeJS installations and we are intending to stick to the LTS releases of NodeJS for this project.
After installing nvm
, this sequence of commands will provide you with an LTS
version of nodejs and access to yarn
:
$ nvm install
$ nvm use
$ corepack enable
$ yarn --help
Note: the .nvmrc
file in this repo determines which node version is used locally.
At present the application is known to be compatible with
nodejs versions 10.x, 14.x and 16.x. For information about
which version is deployed on Heroku for the staging and production
server, see the engines
section in package.json
. Read here for
more info.
cd server && docker-compose up
The database can then be accessed using:
psql postgresql://postgres:password@localhost:5432/server
Redis is used for caching. See REDISCLOUD_URL
in server/.env.example
.
You can connect to redis using the redis-cli
:
cd server && docker-compose exec redis redis-cli
Based on server/.env.example
, create some server/.env
file with appropriate values.
yarn
.yarn dev
. This runs in parallel the node server
and watches/builds code in the worker
(used for sending emails at the moment).This method is useful if you don't want to run the watch process. The tradeoff is that you must manually restart the command after making code changes. The example below shows how to increase the log verbosity for certain parts of the app.
$ cd regen-server
$ DEBUG='express:*,express-sharp*' yarn start
Need verbose logging? Run this SQL command in psql
or similar:
# the line below adjusts log level of the postgres daemon to include a lot of info..
alter database regen_registry set log_statement = 'all';
# the line below adjusts the client (psql) to display level debug or higher messages..
set client_min_messages to 'debug';
See the postgres errors and messages documentation for more info.
We make use of postgraphile to autogenerate a graphql API. Here are some references for understanding how to work with postgraphile:
Refer to migrations/README.md.
The following section describes how to seed your local database with production data in order to facilitate local feature development and testing.
Make sure the production database and your local database are in sync with regards to migrations by verifying the latest migration version number on master
and on your local branch in sql
folder. Otherwise that may cause unexpected behavior when trying to seed your local database with production data.
It's also recommended to start from a local database without any data, otherwise existing data might conflict with production data which could lead to constraint errors (e.g. unique constraint error in the case of multiple user
s with the same email).
Export data from the production database using pg_dump
:
pg_dump -d postgres -h registryproduction.cna6zybeqdns.us-east-1.rds.amazonaws.com -p 5432 -U postgres --file dump --data-only -F c
You'll be asked for the database password, if you don't know where to find it, please contact one of the contributors of this repository.
After entering the password, this might take a few seconds before data is exported into dump
.
postgres
for your local database):pg_restore -d regen_registry -h localhost -p 5432 -U postgres dump
If for some reasons, your database is in a messy state and it's best to just restart from scratch, you may want to drop the database and recreate it.
Make sure your postgres Docker container is running and then run:
yarn run graphile-migrate reset --erase
And you're ready to go again!
Jest is used for testing:
yarn test
Right now, it's using the development database. TODO: Use a separate testing database instead and set up new migration command.
This repo is configured to run prettier with eslint, in addition to some other common linting rules for eslint and eslint typescript. The setup was created by following the eslint getting started as well as the prettier docs for integrating with linters.
To run the linter and formatter for the whole project:
yarn run eslint .
To run the linter and formatter for the whole project, only showing "error"s:
yarn run eslint --quiet .
To run the linter and formatter for a specific file:
yarn run eslint <path-to-files>
To automatically apply fixes where possible use --fix
:
yarn run eslint <path-to-files> --fix
We make use of swagger-ui-express and swagger-jsdoc to generate documentation
for certain API endpoints within the routes folder. Any openapi specifications
that are found within routes in jsdoc style comments will be automatically
converted into swagger-style API documentation. i.e. search for @openapi
to
find samples of openapi docs that we use in the project. This tutorial covers
the basic concepts involved in writing an openapi specification.
SHACL schemas have been migrated to https://github.com/regen-network/regen-registry-standards
These graphs can be stored too in the PostGres database in the schacl_graph
table in order to be queried using GraphQL and used for client-side validation.
The schacl_graph
table has an uri
as primary key and a jsonb column graph
where a SHACL graph is encoded as JSON-LD.
For instance, an entry with http://regen.network/ProjectPlanShape
as URI can be created to store the SHACL graph to validate a project plan.
IRIs can be generated based on JSON-LD data using the following command:
cd iri-gen && yarn gen json_file_path
IRIs can be generated and inserted to the database using the --insert
flag. Note: You need to create an iri-gen/.env
when using this flag for
the staging or production environment. See iri-gen/.env-example
.
cd iri-gen && yarn gen --insert json_file_path
If you need to deploy the app manually, you can download the Heroku CLI and use the following commands to deploy from your local environment.
$ heroku login
Optional: Test the above by viewing the apps available to the our team.
$ heroku apps --team=regen-network
=== Apps in team regen-network
...
regen-server
regen-server-staging
...
If you are deploying to staging:
$ git checkout dev
$ git pull
If you are deploying to production:
$ git checkout master
$ git pull
If you are deploying to staging:
$ heroku git:remote -r regen-server-staging -a regen-server-staging
If you are deploying to production:
$ heroku git:remote -r regen-server -a regen-server
Your git remotes should like this:
$ git remote -v
origin git@github.com:regen-network/regen-server.git (fetch)
origin git@github.com:regen-network/regen-server.git (push)
regen-server-staging https://git.heroku.com/regen-server-staging.git (fetch)
regen-server-staging https://git.heroku.com/regen-server-staging.git (push)
regen-server https://git.heroku.com/regen-server.git (fetch)
regen-server https://git.heroku.com/regen-server.git (push)
To finish a deploy to staging, we need to push the dev
branch:
$ git push regen-server-staging dev:main
To finish a deploy to production, we need to push the master
branch:
$ git push regen-server master:main
Note: When you are deploying a branch, Heroku requires it to be <branch>:main
.
For more on this manual process, see the Heroku docs.