levibostian / ExpressjsBlanky

Blank Express.js project to get up and running FAST.
MIT License
7 stars 0 forks source link

node version for project: 12 (LTS) language used for project: typescript databases used for project: postgres and redis technologies used to deploy project. For the app, kubernetes. For database, debian VM server

ExpressjsBlanky

Boilerplate project for Expressjs apps. The template that I use for the apps that I build.

iOS developer? I have a boilerplate project for you! Android developer? I have a boilerplate project for you!

What are the goals of ExpressjsBlanky?

ExpressjsBlanky has been modified over years of building nodejs rest APIs. Through experience, you are guaranteed to find many annoyances and bugs. After each encounter, some engineering work is done to help remove that annoyance and prevent that bug from happening again.

ExpressjsBlanky comes equipped with the following goals:

Getting started

git clone https://github.com/levibostian/ExpressjsBlanky.git NameOfYourNewApp
cd NameOfYourNewApp
rm -rf .git/
git init
git config user.email "you@example.com"
git config user.name "First Last"
git add .; git commit -m "Initial commit. Created project from levibostian/ExpressjsBlanky boilerplate.";
npm install

Enjoy!

What is included in ExpressjsBlanky?

Go ahead and explore the source code! No need to include all of the details here, but here is a gist of the major components of this project:

Services

This project uses a list of various services to send push notifications, emails, run a CI server, and more. To keep the code base simple, keep the environments close, and avoid runtime complexity/bugs, all of these services are configured with environment variables all defined with a .env file.

The first thing you need to do is create a .env file.

Note: This project relies on the CLI tool, cici for helping with the environments. You may want to read up on the README of the project to understand how it's used in the scripts. Especially used on the CI server.

cp app/.env.example app/.env

Now, let's go into each of the variables, enabling the various services as we go on.

Next, we will use some AWS services.

  1. If you want to use your own custom domain name for your dynamic links, go to the Firebase Console to add your custom URL.
  2. If you use your custom domain or you use the generated URL that Firebase makes for you in the Firebase Console, you need to copy that URL and paste it in: app/constants/index.ts. Edit login > dynamic_link_url You will also want to add your Android package name and/or your iOS package name in the constants file if you have an Android or iOS app.

All of the query parameters you want to add to the dynamic link can be found in these docs.

CI server

Oh, you need a CI server. This project is all setup, but you need to setup the Travis CI service.

Development

Do you wish to build and run this Expressjs application locally on your machine for development? Let's do it.

To develop with ExpressjsBlanky, you need Docker and nvm installed. That is it!

nvm use
npm install
npm run dev:setup
npm run dev

Note: Make sure to create a .env file with your environment configuration you want to use for development.

This will install everything you need and startup the application. Simple!

To debug your code, this project is setup to work with VSCode's built-in debugger. All you need to do is run the "Local development debug" task in VSCode and done! It will even reset the debugger and recompile your code on code change.

Tests

This project is setup to create and run unit and integration tests against your code base super easily. For information on how to write tests, check out the tests/ directory.

npm run test:setup
npm run test

This will run all of your tests: unit and integration.

It's recommended that while you are developing your tests you use the Jest or Jest Runner VSCode extension for when you want to run or debug individual tests.

Deploy

npm run db

Note: This project is not setup with database backups. That is a different project.

I prefer to use AWS API Gateway. Create a AWS API Gateway as a HTTP reverse proxy that connect to your server with: http://ip-address:5000.

npm run redis
Type Port
TCP 222 (for ssh)
TCP 5432 (postgres)
TCP 5000 (for your nodejs app)

This project is setup to using a CI server to deploy your application for you. In order for the CI server to deploy your app, it must contain your environment variables. Since we don't want to store these files in the git repo for security and privacy reasons, they are stored in Travis as secrets.

The CI server should successfully use the tool cici to decrypt the encrypted files for your deployment. Make sure to encrypt the files locally on your machine before you deploy.

Follow this GitHub workflow to run a deploy.

  1. Get all of your pull requests into the master branch that you want in the release. Make sure you updated the Versionfile and CHANGELOG.md with your update information.
  2. Make a git tag. If for the staging environment, make X.X.X-staging. If prod, make X.X.X tag. Replace X.X.X with version in Versionfile.
  3. Git push. The CI server will run.

DB migrations

Note: You must create a migration for all database operations including the very first initial database create. It's highly recommended to not use sequelize.sync() (including in development/testing) to avoid production runtime issues.

Database backups

Database backups are automatically run on a schedule via cronjobs. The backup files are stored in a DigitalOcean Space (aka: object stored like AWS S3). Checkout the section in these docs on setting up Spaces.

// TODO. Talk about setting up automated backups here.

Object stored

Cloud object stored (aka: AWS S3) products are super handy ways to store files. In this project, we use cloud object storage to store assets for the apps (images, files), backup files such as logs and database backups, and possibly more.

DigitalOcean Spaces is a great cloud object storage solution because we use DigitalOcean for everything already, and it's backed by a CDN for fast and affordable asset storage.

Spaces is charged \$5 per Space instance created. Each space can dynamically grow in space needed. Because of that, this project tries to use only 1 Space instance in the DigitalOcean account to save on price.

Because we want to use 1 Space instance to store many different types of data, it takes some organization to manage everything. Each type of file that we store has different requirements.

Public asset requirements:

Backup files requirements:

When you create a new DigitalOcean Space for your API project, this is how it should be setup:

  1. Location - Up to you. Because we will have the global CDN enabled, location does not matter as much. NYC3 is a good choice as it's the default.
  2. Enable CDN. Set the TTL as long as you feel comfortable. The DigitalOcean API allows you to purge the CDN whenever you want so if you decide to update your assets, you can always purge the CDN.
  3. Restrict file listing. No use for it and for security reasons, you can disable it.

Public assets

I am assuming that you do not have many public assets to store. If that's the case, you can upload them and manage them using the DigitalOcean console webapp. Create a directory public-assets with subdirectories inside organizing the many assets. Select the files after you upload them and set them to be public. All of this can be managed from the console.

Private assets

Private files such as database backups are backed by lifecycle policies because we want files to expire automatically to save on cost.

The convention that we follow is to create a directory in your Space called 30day_backups, for example, which say how long the policy is for easy reference. Lifecycle policies are not managed with the DigitalOcean web app console so it's really easy to not understand there is a lifecycle set and mess something up. But because you are putting your files in a directory called 30day_backups, you know that those files will only live for 30 days!

Create directories in the Space for these private files. XXday_backups (example: 30day_backups) and inside that make subdirectories like database or logs.

Create an API token specifically for DigitalOcean Spaces. Keep the token a secret! It has full admin access to the entire Space.

Then, we need to set the lifecycle policy on the Space. One has already been created. See docs/spaces_lifecycle_policy.xml and examples to create one. Once it is created, it's time to set the policy on the Space. Note that only 1 policy can be set on the entire Space. You can set as many rules as you want in the policy.

To set the policy, you need to have s3cmd installed (brew install s3cmd works). Most to all of the operations that you want to perform on Spaces are done with the s3cmd CLI tool. s3cmd is defaulted to work with AWS S3. To work with Spaces, you need to add --host-bucket="%(bucket)s.nyc3.digitaloceanspaces.com" --region=US --host=nyc3.digitaloceanspaces.com as arguments to s3cmd.

Set lifecycle:

s3cmd --host-bucket="%(bucket)s.nyc3.digitaloceanspaces.com" --region=US --host=nyc3.digitaloceanspaces.com --access_key=$DIGITAL_OCEAN_SPACES_ACCESS_KEY --secret_key=$DIGITAL_OCEAN_SPACES_ACCESS_SECRET setlifecycle docs/spaces_lifecycle_policy.xml s3://name-of-space

Get lifecycle that is set:

s3cmd --host-bucket="%(bucket)s.nyc3.digitaloceanspaces.com" --region=US --host=nyc3.digitaloceanspaces.com --access_key=$DIGITAL_OCEAN_SPACES_ACCESS_KEY --secret_key=$DIGITAL_OCEAN_SPACES_ACCESS_SECRET getlifecycle s3://name-of-space

Documentation

This API is uses apidoc for API documentation.

It is pretty easy to use. View the official docs on the templating language and generate them using npm run generate:doc. Docs are generated in the api_doc/ directory.

Author

Levi Bostian image

Contribute

ExpressjsBlanky is not open for contributions at this time. This project showcases my preferred way to build a Expressjs app. If you wish to make edits, by all means, fork the repo and make your edits in your own copy of the repo.