search

nodezoo / nodezoo-workshop

A microservices workshop for the Seneca framework.
http://senecajs.org
MIT License
185 stars 111 forks source link

readme

Nodezoo

nodezoo-workshop

A workshop for the nodezoo project. Nodezoo is a search engine for Node.js modules. The nodezoo search engine is an example of a real-world service built using Node.js microservices. Each microservice is published in its own github repository along with all of the necessary config to run the system locally or live . The codebase is intended to be used as an example, and as a starting point for your own projects.

Below we provide a complete workshop to work through. Our current live system has it's roots in this workshop. By working through the iterations below you can get a feel for how a microservice system is bootstrapped together and how the system evolves as needs change.

Note: This repo contains the nodezoo workshop, to explore and run the live version of nodezoo, please see nodezoo-system project.

Microservices

The micro-services that make up the system are:

Each service should be downloaded and placed in the same folder including this repository.

Iterations

The system is built in a set of iterations so that you can follow its development. This mirrors the way that real microservice projects are developed. Each iteration, more services are introduced.

When working with the individual microservices, it is easier to open a separate terminal for each one.

Not all microservices are available in all iterations, as some are only introduced later.

Each iterations contains a set of tasks to execute. You should try to get them all up and running to verify to yourself that you understand the mechanics of the system.

Each iteration also includes a set of experiments that you can attempt. Use these to develop your understanding of the system - there are no right answers!

Requirements

The basic tools are:

Install these before getting started. Later iterations introduce additional tools, and these will be indicated.

A Note on Docker

This example only places microservices into containers. All other services (e.g. redis) are run as normal from the host machine. This does not prevent you from containerising them of course!

To use the docker command in a terminal, you need to set up the docker environment variables. From the initial docker terminal (as provided by the docker installation), you can run

$ docker-machine env default

to obtain these environment settings. Copy and paste them into new terminals as needed.

Docker runs containers in a host machine. You use the IP address of this host to access containers. The easiest way to get this IP address is to run the command:

$ docker-machine ip default

Finally, from inside docker containers, your microservices will need to talk to the outside world. To do this, they use a special IP address representing your host machine (Host IP). You can obtain this address in multiple ways:

Docker networking can be tricky, and is fragile with respect to network changes, with DNS, for example, failing. When wierdness happens, your best bet is to restart:

$ docker-machine restart default

This will invalidate your environment, so you will need to launch a new docker terminal.

How to use this code

Each microservice repository has a branch for each iteration: i00, i01, etc. You can clone these branches directly - for example:

$ git clone -b i00 https://github.com/nodezoo/nodezoo-web.git nodezoo-web-i00

However you will not be able to save your changes to your own repositories.

To save your own work, it is better to first fork the repository on github.com, and then

$ git clone https://github.com/[YOUR_USER]/nodezoo-web.git
$ cd nodezoo-web
$ git remote add upstream https://github.com/nodezoo/nodezoo-web.git
$ git fetch upstream
$ git checkout upstream/i00
$ git checkout -b i00

This sequence of commands downloads the branch into your local clone of your fork. You can then push your changes back to your own fork.

One you have downloaded all the branches, you can switch between them, across all microservice repositories using the iteration.sh script:

$ ./iteration.sh i00 # moves all to iteration 00
$ ./iteration.sh i01 # moves all to iteration 01
... etc.

These commands must be used before using the above script, for each branch for the first time :

$ git checkout upstream/[BRANCH NAME]
$ git checkout -b [BRANCH NAME]

Install your dependencies

This script only works once the branch has been fully set-up for a first time.

In each branch, you always need to run the following command:

npm install

Then go into the folder nodezoo-workshop/system and run:

npm install

to get the dependent Node.js modules. This must be done each time a branch is changed for each micro-service.

Run build

In the folder nodezoo-web use the following command :

npm run build

IMPORTANT NOTE: the build command is not required on branch i00 - i05

Iteration 00: Getting Started

Branch name: i00

This branch starts with a simple web server. Use this branch to validate your configuration.

microservices

tasks

experiments

Iteration 01: 3 Microservices

Branch name: i01

This branch introduces two microservices that support the web service. Both are stubs that perform no actual work, instead returning hard-coded responses. The focus here is on understanding how simple microservice communication is configured using static addressing with fixed IP addresses and ports.

microservices

tasks

experiments

Iteration 02: Real Functionality

Branch name: i02

This branch introduces infrastructure services that are used by the microservices to perform work. Elasticsearch is used as a search engine, and Redis is used for publish/subscribe messaging. The search can now index and search for Node.js modules, with some manual help.

Prerequisites

microservices

supporting services

tasks

experiments

Iteration 03: Measurement

Branch name: i03

This branch uses influxdb and grafana to chart message flow rates through the system. Influxdb is used due to it's ease of installation and because it is based on plotting time-series data. Grafana is used because it officially supports influx, and is relatively easy to use.

Prerequisites

> CREATE DATABASE seneca_msgstats;
> CREATE USER msgstats WITH PASSWORD 'msgstats';
> GRANT ALL ON seneca_msgstats TO msgstats;

microservices

supporting services

tasks

experiments

Iteration 04: Enhancement

Branch name: i04

This branch shows the use of a message bus to avoid the high coupling and configuration costs of direct service-to-service communication. This is one way to avoid the need for service discovery solutions.

Prerequisites

microservices

supporting services

tasks

experiments

Iteration 05: Mesh Networking

Branch name: i05

This branch shows the use of mesh networking to completely remove the need for service discovery. The seneca-mesh plugin uses the SWIM gossip algorithm to enable microservices to automatically discover the appropriate destinations for messages dynamically.

Prerequisites

microservices

supporting services

tasks

experiments

Contributing

The NodeZoo org encourages open and safe participation.

If you feel you can help in any way, be it with documentation, examples, extra testing, or new features please get in touch.

License

Copyright (c) 2014-2016, Richard Rodger and other contributors. Licensed under MIT.