Science Museum Group: Collections Online

Getting started

1. Install Node.js version 18 or 20. If you have nvm, you can run nvm install to automatically install and/or use an appropriate node version.
2. Install dependencies: npm install
3. Copy .corc.template to .corc in the project route
4. Add required config values to .corc
5. Start the server: npm start

Or use npm run watch to rebuild and restart the server as you edit things.

Testing

Use npm test to run all the tests (including linting).

There are two types of tests that can be run: unit and end-to-end. These can be run individually by running npm run test:unit and/or npm run test:endtoend.

Unit tests

By default, npm run test:unit syncs fixtures from the live index so you need to have a correctly configured .corc file pointing at the latest elasticsearch server.

Offline

Use npm run test:unit:tape to run the unit tests using the existing fixtures provided in the repo.

End-to-end tests

The end-to-end tests require an instance of the application to be running on http://localhost:8000.

Directory structure

.
├── bin         # Executable(s) for starting the server etc.
├── lib         # Shared modules
├── public      # Public resources exposed by the server
├── routes      # API routes
├── schemas     # Joi schemas for input validation
├── templates   # Handlebars templates - layouts, pages, partials and helpers
└── test        # Unit and integration tests

Collections

Index types

The following main 3 document types are available in the index:

• Agent
• Object
• Archive
• MGroup

Other document types:

• Location (museum location / gallery)
• Event (historical)
• Place (geographical)
• Term (thesaurus term)

Display names

The name of the index types isn't always obvious to the public so on the site they are mapped as follows:

• People => Agent
• Objects => Object
• Documents => Archive
• MGroup => Group
• Location => Facility

Routes

To access the pages we currently have templates for, use:

• /
• /search?q={query}
• /objects/{id}/{slug?}
• /people/{id}/{slug?}
• /documents/{id}/{slug?}
• /group/{id}/{slug}

The routes will not work without params, however any random string will lead you to the example pages.

Deployment

We use Travis for CI and production deployment. The .travis.yml file in the root directory contains configuration for deployments.

Setup CI

Follow these steps to setup a new CI environment:

• Create an elasticbeanstalk Node.js app in the eu-west-1 region

  • Make a note of the application name and environment name (you'll need this later)

• Get the name of the S3 bucket it creates

  • It will look like elasticbeanstalk-eu-west-1-431258931377

• Create an IAM user smg-collectionsonline-deploy-travis

  • Save the access key and secret (you'll need this later)

• Create an IAM group smg-collections-online-deploy

• Add the following inline policy (named: DeployPolicy) to the group, substituting the resource values appropriately:

  {
      "Statement": [
          {
              "Effect": "Allow",
              "Action": [
                  "elasticbeanstalk:Check*",
                  "elasticbeanstalk:Describe*",
                  "elasticbeanstalk:List*",
                  "elasticbeanstalk:RequestEnvironmentInfo",
                  "elasticbeanstalk:RetrieveEnvironmentInfo",
                  "ec2:Describe*",
                  "elasticloadbalancing:Describe*",
                  "autoscaling:Describe*",
                  "cloudwatch:Describe*",
                  "cloudwatch:List*",
                  "cloudwatch:Get*",
                  "s3:Get*",
                  "s3:List*",
                  "sns:Get*",
                  "sns:List*",
                  "cloudformation:Describe*",
                  "cloudformation:Get*",
                  "cloudformation:List*",
                  "cloudformation:Validate*",
                  "cloudformation:Estimate*",
                  "rds:Describe*",
                  "sqs:Get*",
                  "sqs:List*"
              ],
              "Resource": "*"
          },
          {
              "Effect": "Allow",
              "Action": "s3:*",
              "Resource": [
                  "arn:aws:s3:::elasticbeanstalk-eu-west-1-431258931377",
                  "arn:aws:s3:::elasticbeanstalk-eu-west-1-431258931377/*"
              ]
          },
          {
              "Effect": "Allow",
              "Action": "elasticbeanstalk:*",
              "Resource": "arn:aws:elasticbeanstalk:eu-west-1:431258931377:*"
          },
          {
              "Effect": "Allow",
              "Action": "elasticbeanstalk:UpdateEnvironment",
              "Resource": "arn:aws:elasticbeanstalk:eu-west-1:431258931377:environment/Default-Environment/My First Elastic Beanstalk Application"
          },
          {
              "Effect": "Allow",
              "Action": "cloudformation:*",
              "Resource": "arn:aws:cloudformation:eu-west-1:431258931377:*"
          },
          {
              "Effect": "Allow",
              "Action": [  
                  "autoscaling:SuspendProcesses",
                  "autoscaling:ResumeProcesses"
              ],
              "Resource": "*"
          },
          {
              "Effect": "Allow",
              "Action": "elasticloadbalancing:*",
              "Resource": "arn:aws:elasticloadbalancing:eu-west-1:431258931377:*"
          }
      ]
  }

• Create a .travis.yml in the project root and add the following:

  language: node_js
  
  node_js:
    - '4'

• Install the Travis CLI tool

• Run travis setup elasticbeanstalk

  $ travis setup elasticbeanstalk
  Access key ID: AKIAIYE5GN7RNPZSZELA
  Secret access key: ****************************************
  Elastic Beanstalk region: |us-east-1| eu-west-1
  Elastic Beanstalk application name: My First Elastic Beanstalk Application
  Elastic Beanstalk environment to update: Default-Environment
  Encrypt secret access key? |yes|
  Deploy only from TheScienceMuseum/collectionsonline? |yes|

• Add the following to the travis config:

  • Zip up the built site before deploy

  before_deploy:
    - zip -q -x .git\* node_modules/\* -r collectionsonline *

  • Add the bucket name, zip file path and skip cleanup to the deploy section:

  deploy:
    bucket_name: elasticbeanstalk-eu-west-1-431258931377
    zip_file: collectionsonline.zip
    skip_cleanup: true

• In AWS elasticbeanstalk navigate to "Software Configuration" for the app

  • Add npm start as the "Node command"
  • Add config as environment vars to the new apps:
    • co_rootUrl
    • co_elasticsearch__host

Development Notes

Scripts

Some scripts need to be run whenever the index is updated. To do this run the command:

npm run index-update

Browser testing courtesy of BrowserStack