search

gbv / jskos-server

Web service to access JSKOS data
https://coli-conc.gbv.de/api/
MIT License
6 stars 4 forks source link
coli-conc jskos service

readme

JSKOS Server

GitHub release API Status License Docker Test standard-readme compliant

Web service to access JSKOS data.

JSKOS Server implements the JSKOS API web service and storage for JSKOS data such as controlled vocabularies, concepts, and concept mappings.

Table of Contents

Install

Docker

The easiest way to install and use JSKOS Server is with Docker and Docker Compose. Please refer to our Docker documentation for more information and instructions.

Dependencies

You need Node.js 18 or Node.js 20 (recommended) to run JSKOS Server. You need to have access to a MongoDB database (minimun v4; v6 or v7 recommended).

Clone and Install

git clone https://github.com/gbv/jskos-server.git
cd jskos-server
npm ci

Configuration

You can customize the application settings via a configuration file. By default, this configuration file resides in config/config.json. However, it is possible to adjust this path via the CONFIG_FILE environment variable. Note that the given path has to be either absolute (i.e. starting with /) or relative to the config/ folder (i.e. it defaults to ./config.json). Note that the path to the configuration file needs to be valid and writable because a namespace key will be generated and written to the file if it doesn't currently exist. Note that if the file exists and contains invalid JSON data, JSKOS Server will refuse to start.

Currently, there are only two environment variables:

You can either provide the environment variables during the command to start the server, or in a .env file in the root folder.

It is also possible to have more specific configuration based on the environment. These are set in config/config.development.json or config/config.production.json. Values from these files have precedent over the user configuration.

All missing keys will be defaulted from config/config.default.json:

{
  "verbosity": "warn",
  "baseUrl": null,
  "env": "development",
  "title": "JSKOS Server",
  "version": null,
  "closedWorldAssumption": true,
  "port": 3000,
  "proxies": [],
  "mongo": {
    "user": "",
    "pass": "",
    "host": "127.0.0.1",
    "port": 27017,
    "db": "jskos-server",
    "options": {
      "connectTimeoutMS": 360000,
      "socketTimeoutMS": 360000,
      "heartbeatFrequencyMS": 10000
    }
  },
  "auth": {
    "algorithm": "RS256",
    "key": null
  },
  "schemes": true,
  "concepts": true,
  "mappings": {
    "read": {
      "auth": false
    },
    "create": {
      "auth": true
    },
    "update": {
      "auth": true,
      "crossUser": false
    },
    "delete": {
      "auth": true,
      "crossUser": false
    },
    "fromSchemeWhitelist": null,
    "toSchemeWhitelist": null,
    "cardinality": "1-to-n"
  },
  "concordances": true,
  "annotations": {
    "read": {
      "auth": false
    },
    "create": {
      "auth": true
    },
    "update": {
      "auth": true,
      "crossUser": false
    },
    "delete": {
      "auth": true,
      "crossUser": false
    },
    "moderatingIdentities": [],
    "mismatchTagVocabulary": null
  },
  "anonymous": false,
  "identityProviders": null,
  "identities": null,
  "ips": null
}

The provided configuration files (user config and environment config) will be validated with the provided JSON Schema file under config/config.schema.json (public URI: https://gbv.github.io/jskos-server/status.schema.json). If validation fails, JSON Server will refuse to start! Please check whether your configuration is correct after each change. If there is something wrong, the console output will try to provide you with enough detail to fix the issue.

If you are running jskos-server behind a reverse proxy, it is necessary to provide the baseUrl key as well as the proxies key in your configuration (example for our production API):** See also:

{
  "baseUrl": "https://coli-conc.gbv.de/api/",
  "proxies": ["123.456.789.101", "234.567.891.011"]
}

With the keys schemes, concepts, mappings, concordances, and annotations, you can configure whether endpoints related to the specific functionality should be available. A minimal configuration file to just server read-only vocabulary and concept information could look like this:

{
  "mappings": false,
  "annotations": false,
  "concordances": false
}

Available actions for schemes, concepts, mappings, and annotations are read, create, update, and delete. By default, all types can be read, while mappings and annotations can be created, updated, and deleted with authentication. Explanantions for additional options:

* Only applies to actions create, update, and delete.

Note that any properties not mentioned here are not allowed!

Mapping Mismatch Tagging for Negative Assessment Annotations

To differentiate why a mapping was annotated with a negative assessment, a mismatch tagging vocabulary can now be configured under annotations.mismatchTagVocabulary. In theory, any vocabulary can be used, but our instance will use a very small "mismatch" vocabulary available in https://github.com/gbv/jskos-data/tree/master/mismatch.

To set up mapping mismatch tagging, add the vocabulary to the configuration:

{
  "annotations": {
    "mismatchTagVocabulary": {
      "uri": "https://uri.gbv.de/terminology/mismatch/"
    }
  }
}

Currently, the vocabulary and its concepts are required to be imported in the same JSKOS Server instance:

# Import vocabulary metadata
npm run import schemes https://raw.githubusercontent.com/gbv/jskos-data/master/mismatch/mismatch-scheme.json
# Reset existing concepts (e.g. if old version has been imported previously)
npm run reset -- -t concepts -s "https://uri.gbv.de/terminology/mismatch/"
# Import vocabulary concepts
npm run import concepts -- --set-api https://raw.githubusercontent.com/gbv/jskos-data/master/mismatch/mismatch-concepts.json

After restarting JSKOS Server, mapping mismatch tagging is available for annotations. To add such a tag to an annotation, add a body field like this:

{
  "motivation": "assessing",
  "bodyValue": "-1",
  "body": [
    {
      "type": "SpecificResource",
      "value": "https://uri.gbv.de/terminology/mismatch/scope",
      "purpose": "tagging"
    }
  ]
}

Currently, this is the only supported format, i.e. body as an array containing an object with type of "SpecificResource", purpose of "tagging", and the tag concept's URI as value.

To identify whether a JSKOS Server instance supports this kind of tagging, check the /status endpoint for the config.annotations.mismatchTagVocabulary key.

Access control

The rights to read, create, update and delete entities via API can be controlled via several configuration settings described above (data import is not limited by these restrictions):

The first control is only checked if it has a superordinated object with contributor and/or creator. This can only be the case for mappings and concepts. The connection to a superordinated object is checked on both the stored object and its modified value, so moving a mapping from one concordance to another is only allowed if access is granted for both. The authenticated user must be listed as creator or contributor of the superordinated object to pass this control.

The second control is only checked when the first control cannot be applied and only on authenticated actions update or delete where anonymous is set to false (this is the default). With crossUser set to false, the authenticated user must be listed as creator of the stored object. With crossUser set to true any authenticated user (optionally limited to those listed in identities) can update or delete the object.

For authenticated actions with anonymous being false creation of a new object will always set its initial creator to the autenticated user and update of an object will always add the user to contributor unless it is already included as creator or contributor. Further modification of creator and contributor (removal and addition of entries) is limited to vocabularies and concordance by authenticated users listed as creator of the object.

Here are some helpful example presets for configuration of "concordances, "mappings", or "annotations".

Read-only access (does not make sense for annotations):

{
  "read": {
    "auth": false
  }
}

Anyone can create, but only logged-in users can update and delete (and only their own items):

{
  "read": {
    "auth": false
  },
  "create": {
    "auth": false
  },
  "update": {
    "auth": true,
    "crossUser": false
  },
  "delete": {
    "auth": true,
    "crossUser": false
  }
}

Anyone can create, logged-in users can update (independent of creator), logged-in users can delete their own items:

{
  "read": {
    "auth": false
  },
  "create": {
    "auth": false
  },
  "update": {
    "auth": true,
    "crossUser": true
  },
  "delete": {
    "auth": true,
    "crossUser": false
  }
}

Anyone can create, as well as update and delete, independent of creator:

{
  "read": {
    "auth": false
  },
  "create": {
    "auth": false
  },
  "update": {
    "auth": false,
    "crossUser": true
  },
  "delete": {
    "auth": false,
    "crossUser": true
  }
}

If write access for concept schemes and/or concepts is necessary, it is recommended that they are secured by only allowing certain users (via identities) or only allowing certain IP addresses (via ips):

Only user with URI https://coli-conc.gbv.de/login/users/c0c1914a-f9d6-4b92-a624-bf44118b6619 can write:

{
  "read": {
    "auth": false
  },
  "create": {
    "auth": true,
    "identities": ["https://coli-conc.gbv.de/login/users/c0c1914a-f9d6-4b92-a624-bf44118b6619"]
  },
  "update": {
    "auth": true,
    "identities": ["https://coli-conc.gbv.de/login/users/c0c1914a-f9d6-4b92-a624-bf44118b6619"]
  },
  "delete": {
    "auth": true,
    "identities": ["https://coli-conc.gbv.de/login/users/c0c1914a-f9d6-4b92-a624-bf44118b6619"]
  }
}

Only localhost can write:

{
  "read": {
    "auth": false
  },
  "create": {
    "auth": false,
    "ips": ["127.0.0.1"]
  },
  "update": {
    "auth": false,
    "ips": ["127.0.0.1"]
  },
  "delete": {
    "auth": false,
    "ips": ["127.0.0.1"]
  }
}

Note that auth is set to false because it refers to authentication via JWT. The IP filter is separate from that. An even more secure way would be to use both JWT authentication with an identities filter as well as an IP filter.

Only user with URI https://coli-conc.gbv.de/login/users/c0c1914a-f9d6-4b92-a624-bf44118b6619 can create, but others can update/delete if they are creator/contributor of an entity:

{
  "read": {
    "auth": false
  },
  "create": {
    "auth": true,
    "identities": ["https://coli-conc.gbv.de/login/users/c0c1914a-f9d6-4b92-a624-bf44118b6619"]
  },
  "update": {
    "auth": true
  },
  "delete": {
    "auth": true
  }
}

A configuration like this will be used to handle concordances in Cocoda. Only selected accounts will be able to create new concordances, but they will be able to add other accounts as creator/contributor so that those accounts will be able to assign mappings to the concordance and edit mappings that belong to the concordance.

Authentication

It is possible to limit certain actions to authenticated users, indicated by the auth option (see example configurations above). Authorization is performed via JWTs (JSON Web Tokens). To configure authentication, you need to provide the JWT algorithm and the key/secret in the configuration file, like this:

"auth": {
  "algorithm": "RS256",
  "key": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA57ZWRoOjXYTQ9yujxAu7\ne3k4+JRBAqGdDVIRRq5vXB2D5nJBIhQjVjylumn+QnTX/MdZx8qn7X96npUwHwIh\nylCgUmsYXcjP08X/AXEcP5bPOkgBBCKjWmcm+p01RQSOM0nSptyxpyXzr2ppWe1b\nuYdRYDWj+JV7vm+jJA4NiFv4UnAhoG5lRATADzu0/6wpMK3dVMBL7L0jQoV5xBAb\nLADOy5hD9XEII3VPkUqDGIKM+Z24flkCIf0lQ7FjsoZ2mmM1SZJ5vPDcjMKreFkX\ncWlcwGHN0PUWZWLhb7c8yYa1rauMcwFwv0d2XyOEfgkqEJdCh8mVT/5jR48D2PNG\ncwIDAQAB\n-----END PUBLIC KEY-----\n"
}

The JWT has to be provided as a Bearer token in the authorization header, e.g. Authorization: Bearer <token>. Currently, all authorized endpoints will be accessible (although PUT/PATCH/DELETE are limited to the user who created the object by default), but later it will be possible to set scopes for certain users (see #47).

The authentication is designed to be used together with an instance of login-server, but it is also possible to use your own JWTs.

JWT Example

The recommended Node.js library for creating JWTs is jsonwebtoken. Note that for simplicity, we are using the HS256 algorithm which is symmetrical. In most cases, it would be better to use RS256 with a libarary like node-rsa instead.

Simple config, restricting the /mappings endpoint with authentication:

{
  "auth": {
    "algorithm": "HS256",
    "key": "yoursecret"
  },
  "mappings": {
    "read": {
      "auth": true
    }
  }
}

Creating a JWT:

const jwt = require("jsonwebtoken")
// Payload is an object containing the user object with an URI:
const data = {
  user: { uri: "urn:test:hallo" }
}
// Sign the token with our secret
const token = jwt.sign(data, "yoursecret", {
  algorithm: "HS256",
  expiresIn: "7d" // valid for 7 days
})

Using the token in a request (using curl):

# Request without header should return ForbiddenAccessError (code 403)
curl localhost:3000/mappings
# Request with header should return JSON data (insert your own token and jskos-server URL of course)
curl -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyIjp7InVyaSI6InRlc3Q6aGFsbG8ifSwiaWF0IjoxNTg5NTMyNDU3LCJleHAiOjE1OTAxMzcyNTd9.fXIxgS0QyFk9Lvz7Z-fkb4tAueMTSNZ4zAuB6iwePq4" localhost:3000/mappings

If you are the only user that is supposed to be authenticated for your instance of jskos-server, you could in theory use something like this to create a token with a long lifetime and use it for all your requests. Please consider the security implications before doing this though.

Login Server Example

If you have multiple users using your instance of jskos-server, it is recommended to use login-server for authentication. login-server uses the asymmetrical RS256 algorithm by default and will create a public/private key pair on first launch. The public key will be in ./public.key and you will need that for the configuration:

{
  "auth": {
    "algorith": "RS256",
    "key": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA57ZWRoOjXYTQ9yujxAu7\ne3k4+JRBAqGdDVIRRq5vXB2D5nJBIhQjVjylumn+QnTX/MdZx8qn7X96npUwHwIh\nylCgUmsYXcjP08X/AXEcP5bPOkgBBCKjWmcm+p01RQSOM0nSptyxpyXzr2ppWe1b\nuYdRYDWj+JV7vm+jJA4NiFv4UnAhoG5lRATADzu0/6wpMK3dVMBL7L0jQoV5xBAb\nLADOy5hD9XEII3VPkUqDGIKM+Z24flkCIf0lQ7FjsoZ2mmM1SZJ5vPDcjMKreFkX\ncWlcwGHN0PUWZWLhb7c8yYa1rauMcwFwv0d2XyOEfgkqEJdCh8mVT/5jR48D2PNG\ncwIDAQAB\n-----END PUBLIC KEY-----\n"
  }
}

After that, you can use login-client to interact with your login-server instance and receive JWTs. When using WebSockets, login-server will periodically send a new JWT before the previous one expires. You can then use that to authenticate your requests to jskos-server. (An example on how to use login-client can be found in the source code of login-server.)

For testing your authentication without a full-fledged solution using login-client, you can use http://localhost:3004/token (where localhost:3004 is your instance of login-server) to request a JWT.

Note about previous additional options for auth:

Data Import

JSKOS Server provides scripts to import JSKOS data into the database or delete data from the database. Right now, mappings, terminologies (concept schemes), concepts, concordances, and annotations, in JSON (object or array of objects) or NDJSON format are supported.

Import Notes

About hierarchies within concepts: Hierarchies are supported. However, only the broader field will be used during import. Both ancestors and narrower will be removed and the respective endpoints (GET /concepts/ancestors and GET /concepts/narrower) will dynamically rebuild these properties. That means that when converting your data, please normalize it so that the hierarchy is expressed via the broader field in JSKOS.

Example scheme (as JSON object) with concepts in a hierarchy (as NDJSON):

{
  "uri": "urn:test:scheme",
  "notation": [
    "TEST"
  ],
  "uriPattern": "^urn:test:concept-(.+)$"
}
{ "topConceptOf": [{ "uri": "urn:test:scheme" }], "uri": "urn:test:concept-a" }
{ "inScheme":     [{ "uri": "urn:test:scheme" }], "uri": "urn:test:concept-a.1",    "broader": [{ "uri": "urn:test:concept-a" }] }
{ "inScheme":     [{ "uri": "urn:test:scheme" }], "uri": "urn:test:concept-a.2",    "broader": [{ "uri": "urn:test:concept-a" }] }
{ "topConceptOf": [{ "uri": "urn:test:scheme" }], "uri": "urn:test:concept-b" }
{ "inScheme":     [{ "uri": "urn:test:scheme" }], "uri": "urn:test:concept-b.1",    "broader": [{ "uri": "urn:test:concept-b" }] }
{ "inScheme":     [{ "uri": "urn:test:scheme" }], "uri": "urn:test:concept-b.1.1",  "broader": [{ "uri": "urn:test:concept-b.1" }] }
{ "inScheme":     [{ "uri": "urn:test:scheme" }], "uri": "urn:test:concept-b.1.2",  "broader": [{ "uri": "urn:test:concept-b.1" }] }

(Note that a notation for the concepts can be omitted because we have defined uriPattern on the concept scheme. Also, we don't need to define inScheme for concepts with topConceptOf.)

About the created property for concept schemes: The import script uses the bulk write endpoints to import data. For concept schemes, this means that any existing data for imported schemes will be overwritten and replaced with the new data. This includes especially the created property which might not exist in your source data and will be set on import if necessary. If you need a consistent created date, make sure that your source data already includes this field.

Import Script

Examples of using the import script:


# Create indexes for all types
npm run import -- --indexes
# Import RVK scheme (from coli-conc API)
npm run import -- schemes https://coli-conc.gbv.de/rvk/api/voc
# Import RVK concepts (this will take a while)
npm run import -- concepts https://coli-conc.gbv.de/rvk/data/2019_1/rvko_2019_1.ndjson
# Import coli-conc concordances
npm run import -- concordances https://coli-conc.gbv.de/api/concordances

# Batch import multiple files or URLs
npm run import-batch -- mappings files.txt
# files.txt should contain one file or URL per line with the full path and no escaping.
# You can, for example, store these batch import files in folder `imports` which is ignored in git.

Note: If you have concepts in your database, make sure to run npm run import -- --indexes at least once. This will make sure all necessary indexes are created. Without this step, the /concepts/suggest and /concepts/search endpoints will not work.

For more information about the import script, run npm run import -- --help.

Reset Script

It is also possible to delete entities from the server via the command line. Running the command will first determine what exactly will be deleted and ask you for confirmation:

# Will delete everything from database
npm run reset
# Will delete mappings from database
npm run reset -- -t mappings
# Will delete all concepts that belong to a certain concept scheme URI
npm run reset -- -s http://uri.gbv.de/terminology/rvk/
# Will delete all mappings that belong to a certain concordance URI
npm run reset -- -c https://gbv.github.io/jskos/context.json
# Will delete entities with certain URIs
npm run reset -- http://rvk.uni-regensburg.de/nt/A http://rvk.uni-regensburg.de/nt/B
# Will show help for more information
npm run reset -- --help

For scripting, you can use the yes command to skip confirmation. Make sure you know what you're doing! Example: yes | npm run reset -- urn:test:uri.

Usage

Run Server

# Development server with hot reload and auto reconnect at localhost:3000 (default)
npm run start

# To run the server in production, run this:
NODE_ENV=production node ./server.js

Run Tests

Tests will use the real MongoDB with -test-${namespace} appended to the database name.

npm test

Run Supplemental Scripts

There are some supplemental scripts that were added to deal with specific sitatuations. These can be called with npm run extra name-of-script. The following scripts are available:

API

General

All API methods stick to the following rules, unless otherwise specified.

Requests

Write access

Responses

GET /status

Returns a status object.

There is a JSON Schema for the format of this endpoint. It is available under /status.schema.json for every jskos-server installation (starting from version 1.0.0). The most recent schema can be accessed here: https://gbv.github.io/jskos-server/status.schema.json

Note that certain properties from the actual configuration will not be shown in the result for /status:

GET /checkAuth

Endpoint to check whether a user is authorized. If type or action are not set, it will use identities/identityProviders that are defined directly under config.

POST /validate

Endpoint to validate JSKOS objects via jskos-validate.

If neither type nor knownSchemes are specified, concept schemes in the data to be validated can be used to validate following concepts in the same request array (see last example below).

GET /validate

Same as POST /validate but JSKOS data to be validated is passed via URL.

GET /data

Returns data for a certain URI or URIs. Can return concept schemes, concepts, concordances, mappings, and annotations. This endpoint does not offer pagination via limit and offset. It will always return all results. Furthermore, there is no certain order to the result set (but it should be consistent across requests). If a certain type of data requires authentication and the user is not authenticated, that type of data will simply not be returned.

Note: As of version 2.0, this endpoint was adjusted to return all types of items that are available in the database, instead of just concepts and concept schemes. The additional parameters, apart from uri, were also removed. For the previous behavior (only without returning concept schemes), see GET /concepts.

GET /concordances

Lists all concordances for mappings.

GET /concordances/:_id

Returns a specific concordance.

POST /concordances

Saves one or more concordances in the database. Note that fromScheme and toScheme must be supported by the jskos-server instance.

PUT /concordances/:_id

Overwrites a concordance in the database.

Note that any changes to the uri, notation, fromScheme, toScheme, extent, distributions, and created properties will be ignored. (No error will be thrown in this case.)

PATCH /concordances/:_id

Adjusts a concordance in the database.

Note that changes to the properties uri, notation, fromScheme, toScheme, created, extent, and distributions are currently not allowed and will result in an InvalidBodyError.

DELETE /concordances/:_id

Deletes a concordance from the database.

Note that only concordances which have no mappings associated can be deleted.

GET /mappings

Returns an array of mappings. Each mapping has a property uri under which the specific mapping can be accessed.

GET /mappings/suggest

Suggests notations used in mappings.

GET /mappings/voc

Lists all concept schemes used in mappings.

GET /mappings/infer

Returns mappings based on stored mappings and mappings derived by inference. If a request to GET /mappings results in stored mappings, only those are returned. If no stored mappings match the request, the following algorithm is applied to infer virtual mappings (this is experimental and not all source schemes are supported):

Inferred mappings don't have fields such as uri, identifier, creator, created... but uri of the mapping used for inference is included in source.

GET /mappings/:_id

Returns a specific mapping.

POST /mappings

Saves a mapping or multiple mappings in the database.

Note that the partOf property is currently not allowed. Associating a mapping with a concordances has to be done in a separate PUT or PATCH request.

PUT /mappings/:_id

Overwrites a mapping in the database.

Note that any changes to the created property will be ignored. Note that changes to partOf (i.e. association with a concordance) are only possible if 1) fromScheme and toScheme are equal between the mapping and the concordance, 2) the authenticated user is creator of the mapping OR if the mapping is already part of a concordance, the user is creator/contributor of that concordance, and 3) the user is creator/contributor of the target concordance (if given).

PATCH /mappings/:_id

Adjusts a mapping in the database.

Note that any changes to the created property will be ignored. Note that changes to partOf (i.e. association with a concordance) are only possible if 1) fromScheme and toScheme are equal between the mapping and the concordance, 2) the authenticated user is creator of the mapping OR if the mapping is already part of a concordance, the user is creator/contributor of that concordance, and 3) the user is creator/contributor of the target concordance (if given).

DELETE /mappings/:_id

Deletes a mapping from the database.

GET /voc

Lists supported terminologies (concept schemes).

POST /voc

Saves a concept scheme or multiple concept schemes in the database. Each concept scheme has to have a unique uri.

PUT /voc

Overwrites a concept scheme in the database. Is identified via its uri field.

Note that any changes to the created property will be ignored.

DELETE /voc

Deletes a concept scheme from the database.

GET /voc/top

Lists top concepts for a concept scheme.

GET /voc/concepts

Lists concepts for a concept scheme.

DELETE /voc/concepts

Deletes all concepts of a certain concept scheme from the database.

GET /voc/suggest

Returns concept scheme suggestions.

GET /voc/search

Currently the same as /voc/suggest with parameter format=jskos.

GET /concepts

Returns detailed data for concepts. Note that there is no certain order to the result set (but it should be consistent across requests). Note that no data is returned if no filtering parameter is given (one of uri, notation, voc, or near).

POST /concepts

Saves a concept or multiple concepts in the database. Each concept has to have a unique uri as well as a concept scheme that is available on the server in the inScheme or topConceptOf field.

PUT /concepts

Overwrites a concept in the database. Is identified via its uri field.

DELETE /concepts

Deletes a concept from the database.

GET /concepts/narrower

Returns narrower concepts for a concept.

Note: The old /narrower endpoint is deprecated as of version 2.0 and will be removed in version 3.0.

GET /concepts/ancestors

Returns ancestor concepts for a concept.

Note: The old /ancestors endpoint is deprecated as of version 2.0 and will be removed in version 3.0.

GET /concepts/suggest

Returns concept suggestions.

Note: The old /suggest endpoint is deprecated as of version 2.0 and will be removed in version 3.0.

GET /concepts/search

Currently the same as /concepts/suggest with parameter format=jskos. Additionally, search supports the parameter properties=[list] as in the other concept methods.

Note: The old /search endpoint is deprecated as of version 2.0 and will be removed in version 3.0.

GET /annotations

Returns an array of annotations. Each annotation has a property id under which the specific annotation can be accessed.

GET /annotations/:_id

Returns a specific annotation.

POST /annotations

Saves an annotation or multiple annotations in the database.

PUT /annotations/:_id

Overwrites an annotation in the database.

Note that any changes to the created property will be ignored.

PATCH /annotations/:_id

Adjusts an annotation in the database.

Note that any changes to the created property will be ignored.

DELETE /annotations/:_id

Deletes an annotation from the database.

Errors

If possible, errors will be returned as a JSON object in the following format (example):

{
  error: "EntityNotFoundError",
  status: 404,
  message: "The requested entity ABC could not be found.",
}

The following errors are currently caught and returned as JSON:

EntityNotFoundError

Status code 404. Will be returned if GET /mappings/:_id or GET /annotations/:_id are requested with an unknown ID.

MalformedBodyError

Status code 400. Will be returned for POST/PUT/PATCH if the body was not JSON or missing.

MalformedRequestError

Status code 400. Will be returned if a required parameter is missing (currently implemented in GET /.../:_id endpoints, but should not be possible to reach).

DuplicateEntityError

Status code 422. Will be returned for POST if an entity with the same ID/URI already exists in the database.

InvalidBodyError

Status code 422. Will be returned for POST/PUT/PATCH if the body was valid JSON, but could not be validated (e.g. does not pass the JSKOS Schema).

CreatorDoesNotMatchError

Status code 403. Will be returned by PUT/PATCH/DELETE endpoints if the authenticated creator does not match the creator of the entity that is being edited.

BackendError

Status code 500. Will be returned if there's a backend error not related to the database or configuration.

DatabaseAccessError

Status code 500. Will be returned if the database is not available or if the current database request failed with an unknown error.

DatabaseInconsistencyError

Status code 500. Will be returned if there is an inconsistency issue with our database. Please contact us with the full error message if this occurs!

ConfigurationError

Status code 500. Will be returned if there is an error in the configuration that prevents the application from working correctly.

ForbiddenAccessError

Status code 403. Will be returned if the user is not allow access (i.e. when not on the whitelist or when an identity provider is missing).

Deployment

The application is currently deployed at http://coli-conc.gbv.de/api/. At the moment, there is no automatic deployment of new versions.

Notes about depolyment on Ubuntu

It is recommended to use a newer version of Node.js. Installing the dependencies might also require installing nodejs-legacy: sudo apt-get install nodejs-legacy (more info here). One possibility for running the application in production on Ubuntu 16.04 is described here. (Information about restarting pm2-based services on system reboot here.)

Update an instances deployed with PM2

# get updates from repository
git pull

# install dependencies
npm ci

# restart the process (adjust process name if needed)
pm2 restart jskos-server

Daily Import

If you'd like to run the import script daily to refresh current mappings, you can for example use a cronjob:

# Runs import script for jskos-server in /srv/cocoda/jskos-server at 1 AM each day.
00 01 * * * cd /srv/cocoda/jskos-server; ./scripts/import.sh

Running Behind a Reverse Proxy

There are certain things to consider when running jskos-server behind a reverse proxy:

  1. Make sure the base URL is configured correctly in config.json so that correct URIs will be generated. Test this by creating a new mapping and making sure the URI of that mapping is correct and accessible.

  2. Provide a list of trusted proxy IPs or ranges in the proxies key in config.json. E.g. "proxies": ["123.456.789.101", "234.567.891.011"]. See also: Express behind proxies.

  3. The reverse proxy should be configured so that the base URL has a trailing slash: https://example.com/api ❌ - https://example.com/api/ ✅ (Note: Not implementing this has no further consequences except that /api will not be accessible.)

  4. The reverse proxy should also be configured so that any URL except the base URL has no trailing slash: https://example.com/api/status/ ❌ - https://example.com/api/status ✅

  5. Make sure the target parameter (i.e. the actual IP and port where jskos-server is running) has a trailing slash.

  6. Make sure the proxy is configured to correctly set the X-Forwarded-For header.

The following would be an example for 2./3./4. with an Apache reverse proxy:

<VirtualHost *:8099>
    Define API_PATH /api
    ServerName example.com

    RewriteEngine on
    # Remove trailing slash from everything EXCEPT the base URL
    RewriteCond %{REQUEST_URI} !^${API_PATH}/$
    RewriteRule ^(.*)/+$ $1 [R=301,L]
    # Force trailing slash for base URL only
    RewriteCond %{REQUEST_URI} ^${API_PATH}$
    RewriteRule ^(.+[^/])$ %{REQUEST_URI}/ [R=301,L]

    # Forward to jskos-server
    ProxyPass ${API_PATH}/ http://127.0.0.1:3000/
    ProxyPassReverse ${API_PATH}/ http://127.0.0.1:3000/

    # ...
</VirtualHost>

Related works

jskos-server is developed together with the cocoda mapping application.

Alternative open source applications for hosting concept schemes include:

See cocoda-sdk for efforts to provide uniform access to vocabulary information from different applications and sources.

Maintainers

Contribute

PRs accepted against the dev branch.

Small note: If editing the README, please conform to the standard-readme specification.

Publish

For maintainers only

Never work on the master branch directly. Always make changes on dev and then run the release script:

npm run release:patch # or minor or major

License

MIT © 2018 Verbundzentrale des GBV (VZG)