search

actionml / harness

Harness is a Machine Learning/AI Server with plugins for many algorithms including the Universal Recommender
Apache License 2.0
283 stars 49 forks source link

readme

CircleCI

Harness Overview

This project implements a microservice based Machine Learning Server. It provides an API for plug-in Engines and implements all services needed for input and query. It is also the platform for the Universal Recommender, which is a Harness Engine.

Harness features include:

Architecture

Operation

Machine Learning Support

Security

Pre-requisites

In its simplest form Harness has few external pre-requisites. To run it on one machine the requirements are:

Docker-Compose

The Docker host tools for use with docker-compose or run it in a single container, dependent services installed separately. Prerequisites:

See the harness-docker-compose project for further information.

Source Build

For installation on a host OS such as an AWS instance without Docker, the minimum requirements are

Each Engine has its own requirements driven by decisions like what compute engine to use (Spark, TensorFlow, Vowpal Wabbit, DL4J, etc) as well as what Libraries it may need. See specific Engines for their extra requirements.

Architecture

At its core Harness is a fast lightweight Server that supplies 2 very flexible APIs and manages Engines. Engines can be used together or as a single solution.

Harness with Multiple Engines

Harness Core

Harness is a REST server with an API for Engines, Events, Queries, Users, and Permissions. Events and Queries end-points handle input and queries to Engine instances. The rest of the API is used for administrative functions and is generally accessed through the CLI.

The Harness Server core is small and fast and so is appropriate in single Algorithm solutions—an instance of The Universal Recommender is a good example. By adding the Harness Auth-server and scalable Engine backend services Harness can also become a full featured multi-tenant SaaS System.

Router

The Harness core is made from a component called a Router, which maintains REST endpoints for the various object collections. It routes all requests and responses to/from Engine Instances. It also supports SSL, OAuth2 signature based authentication, and REST route authorization where these are optional.

Algorithm specific Engines can be implemented without the need to deal REST APIs. Engines start by plugging into the abstract Engine API and inherit all server features including administration, input, and query APIs.

Administrator

The Administrator manages Engine Instances (and optionally Users and Permissions). With release 0.6.0 Harness will support service discovery via integrated Etcd. The Administrator integrates with the Harness Auth Server and Etcd to provide all admin features.

Engines

An Engine is the implementation of a particular algorithm. They are seen in com.actionml.core.engines module. Each Engine must supply required APIs but what they do when invoked is up to the Engine. The format of input and queries is also Engine specific so see the Engine's docs for more info. The inteface for Engines is designed based on years of implementing production worthy ML/AI algorithms and follows patterns that fit about all of the categories of algorithm types.

The common things about all Engines:

Scaling

Harness is a stateless service. It does very little work itself. It delegates most work to the Engines and other Services that is depends on. These services are all best-in-class made to scale from a shared single machine to massive clusters seamlessly. Harness provides a toolbox of these scalable services that can be used by Engines for state, datasets, and computing.

Harness is scaled by scaling the services it and the various Engines use. For example The Universal Recommender will scale by using more or larger nodes for training (Spark), input storage (MongoDB), and model storage (Elasticsearch). In this manner Harness can be scaled either horizontally (for high availability) or vertically (for cost savings) to supply virtually unlimited resources to an Engine Instance.

Compute Engines

The use of scalable Compute Engines allow Big Data to be used in Algorithms. For instance one common Compute Engine is Apache Spark, which along with the Hadoop Distributed File System (HDFS) form a massively scalable platform. Spark and HDFS support are provided in the Harness Toolbox.

Input Store

Due the its extremely flexible data indexing, convenient TTL support, and massive scalability MongoDB is the default data store.

Server REST API

Harness REST is optionally secured by TLS and Authentication. This requires extensions to the typical REST API to support authentication control objects like Users and Roles, here we ignore these for simplicity.

Integral to REST is the notion of a "resource", which is an item that can be addressed by a resource-id. POSTing to a resource type creates a single resource. The resource types defined in Harness are:

For the full Harness REST API and response codes, see the Harness REST Specification

Input and Query

There are 2 primary APIs in the SDK for sending Engine events and making queries.

Disregarding the optional TLS and Authentication, simple input and queries for the UR look like this:

Example Input

curl -H "Content-Type: application/json" -d '
{
   "event" : "buy",
   "entityType" : "user",
   "entityId" : "John Doe",
   "targetEntityType" : "item",
   "targetEntityId" : "iPad",
   "eventTime" : "2019-02-17T21:02:49.228Z"
}' http://localhost:9090/engines/<some-engine-id>/events

Notice that this posts the JSON body to the http://localhost:9090/engines/<some-engine-id>/events endpoint.

Example Query

curl -H "Content-Type: application/json" -d '
{
  "user": "John Doe"
}' http://localhost:9090/engines/<some-engine-id>/queries

Notice that this posts the JSON body to the http://localhost:9090/engines/<some-engine-id>/queries endpoint.

Example Query Result

The result of the query will be in the response body and looks something like this:

{
    "result":[
        {"item":"Pixel Slate","score":1.4384104013442993},
        {"item":"Surface Pro","score":0.28768208622932434}
    ]
}

For specifics of the format and use of events and queries see the Engine specific documentation—for example The Universal Recommender.

Controlling and Communicating with Harness

Installation

There are several ways to install and run Harness. The primary method we release is through container images but a source project is also maintained for thos who wish to use it.

Versions

See versions.md for released versions, or look in the "develop" branch for works in progress.