search

Open-MBEE / mms-alfresco

Model Management System
Apache License 2.0
30 stars 24 forks source link

readme

Model Management System

Language Grade: Java CircleCI

The MMS AMP is a hosted application run atop the community version of an Alfresco Enterprise Content Management Server.

NOTE 1: AMPs for Alfresco as built from this maven project are meant to be run by Alfresco Community Edition v5.1.g (AKA 201605-GA)

Per Alfresco's documentation on Modules, Alfresco Module Packages (AMPs) are installed using the Module Management Tool MMT jar file. The latest version of the two AMPs you will be "exploding" into the "alfresco" and "share" WARs resident within the embedded Tomcat server are available via the download links leading to the two interdependent portions named "mms-repo.amp" and "mms-share.amp", relating to "alfresco.war" and "share.war", respectively.

Download of the alfresco WAR related portion of the "MMS" Alfresco module.

Download of the share WAR related portion of the "MMS" Alfresco Module.

In tandem with the EMS-Webapp (colloquially known as the View Editor), and the Magicdraw Development Kit (MDK) for MagicDraw client and Teamwork Cloud users; this github repo serves as a one-stop shop to set up the Model Management Server per the MMS-MDK-VE Compatibility Matrix.

Developer Setup

Dependencies

Optional Dependencies (optional if using MDK 3.3.3+, only needed for VE realtime updates)

1a. Using Intellij

1b. Import Project from git repo to Eclipse

1c. Import Maven Project into Eclipse

2. Configure Eclipse to use Maven 3.X.X

3. Configure Run Configuration**

Install Dependencies

1. Install and Configure ElasticSearch

2. Install and Configure PostgreSQL

3. Install and Configure ActiveMQ

Running

1a. Running Alfresco

  1. Select file menu Run > Run Configurations
  2. Expand Maven Build
  3. Select mms
    1. Click Run button
      • If you get error:-Dmaven.multiModuleProjectDirectory system property is not set. Check $M2_HOME environment variable and mvn script match. Goto Window -> Preference -> Java -> Installed JREs -> Edit -> Default VM arguments _set -Dmaven.multiModuleProjectDirectory=$M2HOME

1b. Running Alfresco

  1. From mms-ent directory, either run clean-run.sh, run.sh, or ./mvnw install -Prun -Ddependency.surf.version=6.3

2. Testing Alfresco

  1. Enter http://localhost:8080/share/ at a browser's url address textbox.
  2. Enter admin for user name
  3. Enter admin for password

Design Documentation

1. MMS using ElasticSearch and PostgreSQL

General Design

+----------------+   \
| REST API Layer |    \
|----------------|     \
|   WebScripts   |      MMS
|----------------|     /
|Storage | Layer |    /
+----------------+   /
  /\         /\
  ||         ||
  \/         \/
+-----+    +----+
| RDB |    | ES |
+-----+    +----+
(Graph)    (Data)

2. Configuration Schema

3. Graph Database

The nodes in the graph contain pointers to ElasticSearch documents which contain the real information

Each of these pointers is the “latest one”

The history of each node can be retrieved via a query to ElasticSearch (see ElasticHelper.java)

4. Graph Schema

Nodes and edges both have a type associated with them The type must exist in the NodeTypes and EdgeTypes tables

Some assertions about the graph:

5. Graph

Structure of the graph for each "project"

           +---------+
        +--| Commits |
+===+   |  +---------+          +-------------+    +----------+
| P |   |                   +---| Holding bin |----| Elements |
| R |   |  +-------+        |   +-------------+    +----------+
| O |   +--| Nodes |--------|
| J |   |  +-------+        |   +----------+
| E |   |                   +---| Elements |
| C | --|  +-------+            +----------+
| T |   +--| Edges |
|   |   |  +-------+
|   |   |
|   |   |  +----------------+
+===+   +--| Refs           |
           +----------------+

6. Example webscript in new world: modelpost

Create instance of EmsNodeUtil Get all other relevant information, validation, etc. form request Figure out if the elements are to go to the holding bin or not Calculate the qualified name and ID information for each node Add metadata for each element Store elements in ES Update graph Create JSON response and return

7. Example webscript in new world: modelget

Create instance of EmsNodeUtil Get all other relevant information, validation, etc. form request Access PG to get the ElasticSearch IDs for all documents that we are interested in Access ES via ElasticHelper and get those IDs Create JSON response and return

8. General Pattern for WebScripts

Create instance of EmsNodeUtil Get all other relevant information, validation, etc. form request Understand how to:

All the work is to be done here for each webscript. The pre and post should always be the same.

Create JSON response and return

Testing

Initializing Organizations, Projects, and test elements

Use the following curl commands to post an initial organization + project:

curl -w "\n%{http_code}\n" -H "Content-Type: application/json" -u admin:admin --data '{"orgs": [{"id": "vetest", "name": "vetest"}]}' -X POST "http://localhost:8080/alfresco/service/orgs"
curl -w "\n%{http_code}\n" -H "Content-Type: application/json" -u admin:admin --data '{"projects": [{"id": "123456","name": "vetest","type": "Project"}]}' -X POST "http://localhost:8080/alfresco/service/orgs/vetest/projects"

Then you can post some elements. For convenience, there is a json file in runner/src/test/robotframework/JsonData. Using the project from above:

curl -w "\n%{http_code}\n" -H "Content-Type: application/json" -u admin:admin --data @JsonData/PostNewElements.json -X POST "http://localhost:8080/alfresco/service/projects/123456/refs/master/elements"

Make sure the elements went in:

curl -w "\n%{http_code}\n" -H "Content-Type: application/json" -u admin:admin -X GET "http://localhost:8080/alfresco/service/projects/123456/refs/master/elements/123456?depth=-1"

Robotframework test suite

Robot tests can be run with the following maven profiles in the mms-ent directory:

./mvnw install -Prun,robot-tests

Please note that tests should be run on a clean instance, therefore, it may be helpful to run clean.sh before running the tests

The Robotframework tests require the 'requests' and 'robotframework-requests' python modules. Install it as follows:

pip install --target=runner/src/test/robotframework/libraries requests
pip install --target=runner/src/test/robotframework/libraries robotframework-requests

OR:

pip install --target=$HOME/.m2/repository/org/robotframework/robotframework/{ROBOTPLUGINVERSION}/Lib requests
pip install --target=$HOME/.m2/repository/org/robotframework/robotframework/{ROBOTPLUGINVERSION}/Lib robotframework-requests

Changing debug levels on the fly

If you need to change debug levels on the fly, use the following endpoint.

alfresco/service/loglevel

It takes as input JSON that specifies the classes and the log levels. For example:

[
  {
    "classname": "gov.nasa.jpl.view_repo.webscripts.ModelGet",
    "loglevel": "DEBUG"
  }
]

API Documentation

API Documentation is located at the following endpoints:

Swagger CodeGen:

alfresco/mms/index.html

Swagger UI:

alfresco/mms/swagger-ui/index.html

Swagger YAML file:

alfresco/mms/mms.swagger.yaml

Migrations after 3.2

For versions after 3.2, most notably 3.3.0, an automigration step has been included to run necessary migrations automatically during the initial startup of alfresco after the upgrade. Should this migration fail for any reason, you can trigger the migration manually by accessing the following endpoint:

alfresco/service/migrate/{targetVersion}

where {targetVersion} is the version that you are upgrading to. Example:

alfresco/service/migrate/3.3.0

This operation is idempotent and can be safely run multiple times.

Deployment Requirements

For deploying MMS and VE five machines are recommended + 1 optional TeamWorkCloud machine.

The given configuration below are in terms of AWS instances which define required CPU and RAM, assuming CentOS as operating system.

Data is stored using Amazon EBS volumes, where 100GB is a good start.

For Postgres Amazon's RDS service is used, where 10-15 GB is a good start.

The minimal configuration (can handle about 300k elements) is

For a single machine configuration you need one which is powerful enough to cover the above.

A 64 GB configuration can hold about 2 billion elements but might be slow, so you might want to switch to a powerful CPU configuration:

t3.xlarge -> r5.large -> r5.xlarge

Contributing

To learn how you can get involved in a variety of ways, please see Contibuting to OpenMBEE.