Fedora Messaging Application Toolbox

A collection of ready-to-use messaging applications for use with Fedora. These applications use Apache Camel.

Additional background information is available on the Fedora Wiki on the Integration Services page.

Prerequisites

• Java 11
• Solr
• Fedora 6

NOTE: This is project is currently in a state of flux as we are in the process of upgrading it to support Java 11 and Camel 3.9.x Currently the Solr, ActiveMQ, Reindexing, and LDPath microservices are available. Triplestore indexing and Fixity are coming soon.

Building

To build these projects use this command

MAVEN_OPTS="-Xmx1024m" mvn clean install

Running the toolbox

The camel toolbox can be run as a java cli application. Once the maven build is done, run the toolbox driver:

java -jar fcrepo-camel-toolbox-app/target/fcrepo-camel-toolbox-app-<version>-driver.jar -c configuration.properties

where your configuration.properties file is a standard java properties file.

Docker Compose

The Camel Toolbox can be started using Docker Compose which will create containers for Fedora, Fuseki, Solr, and the Camel Toolbox application.

Configuration for the Camel Toolbox can be done through the docker-compose/camel-toolbox-config/configuration.properties or through environment variables (not yet available) as standard java properties as key value pairs. To run with the docker containers the following properties are set by default:

jms.brokerUrl=tcp://fcrepo:61616
fcrepo.baseUrl=http://fcrepo:8080/fcrepo/rest

solr.indexing.enabled=true
solr.baseUrl=http://solr:8983/solr/fcrepo

triplestore.indexing.enabled=true
triplestore.baseUrl=http://fuseki:3030/fcrepo

audit.enabled=true
fixity.enabled=true
fcrepo.authHost=fcrepo
reindexing.rest.host=0.0.0.0

Then to start, the Camel Toolbox, Fedora, Fuseki, and Solr containers run

cd ./docker-compose
docker-compose up -d

If you need to rebuild the docker image locally you can do so like so:

mvn clean install
FCREPO_CAMEL_TOOLBOX_VERSION=$(mvn org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate -Dexpression=project.version -q -DforceStdout)
docker buildx build --load --tag="fcrepo/fcrepo-camel-toolbox" --tag="fcrepo/fcrepo-camel-toolbox:${FCREPO_CAMEL_TOOLBOX_VERSION}" .

Note

Please note: the RDF representation of Fedora Resources is sensitive to the Host header supplied by any client. This can lead to potentially surprising effects from the perspective of the applications in this Messaging Toolbox.

For example, if the fcrepo-indexing-triplestore connects to Fedora at http://localhost:8080 but another client modifies Fedora resources at http://repository.example.edu, you may end up with incorrect and/or duplicate data in downstream applications. It is far better to force clients to connect to Fedora over a non-localhost network interface. Depending on your deployment needs, you may also consider setting a static Host header in a proxy. For instance, with nginx, to proxy Fedora over a standard web port, this configuration may suffice:

    location /fcrepo {
        proxy_pass http://localhost:8080;
        proxy_http_version 1.1;
        proxy_set_header Host repository.example.edu;
    }

Any such reverse proxy will work. Then, if port 8080 is inaccessible from outside the deployment server, and all clients (including these messaging toolbox applications) access Fedora with the baseUrl set to something like: http://repository.example.edu/fcrepo/rest, then the asynchonous integrations will be less prone to configuration errors.

Global Properties

ActiveMQ Service

This implements a connector to an ActiveMQ broker.

Properties

Repository Indexer (Solr)

This application listens to Fedora's event stream and indexes objects into an external Solr server.

Properties

Repository Indexer (Triplestore)

This application listens to Fedora's event stream and indexes objects into an external triplestore.

Properties

Reindexing Service

This application implements a reindexing service so that any node hierarchy in fedora (e.g. the entire repository or some subset thereof) can be reindexed by a set of external services.

One can specify which applications/endpoints to send these reindexing events, by POSTing a JSON array to the re-indexing service:

curl -XPOST localhost:9080/reindexing/fedora/path -H"Content-Type: application/json" \
    -d '["broker:queue:solr.reindex","broker:queue:fixity","broker:queue:triplestore.reindex"]'

Properties

HTTP Message Forwarding Service (HTTP)

This application listens to Fedora's event stream and forwards message identifiers and event types as JSON POSTs to an HTTP endpoint.

Properties

Note: you MUST set the http.baseUrl property in order for this service to do anything meaningful.

Fixity Checking Service

This application can be used in conjunction with the Repository Re-Indexer to verify the checksums for all Binary resources in the repository.

Properties

Repository Audit Service (Triplestore)

This application listens to Fedora's event stream, and stores audit-related events in an external triplestore. Both Jena Fuseki and Open RDF Sesame are supported.

More information about the audit service is available on the Fedora wiki.

Properties

Troubleshooting

java.lang.IllegalArgumentException: Credentials may not be null

Check the configuration.properties to ensure that the fcrepo.baseUrl and fcrepo.authHost have the same hostname. If they differ, the http client will not be able to find the credentials passed in for authentication.

Maintainers

Current maintainers:

• Danny Bernstein