search

fcrepo-exts / fcrepo-camel-toolbox

A collection of ready-to-use messaging applications with fcrepo-camel
Apache License 2.0
13 stars 26 forks source link

fcrepo-reindexing does not honor jms.brokerUrl configuration setting #116

Closed zozlak closed 7 years ago

zozlak commented 7 years ago

It seems to me that the reindexing plugin tries to connect to the ActiveMQ using a hardcoded address tcp://localhost:61616.

I have jms.brokerUrl=tcp://fedora:61616 set in the org.fcrepo.camel.service.activemq.cfg and it is honored by the fcrepo-indexing-triplestore plugin but not by the reindexing plugin.

When I trigger reindex, I am getting in the Karaf log: 

FailoverTransport  | 135 - org.apache.activemq.activemq-osgi - 5.14.0 | Failed to connect to [tcp://localhost:61616] after: 20 attempt(s) continuing to retry.

I also can not see any reindexing plugin configuration property allowing me to define ActiveMQ on the other server. Is there a way to do so?

acoburn commented 7 years ago

the jms.brokerUrl doesn't exist in the fcrepo-reindexing application in the 4.6.x series. Can you point to where you see that it is being used? Can you also provide more information on your environment -- version of karaf, version of camel, version of fcrepo-camel-toolbox, applications installed.

zozlak commented 7 years ago

My complete scenario is as follows.

I have Fedora coupled with a triplestore and I want to populate the triplestore after restoring Fedora content. As restore does not generate ActiveMQ messages, I need to trigger reindexing manually and that is where the reindexing plugin comes in.

My setup is:

My Camel Toolbox Plugins configuration files are as follows:

The triplestore indexing plugin works well with these settings. When I create or modify a resource, the data in the triplestore are updated.

But when I trigger the reindexing with curl -i -X POST http://localhost:9080/reindexing -H 'content-type: application/json' -d '["activemq:queue:triplestore.reindex"]' run inside the Karaf's Docker container, I get in the Karaf log:

2016-12-02 12:21:26,511 | INFO  | tp1985176132-333 | ReindexingRouter                 | 159 - org.fcrepo.camel.fcrepo-reindexing - 4.7.0.SNAPSHOT | Initial indexing path:
2016-12-02 12:21:31,680 | WARN  | ActiveMQ Task-1  | FailoverTransport                | 53 - org.apache.activemq.activemq-osgi - 5.14.0 | Failed to connect to [tcp://localhost:61616] after: 10 attempt(s) continuing to retry.

So it looks like the reindexing was triggered but can not connect to the ActiveMQ on tcp://localhost:61616.

If I understand correctly my setup, this is because the ActiveMQ does not run (from the Karaf's perspective) on tcp://localhost:61616 but on tcp://fedora:61616 and I have no mean to tell that to the reindexing plugin.

As you probably noticed I am compiling the plugins with the -DskipTests switch. This is because the reindexing plugin failed to perform its tests. You can find the full build log here. Again it looks like the tests assume that there is a Fedora instance running locally which is not true in my setup.

acoburn commented 7 years ago

Why are you compiling from source? I would highly recommend not compiling from source. There is no need to: those artifacts are all on Maven Central. Furthermore, I would suggest not using feature:repo-add mvn:org.fcrepo.camel/toolbox-features/LATEST/xml/features; instead, identify the version you want to use: i.e. feature:repo-add mvn:org.fcrepo.camel/toolbox-features/4.6.2/xml/features.

Even better, you would add that feature repository to $KARAF_HOME/etc/org.apache.karaf.features.cfg like so:

featuresRepositories = \
    ..., \
    mvn:org.apache.activemq/activemq-karaf/5.14.0/xml/features, \
    mvn:org.apache.camel.karaf/apache-camel/2.17.3/xml/features, \
    mvn:org.fcrepo.camel/fcrepo-camel/4.4.3/xml/features, \
    mvn:org.fcrepo.camel/toolbox-features/4.6.2/xml/features

featuresBoot = \
    ..., \
    fcrepo-service-camel, \
    fcrepo-service-activemq, \
    fcrepo-indexing-triplestore, \
    fcrepo-reindexing

Then, you can, at any time simply remove the $KARAF_HOME/data directory and "start fresh" by rebooting the container.

acoburn commented 7 years ago

Just as a note -- I am running the camel-toolbox/4.6.2 in karaf/4.0.5 with a remote activemq broker (separate from Fedora), a remote fedora/4.7.0 connection and a remote triplestore. The only thing to note is that you need to set fcrepo.baseUrl=http://example.org/fcrepo/rest. In your case, that would be http://fedora:8080/rest.

zozlak commented 7 years ago

And how does your reindexing plugin knows where the activemq broker is?

Or maybe by "remote activemq broker" you mean it is remote to Fedora but on the same host as Karaf?

acoburn commented 7 years ago

The reindexing application finds the broker by looking it up in the OSGi service registry:

https://github.com/fcrepo4-exts/fcrepo-camel-toolbox/blob/master/fcrepo-reindexing/src/main/resources/OSGI-INF/blueprint/blueprint.xml#L22

And by "remote", I mean it is on a separate machine, on a separate network that I do not control.

zozlak commented 7 years ago

Why are you compiling from source?

I can sum it up as lack of knowledge on deploying Java applications.

By the way it will be nice to include somewhere a short guide on deploying the fcrepo-camel-toolbox for folks like me, who do not know (almost) anything on Java development stack (Maven, Karaf, OSGi containers, etc.).

For example in your last post you show me the right blueprint. It is nice but for me it only turns one question into the other. I guess it means that for the reindexing plugin I need to set up a separate activemq broker and somehow register in in the reindexing plugin Karaf. But can I use the broker already run by Fedora or should I need a separate one? And how I can register a broker in the reindexing plugin Karaf? (and btw why the triplestore indexing plugin reads the broker location from the configuration settings and not from the OSGI service registry?)

acoburn commented 7 years ago

@zozlak the message broker is shared across all camel components, including the triplestore indexer. Most users will make use of the AMQ broker that comes bundled with Fedora, so unless you have particular failover/high-availability needs or are especially familiar with message brokers, I would not suggest setting up multiple brokers. All configuration related to message brokers is handled by the $KARAF_HOME/etc/org.fcrepo.camel.service.activemq.cfg file. And as a further note, the Islandora project does not compile this from source -- they use a script to load the components into karaf: https://github.com/Islandora-CLAW/CLAW/blob/master/install/configs/karaf/fcrepo_camel_toolbox.script

zozlak commented 7 years ago

As you can see in my previous post I have jms.brokerUrl=tcp://fedora:61616 set up in the $KARAF_HOME/etc/org.fcrepo.camel.service.activemq.cfg. That is exactly how I supposed it to work (all plugins share activemq settings provided by the fcrepo-activemq plugin) and it does the job for the fcrepo-indexing-triplestore plugin. But when I trigger reindexing with curl -i -X POST http://localhost:9080/reindexing -H 'content-type: application/json' -d '["activemq:queue:triplestore.reindex"]' I am getting in Karaf logs:

2016-12-02 12:21:26,511 | INFO  | tp1985176132-333 | ReindexingRouter                 | 159 - org.fcrepo.camel.fcrepo-reindexing - 4.7.0.SNAPSHOT | Initial indexing path:
2016-12-02 12:21:31,680 | WARN  | ActiveMQ Task-1  | FailoverTransport                | 53 - org.apache.activemq.activemq-osgi - 5.14.0 | Failed to connect to [tcp://localhost:61616] after: 10 attempt(s) continuing to retry.

and I wonder from where this tcp://localhost:61616 is coming (while it should be tcp://fedora:61616 I have in the $KARAF_HOME/etc/org.fcrepo.camel.service.activemq.cfg).

acoburn commented 7 years ago

Yes, and you are also using a 4.7.0-SNAPSHOT version of fcrepo-camel-toolbox, which is not suitable for what you are trying to do.

Here is what you should do:

1) stop karaf 2) remove the entire $KARAF_HOME/data directory 3) Edit the file $KARAF_HOME/etc/org.apache.karaf.features.cfg so that it contains these features:

featuresRepositories = \
    ..., \
    mvn:org.apache.activemq/activemq-karaf/5.14.0/xml/features, \
    mvn:org.apache.camel.karaf/apache-camel/2.17.3/xml/features, \
    mvn:org.fcrepo.camel/fcrepo-camel/4.4.3/xml/features, \
    mvn:org.fcrepo.camel/toolbox-features/4.6.2/xml/features

featuresBoot = \
    ..., \
    fcrepo-service-camel, \
    fcrepo-service-activemq, \
    fcrepo-indexing-triplestore, \
    fcrepo-reindexing

4) restart karaf

In the future, upgrading these features will require only stopping karaf, editing that file with the updated versions, removing the data directory, and starting karaf.

zozlak commented 7 years ago

I followed your instruction. features:list | grep fcrepo gives me:

fcrepo-camel                            | 4.4.3            |          | Started     | fcrepo-camel-4.4.3          |
fcrepo-indexing-solr                    | 4.6.2            |          | Uninstalled | toolbox-features-4.6.2      |
fcrepo-fixity                           | 4.6.2            |          | Uninstalled | toolbox-features-4.6.2      |
fcrepo-indexing-triplestore             | 4.6.2            | x        | Started     | toolbox-features-4.6.2      |
fcrepo-reindexing                       | 4.6.2            | x        | Started     | toolbox-features-4.6.2      |
fcrepo-audit-triplestore                | 4.6.2            |          | Uninstalled | toolbox-features-4.6.2      |
fcrepo-ldpath                           | 4.6.2            |          | Uninstalled | toolbox-features-4.6.2      |
fcrepo-service-ldcache-file             | 4.6.2            |          | Uninstalled | toolbox-features-4.6.2      |
fcrepo-marmotta-osgi                    | 4.6.2            |          | Uninstalled | toolbox-features-4.6.2      |
fcrepo-serialization                    | 4.6.2            |          | Uninstalled | toolbox-features-4.6.2      |
fcrepo-service-activemq                 | 4.6.2            | x        | Started     | toolbox-features-4.6.2      |
fcrepo-service-camel                    | 4.6.2            | x        | Started     | toolbox-features-4.6.2      |

and for features:list | grep activemq I get:

activemq-broker-noweb                   | 5.14.0           |          | Uninstalled | activemq-5.14.0             | Full ActiveMQ broker with default configuration
activemq-broker                         | 5.14.0           |          | Uninstalled | activemq-5.14.0             | Full ActiveMQ broker with default configuration a
activemq-camel                          | 5.14.0           |          | Started     | activemq-5.14.0             |
activemq-web-console                    | 5.14.0           |          | Uninstalled | activemq-5.14.0             |
activemq-blueprint                      | 5.14.0           |          | Uninstalled | activemq-5.14.0             |
activemq-client                         | 5.14.0           |          | Started     | activemq-core-5.14.0        | ActiveMQ client libraries
activemq-cf                             | 5.14.0           |          | Uninstalled | activemq-core-5.14.0        | ActiveMQ ConnectionFactory from config
activemq                                | 5.14.0           |          | Uninstalled | activemq-core-5.14.0        | ActiveMQ broker libraries

Anyway nothing changed. I am still getting 

2016-12-06 10:41:36,946 | INFO  | tp1477667022-698 | ReindexingRouter                 | 135 - org.fcrepo.camel.fcrepo-reindexing - 4.6.2 | Initial indexing path: 
2016-12-06 10:41:42,125 | WARN  | ActiveMQ Task-1  | FailoverTransport                | 24 - org.apache.activemq.activemq-osgi - 5.14.0 | Failed to connect to [tcp://localhost:61616] after: 10 attempt(s) continuing to retry.

just after triggering the reindex with a POST request. These two messages are the only ones displayed in the Karaf log after triggering reindex.

A little more evidence:

acoburn commented 7 years ago

This is either a configuration error or a networking error. Either way, it is particular to your environment.

From an earlier post, you mentioned that you have this line in $KARAF_HOME/etc/org.fcrepo.camel.reindexing.cfg:

reindexing.stream=broker:queue:triplestore.reindex

That is wrong; it should be:

reindexing.stream=broker:queue:reindexing

(or at least it should be any value except what you have).

Alternately, if this is a networking issue, I would recommend debugging it with telnet, such as is described here: https://access.redhat.com/documentation/en-US/Fuse_Message_Broker/5.4/html/Connectivity_Guide/files/BHIJBDJH.html