search

Islandora / documentation

Contains islandora's documentation and main issue queue.
MIT License
104 stars 71 forks source link

Add API-X to vagrant install #504

Closed dannylamb closed 7 years ago

dannylamb commented 7 years ago

Installation instructions are outlined here: https://github.com/fcrepo4-labs/fcrepo-api-x.

tl;dr Run this from the karaf client and see what breaks.

$ feature:repo-add mvn:org.fcrepo.apix/fcrepo-apix-karaf/LATEST/xml/features
$ feature:install fcrepo-api-x
ruebot commented 7 years ago

in progress

ruebot commented 7 years ago 
karaf@root()> feature:repo-add mvn:org.fcrepo.apix/fcrepo-apix-karaf/LATEST/xml/features
Adding feature url mvn:org.fcrepo.apix/fcrepo-apix-karaf/LATEST/xml/features
Error executing command: Error resolving artifact org.fcrepo.apix:fcrepo-apix-karaf:xml:features:LATEST : mvn:org.fcrepo.apix/fcrepo-apix-karaf/LATEST/xml/features

I'll bug A. Birkland or Elliot M. in #fcrepo irc.

acoburn commented 7 years ago

Misspelling -- try api-x. I.e. feature:repo-add mvn:org.fcrepo.apix/fcrepo-api-x-karaf/LATEST/xml/features

You may also want to try using 0.1.0 instead of LATEST

ruebot commented 7 years ago

@acoburn good catch! I'll send a PR to their repo.

acoburn commented 7 years ago

@ruebot: just did: https://github.com/fcrepo4-labs/fcrepo-api-x/pull/104

ruebot commented 7 years ago

@acoburn thank you!

ruebot commented 7 years ago

PR: https://github.com/Islandora-CLAW/claw_vagrant/pull/3

dannylamb commented 7 years ago

Lol, we gotta stop saying 'partially resolves' if there's more than one pull. Twice in one day.

ruebot commented 7 years ago

Oh shit. I put "resolves" in that commit message!

dannylamb commented 7 years ago

Happened earlier with @bryjbrown today too. Sometimes github is too smart for its own good.

dannylamb commented 7 years ago

Closed by accident due to 'resolved' being in a message.

dannylamb commented 7 years ago

@ruebot I just created https://github.com/fcrepo4-labs/fcrepo-api-x/issues/106 to describe the issue I'm running into with installing API-X. I'll have to circle back to this once it gets sorted out.

ruebot commented 7 years ago

@dannylamb Great! Let me know if you need a hand on my end.

ruebot commented 7 years ago

Looking at this again. When I did a feature:install fcrepo-api-x on a clean vagrant (https://github.com/Islandora-CLAW/claw_vagrant/commit/7381e74413e88671a25840804119d007996656c1), the command completes with no errors, but I'm getting this over and over and over and over in karaf logs.

2017-03-31 02:43:14,024 | INFO  | pool-68-thread-3 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/services
2017-03-31 02:43:14,028 | INFO  | pool-68-thread-3 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Container http://localhost:8080/fcrepo/rest/apix/services does not exist, adding
2017-03-31 02:43:14,106 | INFO  | pool-68-thread-1 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/extensions
2017-03-31 02:43:14,111 | INFO  | pool-68-thread-2 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/ontologies
2017-03-31 02:43:14,118 | INFO  | pool-68-thread-2 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Container http://localhost:8080/fcrepo/rest/apix/ontologies does not exist, adding
2017-03-31 02:43:14,119 | INFO  | pool-68-thread-1 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Container http://localhost:8080/fcrepo/rest/apix/extensions does not exist, adding
2017-03-31 02:43:15,034 | INFO  | pool-68-thread-3 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/services
2017-03-31 02:43:15,037 | INFO  | pool-68-thread-3 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Container http://localhost:8080/fcrepo/rest/apix/services does not exist, adding
2017-03-31 02:43:15,129 | INFO  | pool-68-thread-2 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/ontologies
2017-03-31 02:43:15,131 | INFO  | pool-68-thread-1 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/extensions

...and if I scroll up further:

-03-31 02:41:45,538 | INFO  | pool-2-thread-1  | BlueprintCamelContext            | 59 - org.apache.camel.camel-core - 2.18.1 | Apache Camel 2.18.1 (CamelContext: apix-listener) uptime 
2017-03-31 02:41:45,540 | INFO  | pool-2-thread-1  | BlueprintCamelContext            | 59 - org.apache.camel.camel-core - 2.18.1 | Apache Camel 2.18.1 (CamelContext: apix-listener) is shutdown in 0.033 seconds
2017-03-31 02:41:45,546 | ERROR | pool-2-thread-1  | BlueprintContainerImpl           | 12 - org.apache.aries.blueprint.core - 1.7.1 | Unable to start blueprint container for bundle fcrepo-api-x-listener/0.1.0
org.osgi.service.blueprint.container.ComponentDefinitionException: Unable to instantiate components
    at org.apache.aries.blueprint.container.BlueprintContainerImpl.instantiateEagerComponents(BlueprintContainerImpl.java:728)[12:org.apache.aries.blueprint.core:1.7.1]
    at org.apache.aries.blueprint.container.BlueprintContainerImpl.doRun(BlueprintContainerImpl.java:411)[12:org.apache.aries.blueprint.core:1.7.1]
    at org.apache.aries.blueprint.container.BlueprintContainerImpl.run(BlueprintContainerImpl.java:276)[12:org.apache.aries.blueprint.core:1.7.1]
    at org.apache.aries.blueprint.container.BlueprintExtender.createContainer(BlueprintExtender.java:300)[12:org.apache.aries.blueprint.core:1.7.1]
    at org.apache.aries.blueprint.container.BlueprintExtender.createContainer(BlueprintExtender.java:269)[12:org.apache.aries.blueprint.core:1.7.1]
    at org.apache.aries.blueprint.container.BlueprintExtender.createContainer(BlueprintExtender.java:265)[12:org.apache.aries.blueprint.core:1.7.1]
    at org.apache.aries.blueprint.container.BlueprintExtender.modifiedBundle(BlueprintExtender.java:255)[12:org.apache.aries.blueprint.core:1.7.1]
    at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.customizerModified(BundleHookBundleTracker.java:500)[21:org.apache.aries.util:1.1.1]
    at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.customizerModified(BundleHookBundleTracker.java:433)[21:org.apache.aries.util:1.1.1]
    at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$AbstractTracked.track(BundleHookBundleTracker.java:725)[21:org.apache.aries.util:1.1.1]
    at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.bundleChanged(BundleHookBundleTracker.java:463)[21:org.apache.aries.util:1.1.1]
    at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$BundleEventHook.event(BundleHookBundleTracker.java:422)[21:org.apache.aries.util:1.1.1]
    at org.apache.felix.framework.util.SecureAction.invokeBundleEventHook(SecureAction.java:1179)[org.apache.felix.framework-5.6.1.jar:]
    at org.apache.felix.framework.EventDispatcher.createWhitelistFromHooks(EventDispatcher.java:730)[org.apache.felix.framework-5.6.1.jar:]
    at org.apache.felix.framework.EventDispatcher.fireBundleEvent(EventDispatcher.java:485)[org.apache.felix.framework-5.6.1.jar:]
    at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4541)[org.apache.felix.framework-5.6.1.jar:]
    at org.apache.felix.framework.Felix.startBundle(Felix.java:2172)[org.apache.felix.framework-5.6.1.jar:]
    at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:998)[org.apache.felix.framework-5.6.1.jar:]
    at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:984)[org.apache.felix.framework-5.6.1.jar:]
    at org.apache.karaf.features.internal.service.FeaturesServiceImpl.startBundle(FeaturesServiceImpl.java:1287)[8:org.apache.karaf.features.core:4.0.8]
    at org.apache.karaf.features.internal.service.Deployer.deploy(Deployer.java:860)[8:org.apache.karaf.features.core:4.0.8]
    at org.apache.karaf.features.internal.service.FeaturesServiceImpl.doProvision(FeaturesServiceImpl.java:1176)[8:org.apache.karaf.features.core:4.0.8]
    at org.apache.karaf.features.internal.service.FeaturesServiceImpl$1.call(FeaturesServiceImpl.java:1074)[8:org.apache.karaf.features.core:4.0.8]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_121]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_121]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_121]
    at java.lang.Thread.run(Thread.java:745)[:1.8.0_121]
Caused by: java.lang.NoClassDefFoundError: org/apache/camel/component/jms/JmsComponent
    at java.lang.ClassLoader.defineClass1(Native Method)[:1.8.0_121]
    at java.lang.ClassLoader.defineClass(ClassLoader.java:763)[:1.8.0_121]
    at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.defineClass(BundleWiringImpl.java:2370)
    at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.findClass(BundleWiringImpl.java:2154)
    at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1542)
    at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:79)
    at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:2018)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)[:1.8.0_121]
    at org.apache.felix.framework.BundleWiringImpl.getClassByDelegation(BundleWiringImpl.java:1415)
    at org.apache.felix.framework.BundleWiringImpl.searchImports(BundleWiringImpl.java:1595)
    at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1525)
    at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:79)
    at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:2018)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)[:1.8.0_121]
    at org.apache.felix.framework.Felix.loadBundleClass(Felix.java:1925)[org.apache.felix.framework-5.6.1.jar:]
    at org.apache.felix.framework.BundleImpl.loadClass(BundleImpl.java:978)[org.apache.felix.framework-5.6.1.jar:]
    at org.apache.aries.blueprint.container.BlueprintContainerImpl.loadClass(BlueprintContainerImpl.java:467)[12:org.apache.aries.blueprint.core:1.7.1]
    at org.apache.aries.blueprint.container.BlueprintRepository.loadClass(BlueprintRepository.java:419)[12:org.apache.aries.blueprint.core:1.7.1]
    at org.apache.aries.blueprint.container.GenericType.parse(GenericType.java:135)
    at org.apache.aries.blueprint.di.AbstractRecipe.doLoadType(AbstractRecipe.java:168)[12:org.apache.aries.blueprint.core:1.7.1]
    at org.apache.aries.blueprint.di.AbstractRecipe.loadType(AbstractRecipe.java:161)[12:org.apache.aries.blueprint.core:1.7.1]
    at org.apache.aries.blueprint.container.BeanRecipe.loadClass(BeanRecipe.java:250)
    at org.apache.aries.blueprint.container.BeanRecipe.getType(BeanRecipe.java:917)
    at org.apache.aries.blueprint.container.BeanRecipe.getInstanceFromType(BeanRecipe.java:341)
    at org.apache.aries.blueprint.container.BeanRecipe.getInstance(BeanRecipe.java:282)
    at org.apache.aries.blueprint.container.BeanRecipe.internalCreate2(BeanRecipe.java:830)
    at org.apache.aries.blueprint.container.BeanRecipe.internalCreate(BeanRecipe.java:811)
    at org.apache.aries.blueprint.di.AbstractRecipe$1.call(AbstractRecipe.java:79)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_121]
    at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:88)[12:org.apache.aries.blueprint.core:1.7.1]
    at org.apache.aries.blueprint.container.BlueprintRepository.createInstances(BlueprintRepository.java:255)[12:org.apache.aries.blueprint.core:1.7.1]
    at org.apache.aries.blueprint.container.BlueprintRepository.createAll(BlueprintRepository.java:186)[12:org.apache.aries.blueprint.core:1.7.1]
    at org.apache.aries.blueprint.container.BlueprintContainerImpl.instantiateEagerComponents(BlueprintContainerImpl.java:724)[12:org.apache.aries.blueprint.core:1.7.1]
    ... 26 more
Caused by: java.lang.ClassNotFoundException: org.apache.camel.component.jms.JmsComponent not found by org.apache.activemq.activemq-osgi [55]
    at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1574)[org.apache.felix.framework-5.6.1.jar:]
    at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:79)[org.apache.felix.framework-5.6.1.jar:]
    at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:2018)[org.apache.felix.framework-5.6.1.jar:]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)[:1.8.0_121]
    ... 59 more

...and a sudo service karaf-service stop hangs for quite a while, and the karaf logs just have this looping over and over:

2017-03-31 02:50:40,035 | INFO  | 1 - ShutdownTask | DefaultShutdownStrategy          | 59 - org.apache.camel.camel-core - 2.18.1 | Waiting as there are still 4 inflight and pending exchanges to complete, timeout in 126 seconds. Inflights per route: [load-service-uri = 1, load-prepare = 1, loader-http = 1, load-extension = 1]
2017-03-31 02:50:40,035 | DEBUG | 1 - ShutdownTask | DefaultShutdownStrategy          | 59 - org.apache.camel.camel-core - 2.18.1 | There are 2 inflight exchanges:
    InflightExchange: [exchangeId=ID-claw-44506-1490928104423-2-2, fromRouteId=load-extension, routeId=load-extension, nodeId=to20, elapsed=520710, duration=520734]
    InflightExchange: [exchangeId=ID-claw-44506-1490928104423-2-3, fromRouteId=loader-http, routeId=load-service-uri, nodeId=process9, elapsed=520486, duration=520603]
2017-03-31 02:50:40,279 | INFO  | pool-68-thread-1 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/extensions
2017-03-31 02:50:40,280 | INFO  | pool-68-thread-1 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Container http://localhost:8080/fcrepo/rest/apix/extensions does not exist, adding
2017-03-31 02:50:40,306 | INFO  | pool-68-thread-2 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/ontologies
2017-03-31 02:50:40,307 | INFO  | pool-68-thread-2 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Container http://localhost:8080/fcrepo/rest/apix/ontologies does not exist, adding
2017-03-31 02:50:40,308 | INFO  | pool-68-thread-3 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/services
2017-03-31 02:50:40,309 | INFO  | pool-68-thread-3 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Container http://localhost:8080/fcrepo/rest/apix/services does not exist, adding
2017-03-31 02:50:41,036 | INFO  | 1 - ShutdownTask | DefaultShutdownStrategy          | 59 - org.apache.camel.camel-core - 2.18.1 | Waiting as there are still 4 inflight and pending exchanges to complete, timeout in 125 seconds. Inflights per route: [load-service-uri = 1, load-prepare = 1, loader-http = 1, load-extension = 1]
2017-03-31 02:50:41,036 | DEBUG | 1 - ShutdownTask | DefaultShutdownStrategy          | 59 - org.apache.camel.camel-core - 2.18.1 | There are 2 inflight exchanges:
    InflightExchange: [exchangeId=ID-claw-44506-1490928104423-2-2, fromRouteId=load-extension, routeId=load-extension, nodeId=to20, elapsed=521711, duration=521735]
    InflightExchange: [exchangeId=ID-claw-44506-1490928104423-2-3, fromRouteId=loader-http, routeId=load-service-uri, nodeId=process9, elapsed=521487, duration=521604]

...and finally, when I start the service again - sudo service karaf-service start - I'm still getting this over and over in the karaf logs:

2017-03-31 02:58:09,159 | INFO  | pool-26-thread-3 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/services
2017-03-31 02:58:09,160 | INFO  | pool-26-thread-3 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Container http://localhost:8080/fcrepo/rest/apix/services does not exist, adding
2017-03-31 02:58:09,957 | INFO  | pool-26-thread-1 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/extensions
2017-03-31 02:58:09,959 | INFO  | pool-26-thread-1 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Container http://localhost:8080/fcrepo/rest/apix/extensions does not exist, adding
2017-03-31 02:58:10,173 | INFO  | pool-26-thread-3 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/services
2017-03-31 02:58:10,175 | INFO  | pool-26-thread-2 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/ontologies
2017-03-31 02:58:10,176 | INFO  | pool-26-thread-2 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Container http://localhost:8080/fcrepo/rest/apix/ontologies does not exist, adding
2017-03-31 02:58:10,177 | INFO  | pool-26-thread-3 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Container http://localhost:8080/fcrepo/rest/apix/services does not exist, adding
2017-03-31 02:58:10,962 | INFO  | pool-26-thread-1 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/extensions
2017-03-31 02:58:10,964 | INFO  | pool-26-thread-1 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Container http://localhost:8080/fcrepo/rest/apix/extensions does not exist, adding
2017-03-31 02:58:11,178 | INFO  | pool-26-thread-2 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/ontologies
2017-03-31 02:58:11,180 | INFO  | pool-26-thread-3 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/services
2017-03-31 02:58:11,180 | INFO  | pool-26-thread-2 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Container http://localhost:8080/fcrepo/rest/apix/ontologies does not exist, adding
2017-03-31 02:58:11,182 | INFO  | pool-26-thread-3 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Container http://localhost:8080/fcrepo/rest/apix/services does not exist, adding
2017-03-31 02:58:11,966 | INFO  | pool-26-thread-1 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Looking for container http://localhost:8080/fcrepo/rest/apix/extensions
2017-03-31 02:58:11,969 | INFO  | pool-26-thread-1 | LdpContainerRegistry             | 189 - fcrepo-api-x-jena - 0.1.0 | Container http://localhost:8080/fcrepo/rest/apix/extensions does not exist, adding
ajs6f commented 7 years ago

That looks vaguely like a provisioning problem, like the camel-jms feature wasn't getting loaded or something. But that would seem really weird.

ajs6f commented 7 years ago

Pinging @emetsger and @birkland on this-- before I launch into it, does this look familiar at all? Any thoughts?

birkland commented 7 years ago

Hi all, I've been on the road, and now don't have internet at the moment except for my phone. Will try installing in Karaf by hand once I'm up and running again.

One class of messages (looking for container, adding, etc) are due to API-X being unable to contact Fedora. It just infinitely tries to verify that various registry containers are present. If you start API-X in isolation, it will keep doing this until you start Fedora.

As far as the OSGi provisioning, that is an unfamiliar error. The Karaf feature file declares fcrepo-service-activemq and fcrepo-camel as dependencies. I would think that camel-jms would simply be pulled in transitively. The Dockerfile used to create the docker image does nothing more than feature:install fcrepo-api-x, and works.

Once I get internet, I'll try to see if the errors you've been seeing can be reproduced.

On Apr 24, 2017 5:42 PM, "A. Soroka" notifications@github.com wrote:

Pinging @emetsger https://github.com/emetsger and @birkland https://github.com/birkland on this-- before I launch into it, does this look familiar at all? Any thoughts?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Islandora-CLAW/CLAW/issues/504#issuecomment-296830699, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfm-bAR_T1U3T-EGB0zmL_3L4wTffQdks5rzRdLgaJpZM4LrlIo .

ajs6f commented 7 years ago

Thanks, @birkland! I agree that camel-jms should just be pulled in, and if it weren't, other people should be seeing this kind of thing. So maybe somehow the bundles in camel-jms or bundles on which it relies aren't getting installed in this particular situation for some weird reason?

As for the other messages, that seems to imply that either Fedora isn't starting (which seems unlikely) or that API-X is looking in the wrong place for it, right?

birkland commented 7 years ago

I've found a lot of quirkiness in how Karaf provisions things; it's really hard to track down where things go wrong. Unfortunately, I have not been able to try/reproduce this.

As far as the other messages, yes it may be misconfigured. In org.fcrepo.apix.jena.cfg, use the properties registry.{extension, ontology, service}.ldp.container to specify the URIs to containers in Fedora. If the containers do not exist, it'll attempt to create.

So something like: registry.service.ldp.container = http://fcrepo.host:12345/fcrepo/rest/whatever/services

On Apr 25, 2017 3:29 PM, "A. Soroka" notifications@github.com wrote:

Thanks, @birkland https://github.com/birkland! I agree that camel-jms should just be pulled in, and if it weren't, other people should be seeing this kind of thing. So maybe somehow the bundles in camel-jms or bundles on which it relies aren't getting installed in this particular situation for some weird reason?

As for the other messages, that seems to imply that either Fedora isn't starting (which seems unlikely) or that API-X is looking in the wrong place for it, right?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Islandora-CLAW/CLAW/issues/504#issuecomment-297139394, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfm-d3ofFbkaSdu5zJnQQuJcfaDDswYks5rzkmpgaJpZM4LrlIo .

ajs6f commented 7 years ago

Okay, thanks @birkland, those are some good pointers. I'll carry on.

birkland commented 7 years ago

If you have any suggestions for simplification, please pass them along. What is there right now might be redundant in a few areas, or hard to follow. A clean up might be worthwhile. What's there right now largely bakes blueprint happy, without much regard to the humans.

On Apr 25, 2017 5:42 PM, "A. Soroka" notifications@github.com wrote:

Okay, thanks @birkland https://github.com/birkland, those are some good pointers. I'll carry on.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Islandora-CLAW/CLAW/issues/504#issuecomment-297173644, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfm-QRVsbmvVXZvEybuzuk2Fx07pnGSks5rzmibgaJpZM4LrlIo .

whikloj commented 7 years ago

Hey!

I am getting this exact problem on a Islandora 1.X camel route I've been writing.

I think it might have to do with camel and activemq version issues.

I haven't solved this in my route, but I'll add some info I found later.

On 25 Apr 2017 4:45 p.m., "Aaron Birkland" notifications@github.com wrote:

If you have any suggestions for simplification, please pass them along. What is there right now might be redundant in a few areas, or hard to follow. A clean up might be worthwhile. What's there right now largely bakes blueprint happy, without much regard to the humans.

On Apr 25, 2017 5:42 PM, "A. Soroka" notifications@github.com wrote:

Okay, thanks @birkland https://github.com/birkland, those are some good pointers. I'll carry on.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Islandora-CLAW/CLAW/issues/504# issuecomment-297173644, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfm- QRVsbmvVXZvEybuzuk2Fx07pnGSks5rzmibgaJpZM4LrlIo .

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Islandora-CLAW/CLAW/issues/504#issuecomment-297174545, or mute the thread https://github.com/notifications/unsubscribe-auth/ACua4SCY56FI31LD_SrfVxN79rntWUrEks5rzmlwgaJpZM4LrlIo .

dannylamb commented 7 years ago

I've experienced it too when working working on a local Karaf before sending to the one in Vagrant. Even with the camel-jms feature, the activemq feature fails to really activate, despite saying so.

I'm pretty sure we should try as @acoburn has suggested here and see if that resolves it.

ruebot commented 7 years ago

@dannylamb that's what I was trying, and my above comment is the output from those attempts. At least I think it was. Memory is getting hazy :smile:

birkland commented 7 years ago

I'm about to try doing this locally, and will verify that Docker still builds a working image. Is there anything else you're adding to Karaf, or is it just API-X?

dannylamb commented 7 years ago

@ruebot Sorry bout that. I misunderstood the lengthy upscroll on this issue.

@birkland We're also adding some Amherst extensions as well as Alpaca, but you can verify this in isolation using anything with fcrepo-service-activemq.

whikloj commented 7 years ago

This is what I found and am since trying downgrading ActiveMQ in my app. https://issues.apache.org/jira/browse/CAMEL-10405?focusedCommentId=15595258&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15595258

DiegoPino commented 7 years ago

@whikloj could it be the missing/explicit connection factory? according to this http://camel.465427.n5.nabble.com/camel-2-18-activemq-not-working-under-karaf-td5789455.html

And API-X does https://github.com/fcrepo4-labs/fcrepo-api-x/blob/551969ca50394035e4a64b2ff554ea85f4ac90e4/proof-of-concept/indexing-triplestore/src/main/resources/applicationContext.xml#L9-L11

Maybe I'm looking at the wrong spot in API-X, but for Karaf 4+ and camel 2.18.x connection factory seems to be required.

dannylamb commented 7 years ago

@DiegoPino We are using fcrepo-service-activemq, which uses a connection factory. https://github.com/fcrepo4-exts/fcrepo-camel-toolbox/blob/master/fcrepo-service-activemq/src/main/resources/OSGI-INF/blueprint/blueprint.xml

dannylamb commented 7 years ago

@whikloj Hrm... the link you found seems to be for running an embedded broker in Karaf, which is ideally what we'd be doing. But we're being lazy and using the embedded broker in fcrepo, which does seem to be working somehow. Still, if you attempt the downgrade, I'm curious to know what your results are.

We're unfortunately stuck wtih Camel 2.18+ for fcrepo-camel and the bits of fcrepo-camel-toolbox we use. I'd rather not swap broker technologies right now, but it may be an inevitability.

whikloj commented 7 years ago

@dannylamb yeah I'll let you know. My love of Camel is tempered by my hate...also for Camel 😄

birkland commented 7 years ago

OK, I was able to get everything up and running by hand in Karaf; running against a separate fedora. It worked; no jms issue or configuration issues. Here's what I did.

It worked! A few defaults are relied upon (i.e. the broker is at localhost:61616), but as far as a smoke test I didn't see the JMS provisioning cause problems.

ajs6f commented 7 years ago

Whoa, tons of activity while I was away! Nice to see everyone ganging up on an issue and treating it like Rocky Balboa treats a side of beef. IIUC the results so far:

  1. There is general agreement that this seems to be about JMS and possibly API-X config.
  2. @birkland has gotten a "clean" API-X up and running from scratch, which implies that there is something more CLAW-Vagrant specific about this.
  3. We are currently relying on the broker config included with Fedora, which may not be causing this, but is definitely not optimal.

Okay, unless someone thinks this is crazy, I'm going to try:

  1. Keeping versions as they are, for the reasons @dannylamb gave.
  2. Breaking out an independent AMQ broker housed in Karaf that every other component uses, because if API-X, Fedora, and camel-toolbox all need this one service, that means (to me) that it's time to get it out in the clear and make sure it is strong and well-config'd.
  3. Making sure that everything that relies on that central AMQ uses store-and-forward to connect to it, so that we don't have startup ordering fragility.

Does that sound like a reasonable place to go for CLAW-Vagrant?

dannylamb commented 7 years ago

@ajs6f According to Claus in a Jira issue that @whikloj provided we can't run a ActiveMQ in Karaf and use Camel 2.18+.

Maybe just do an apt-get install activemq to run it as a service if you want to break it out?

ajs6f commented 7 years ago

@dannylamb Oops, didn't catch that. I will investigate and if I can't see a tighter workaround, then yes, I would propose a process-independent broker.

birkland commented 7 years ago

So far, the incompatibilities between ActiveMQ and Camel 2.18+ in Karaf seem to be fairly benign. See: https://github.com/fcrepo4-labs/fcrepo-api-x/issues/94

So consuming from ActiveMQ 5.14 + Camel 2.18 works OK (albeit with some exceptions that don't seem to have any observable effect). I've never attempted to put a broker in Karaf, but I don't see any reason why It wouldn't work. The incompatibility between ActiveMQ 5.14 and Camel 2.18 is all on the client side, as far as I understand it.

ajs6f commented 7 years ago

Well, @davsclaus was pretty firm about it in that Camel ticket and no one would know better than he, but the email @DiegoPino uncovered does seem to imply, as you say, that AMQ5.14-in-Karaf can support Camel 2.18. I think I will have to just stick my foot in that bucket and see how far I can drag it.

ajs6f commented 7 years ago

@ruebot You didn't end up with a branch for this, right? You were just working off of a clean master vagrant and manually trying to get API-X into the runtime, right?

whikloj commented 7 years ago

The issue I have is less about the broker being inside Karaf and more about using camel-jms and ActiveMQ together inside karaf. ie. this is what fails consistently https://github.com/whikloj/islandora-1x-derivative-toolkit/blob/master/islandora-1x-gatekeeper/src/main/resources/OSGI-INF/blueprint/blueprint.xml#L36-L39

ajs6f commented 7 years ago

@whikloj Seems like that is relevant here though, eh? Because camel-toolbox will be in the same position of trying to use a camel-jms client within Karaf to an AMQ broker also within the same Karaf?

ruebot commented 7 years ago

@ajs6f nope, no branch on this since https://github.com/Islandora-CLAW/claw_vagrant/commit/8eb5513a561d444463fd36eddb4f7b9ae0d320e8 -- I was just testing on a master build a while back and trying to implement what @acoburn recommended in the API-X ticket.

whikloj commented 7 years ago

@ajs6f I agree but this could also be relative amateurish work in OSGI that is causing the problems. But if I figure out something, I will definitely report back.

ajs6f commented 7 years ago

@ruebot @whikloj Okay, sounds like it is as least worth me starting where @ruebot got to and just seeing how far I get with the new knowledge in this ticket.

ajs6f commented 7 years ago

After a good bit of tinkering about today, most of which was really just me learning claw_vagrant, I think I understand that the root of the container problem is that API-X is unaware of CLAW's Syn authN system, and thus fails to create the containers it wants on startup. The JMS question is a different story, but I am currently inclined to leave it be for the nonce and fix the authN problem first, to get a clean API-X startup.

This hypothesis does have the advantage of predicting that (as happened) @birkland would be able to start API-X cleanly in a non-CLAW env, but that I (in claw-vagrant) would see the same thing @ruebot did, which I did.

birkland commented 7 years ago

Hi @ajs6f ,

Ah, I see. API-X needs to be able to bootstrap and maintain itself (i.e. create containers in the repository for registries, persist extension definitions, ontologies, etc). If CLAW's repo is locked down, then API-X would need to be configured with credentials in order to maintain the data it needs to. So far, we haven't needed to do that. I think some of the API-X blueprint files would need to be updated in order to make that possible.

fcrepo-api-x-jena/LdpContainerRegistry is the base impl that does such writing. It has an HttpClient injected here. That's a service reference from the bundle's blueprint file. The blueprint file from fcrepo-api-x-registry exports an HttpClient as a service, but unfortunately it isn't configured for authentication.

One approach to adding auth would be to configure the client exported in fcrepo-api-x-registry to authenticate with the appropriate credentials to the desired realm (i.e the Fedora instance, at whatever host it's at). That would require updating the one blueprint file.

Another approach might be to not import an HttpClient as a service reference in fcrepo-api-x-jena, and just build a new one there in its blueprint file; adding the appropriate config properties to add credentials when auth is desired.

I'm not sure which approach is best...

dannylamb commented 7 years ago

If it's possible for CLAW to publish its own in Alpaca, then that's probably the least invasive way to do it. Api-X can provide a default that we just would supplant with our own.

As far as getting that token, we could provide a static one for Api-X or find some other way to get it by giving Api-X a Drupal user/pass and using that to get a token.

ajs6f commented 7 years ago

From experience with exactly this problem in Jena, I think it's better long-term to treat with actual clients, rather than try to think of every possible option or weirdness that people might want to inflict on HTTP. But in additional to HttpClient, we would want also to have HttpContext available, for authN patterns that require state in the conversation.

ajs6f commented 7 years ago

Sounds like the immediate short-term fix for CLAW is to set the right config in blueprint using Vagrant setup scripts or the like. For the long term, API-X can decide on a pattern and CLAW can adjust to match. That sound good?

birkland commented 7 years ago

Sounds good to me. I need to better understand the Auth setup CLAW uses, buy would be happy to implement whatever it takes on the API-X end to make it straightforward!

dannylamb commented 7 years ago

@birkland, Based on some cursory looks, it appears that separating the blueprint from the code would let us provide our own config and therefore http client. Or we can take the one that's declared as a service in api-x, give it a jndi name, and then make the bean refs that look for it accept a configurable jndi name. In Alpaca, CLAW can just provide its own bundle that exports an http client as another name and just update configuration to pull in the right one from jndi. I think that'd be the least invasive option.

I can help out with that right after I wrap up https://github.com/Islandora-CLAW/CLAW/issues/597, which should be soon.