search

cschneider / Karaf-Tutorial

http://cschneider.github.io/Karaf-Tutorial/
Apache License 2.0
271 stars 305 forks source link

tasklist-itests: is it reliable? #38

Closed ghost closed 6 years ago

ghost commented 6 years ago

Hello,

I was doing the tasklist tutorial and it went all fine, meaning that I could do everything shown there successfully (model, persistance, ui and command worked fine).

The only changes I did were to main pom (changed to Java8) and servlet (using "alias" property) as I'm on Karaf 4.0.10: ` corrado@powerdesk2:~/karaf/SRC/Karaf-Tutorial/tasklist$ git status On branch master Your branch is up-to-date with 'origin/master'. Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git checkout -- ..." to discard changes in working directory)

modified:   pom.xml
modified:   tasklist-ui/src/main/java/net/lr/tasklist/ui/TaskListServlet.java

no changes added to commit (use "git add" and/or "git commit -a") `

This is the env: ` karaf@root()> info Karaf Karaf version 4.0.10 Karaf home /home/corrado/karaf/ENV Karaf base /home/corrado/karaf/ENV OSGi Framework org.apache.felix.framework-5.6.4

JVM Java Virtual Machine Java HotSpot(TM) 64-Bit Server VM version 25.112-b15 Version 1.8.0_112 Vendor Oracle Corporation Pid 9096 Uptime 20 hours 55 minutes Total compile time 14.135 seconds Threads Live threads 60 Daemon threads 45 Peak 69 Total started 396 Memory Current heap size 130,033 kbytes Maximum heap size 466,432 kbytes Committed heap size 216,576 kbytes Pending objects 0 Garbage collector Name = 'PS Scavenge', Collections = 26, Time = 0.153 seconds Garbage collector Name = 'PS MarkSweep', Collections = 21, Time = 0.722 seconds Classes Current classes loaded 7,203 Total classes loaded 7,307 Total classes unloaded 104 Operating system Name Linux version 3.16.0-4-amd64 Architecture amd64 Processors 8 `

My question is about the itest, which apparently "went fine":

[INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] tasklist ........................................... SUCCESS [ 0.596 s] [INFO] tasklist-model ..................................... SUCCESS [ 1.310 s] [INFO] tasklist-persistence ............................... SUCCESS [ 0.390 s] [INFO] tasklist-ui ........................................ SUCCESS [ 0.294 s] [INFO] tasklist-features .................................. SUCCESS [ 0.205 s] [INFO] tasklist-command ................................... SUCCESS [ 0.249 s] [INFO] Integration Tests using pax exam ................... SUCCESS [01:13 min] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 01:16 min [INFO] Finished at: 2017-10-09T09:28:06+02:00 [INFO] Final Memory: 41M/506M [INFO] ------------------------------------------------------------------------

But this is what should have been tested, right?

I mean: "test1()" should have been executed, right?

` public class KarafTest { @Inject protected BundleContext bundleContext;

@Inject
protected TaskService taskService;

...

@Test
public void test1() throws Exception {
    Collection<net.lr.tasklist.model.Task> tasks = taskService.getTasks();
    Assert.assertEquals(2, tasks.size());
}

} `

Well, I had another instance of Karaf running, the one I reported the "karaf@root()> info" logs from, so the "pax-exam-generated-instance" failed starting:

`

T E S T S

Running net.lr.tasklist.itests.KarafTest log4j:WARN No appenders could be found for logger (org.ops4j.pax.exam.spi.DefaultExamSystem). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Exception in thread "JMX Connector Thread [service:jmx:rmi://0.0.0.0:44444/jndi/rmi://0.0.0.0:1099/karaf-root]" java.lang.RuntimeException: Port already in use: 44444; You may have started two containers. If you need to start a second container or the default ports are already in use update the config file etc/org.apache.karaf.management.cfg and change the Registry Port and Server Port to unused ports at org.apache.karaf.management.ConnectorServerFactory$1.run(ConnectorServerFactory.java:268)

   / //_/____ __________ _/ __/      
  / ,<  / __ `/ ___/ __ `/ /_        
 / /| |/ /_/ / /  / /_/ / __/        
/_/ |_|\__,_/_/   \__,_/_/

Apache Karaf (4.0.8)

Hit '' for a list of available commands and '[cmd] --help' for help on a specific command. Hit '' or type 'system:shutdown' or 'logout' to shutdown Karaf.

karaf@root()> 2017-10-09 09:28:04,589 | WARN | pool-22-thread-1 | Activator | 56 - org.apache.karaf.shell.ssh - 4.0.8 | Exception caught while starting SSH server java.net.BindException: Address already in use at ... at org.apache.karaf.features.internal.service.FeaturesServiceImpl$1.call(FeaturesServiceImpl.java:1074)[8:org.apache.karaf.features.core:4.0.8] at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_112] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_112] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_112] at java.lang.Thread.run(Thread.java:745)[:1.8.0_112]

2017-10-09 09:28:05,071 | WARN | FelixStartLevel | core | 24 - org.apache.aries.jmx.core - 1.1.7 | Task rejected for JMX Notification dispatch of event [org.osgi.framework.ServiceEvent[source=[javax.management.MBeanServer]]] - Dispatcher may have been shutdown Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 68.066 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] ... `

If so, how could it be that it "found 2 tasks"?

@Test public void test1() throws Exception { Collection<net.lr.tasklist.model.Task> tasks = taskService.getTasks(); Assert.assertEquals(2, tasks.size()); }

Notice the "other Karaf instance" had 3 tasks:

` karaf@root()> task:list id | title

1 | Buy some coffee
2 | Finish karaf tutorial 4 | maciao
`

So, where the heck the "injected TaskService" comes from?

@Inject protected TaskService taskService;

Shouldn't that come from the "failed to start" karaf instance?

Shouldn't that be null, then?

Shouldn't the integration test fails, more in general, if the "target runtime" fails to run?

I'm attaching the full logs for "mvn clean install".

cheers corrado test-failure.txt

cschneider commented 6 years ago

The test should fail if the Karaf test instance fails to start.

In your case though it started fine. It just was not able to off JMX and SSH but all that was needed by the test seemed to have come up fine.

The best way is to fine tune the test to not even try to start JMX and SSH but I did not do this for the tutorial but those errors in the log are not a real problem .. just a bit confusing.

Christian

2017-10-09 9:51 GMT+02:00 corrado notifications@github.com:

Hello,

I was doing the tasklist tutorial http://www.liquid-reality.de/display/liquid/2011/02/15/Karaf+Tutorial+Part+1+-+Installation+and+First+application and it went all fine, meaning that I could do everything shown there successfully (model, persistance, ui and command worked fine).

The only changes I did were to main pom (changed to Java8) and servlet (using "alias" property) as I'm on Karaf 4.0.10: ` corrado@powerdesk2:~/karaf/SRC/Karaf-Tutorial/tasklist$ git status On branch master Your branch is up-to-date with 'origin/master'. Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git checkout -- ..." to discard changes in working directory)

modified: pom.xml modified: tasklist-ui/src/main/java/net/lr/tasklist/ui/TaskListServlet.java

no changes added to commit (use "git add" and/or "git commit -a") `

This is the env: ` karaf@root()> info Karaf Karaf version 4.0.10 Karaf home /home/corrado/karaf/ENV Karaf base /home/corrado/karaf/ENV OSGi Framework org.apache.felix.framework-5.6.4

JVM Java Virtual Machine Java HotSpot(TM) 64-Bit Server VM version 25.112-b15 Version 1.8.0_112 Vendor Oracle Corporation Pid 9096 Uptime 20 hours 55 minutes Total compile time 14.135 seconds Threads Live threads 60 Daemon threads 45 Peak 69 Total started 396 Memory Current heap size 130,033 kbytes Maximum heap size 466,432 kbytes Committed heap size 216,576 kbytes Pending objects 0 Garbage collector Name = 'PS Scavenge', Collections = 26, Time = 0.153 seconds Garbage collector Name = 'PS MarkSweep', Collections = 21, Time = 0.722 seconds Classes Current classes loaded 7,203 Total classes loaded 7,307 Total classes unloaded 104 Operating system Name Linux version 3.16.0-4-amd64 Architecture amd64 Processors 8 `

My question is about the itest, which apparently "went fine":

[INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] tasklist ........................................... SUCCESS [ 0.596 s] [INFO] tasklist-model ..................................... SUCCESS [ 1.310 s] [INFO] tasklist-persistence ............................... SUCCESS [ 0.390 s] [INFO] tasklist-ui ........................................ SUCCESS [ 0.294 s] [INFO] tasklist-features .................................. SUCCESS [ 0.205 s] [INFO] tasklist-command ................................... SUCCESS [ 0.249 s] [INFO] Integration Tests using pax exam ................... SUCCESS [01:13 min] [INFO]

[INFO] BUILD SUCCESS [INFO] ------------------------------ ------------------------------------------ [INFO] Total time: 01:16 min [INFO] Finished at: 2017-10-09T09:28:06+02:00 [INFO] Final Memory: 41M/506M [INFO] ------------------------------------------------------------

But this is what should have been tested, right?

I mean: "test1()" should have been executed, right?

` public class KarafTest { @Inject https://github.com/inject protected BundleContext bundleContext;

@Inject protected TaskService taskService;

...

@Test public void test1() throws Exception { Collection tasks = taskService.getTasks(); Assert.assertEquals(2, tasks.size()); }

} `

Well, I had another instance of Karaf running, the one I reported the "karaf@root()> info" logs from, so the "pax-exam-generated-instance" failed starting: ` T E S T S

Running net.lr.tasklist.itests.KarafTest log4j:WARN No appenders could be found for logger (org.ops4j.pax.exam.spi. DefaultExamSystem). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Exception in thread "JMX Connector Thread [service:jmx:rmi://0.0.0.0: 44444/jndi/rmi://0.0.0.0:1099/karaf-root]" java.lang.RuntimeException: Port already in use: 44444; You may have started two containers. If you need to start a second container or the default ports are already in use update the config file etc/org.apache.karaf.management.cfg and change the Registry Port and Server Port to unused ports at org.apache.karaf.management.ConnectorServerFactory$1.run( ConnectorServerFactory.java:268)

/ ///_ ____

/ / / ,< / / / __/ / / /| |/ // / / / / / / __/ // ||_,// ,//

Apache Karaf (4.0.8)

Hit '' for a list of available commands and '[cmd] --help' for help on a specific command. Hit '' or type 'system:shutdown' or 'logout' to shutdown Karaf.

karaf@root()> 2017-10-09 09:28:04,589 | WARN | pool-22-thread-1 | Activator | 56 - org.apache.karaf.shell.ssh - 4.0.8 | Exception caught while starting SSH server java.net.BindException: Address already in use at ... at org.apache.karaf.features.internal.service.FeaturesServiceImpl$1.call( FeaturesServiceImpl.java:1074)[8:org.apache.karaf.features.core:4.0.8] at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_112] at java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1142)[:1.8.0_112] at java.util.concurrent.ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:617)[:1.8.0_112] at java.lang.Thread.run(Thread.java:745)[:1.8.0_112]

2017-10-09 09:28:05,071 | WARN | FelixStartLevel | core | 24 - org.apache.aries.jmx.core - 1.1.7 | Task rejected for JMX Notification dispatch of event [org.osgi.framework.ServiceEvent[source=[javax.management.MBeanServer]]]

  • Dispatcher may have been shutdown Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 68.066 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] ... `

If so, how could it be that it "found 2 tasks"?

@Test public void test1() throws Exception { Collection tasks = taskService.getTasks(); Assert.assertEquals(2, tasks.size()); }

Notice the "other Karaf instance" had 3 tasks: ` karaf@root()> task:list id | title

1 | Buy some coffee 2 | Finish karaf tutorial 4 | maciao `

So, where the heck the "injected TaskService" comes from?

@Inject protected TaskService taskService;

Shouldn't that come from the "failed to start" karaf instance?

Shouldn't that be null, then?

Shouldn't the integration test fails, more in general, if the "target runtime" fails to run?

I'm attaching the full logs for "mvn clean install".

cheers corrado test-failure.txt https://github.com/cschneider/Karaf-Tutorial/files/1367781/test-failure.txt

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cschneider/Karaf-Tutorial/issues/38, or mute the thread https://github.com/notifications/unsubscribe-auth/AAdk6KVIOE3d9Ql2d-bmRJ-gbB8JUaWFks5sqdCegaJpZM4PyG_g .

-- -- Christian Schneider http://www.liquid-reality.de https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.liquid-reality.de

Computer Scientist http://www.adobe.com

ghost commented 6 years ago

Hi Chris, thx for your reply.

In general that's ok (I mean fine tuning and not starting JMX and SSH), but here also jetty and paxweb got an "already used IP" exception. These components (needed for the tasklist-ui module) were not needed by that single test, which is restricted to test just the tasklist-model and tasklist-persistence modules, that's why the single test succeeded. I'm ok with that.

But in the case the integration testing covered the tasklist-ui module too, it should fail, or depend on what's on the other instance, right? This should apply to all -rest modules too, right?

PS: running the integration test created a folder containing something like a "karaf distribution":

screenshot from 2017-10-09 12-37-31

how far is this folder from being a real "custom karaf distribution"?

cheers corrado

cschneider commented 6 years ago

Yes .. tests that depend on a fixed port will fail or do strange things if that port is already used in the system.

What I have seen is that people use a random free port and set karaf to use it for e.g. http.

Pax exam uses the plain karaf archive and then install the given features and config. This is not the same as a custom distro.

In a custom distro you typically deploy your bundles and feature xmls into the karaf system dir and already preinstall the features and configs.

If you want to do this then you can let pax exam point to your custom distro.

Christian

2017-10-09 12:30 GMT+02:00 corrado notifications@github.com:

Hi Chris, thx for your reply.

In general that's ok (I mean fine tuning and not starting JMX and SSH), but here also jetty and paxweb got an "already used IP" exception. These components (needed for the tasklist-ui module) were not needed by that single test, which is restricted to test just the tasklist-model and tasklist-persistence modules, that's why the single test succeeded. I'm ok with that.

But in the case the integration testing covered the tasklist-ui module too, it should fail, or depend on what's on the other instance, right? This should apply to all -rest modules too, right?

PS: running the integration test created a folder containing something like a "karaf distribution":

[image: screenshot from 2017-10-09 12-37-31] https://user-images.githubusercontent.com/25619972/31334179-630a8728-aced-11e7-9be2-f453c85bd265.png

how far is this folder from being a real "custom karaf distribution"?

cheers corrado

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cschneider/Karaf-Tutorial/issues/38#issuecomment-335119700, or mute the thread https://github.com/notifications/unsubscribe-auth/AAdk6IdqHJn_mDdHP_Ye8PmRKZT6aSzsks5sqfW5gaJpZM4PyG_g .

-- -- Christian Schneider http://www.liquid-reality.de https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.liquid-reality.de

Computer Scientist http://www.adobe.com

ghost commented 6 years ago

thx Chris, excellent explanation indeed, appreciated.

May I abuse your competence and kindness by asking one more question, before closing the issue?

As you can see from the links below, I'm going thru several of these tutorials, as I'm trying to "port" (and extend) a simple tomcat webapp to karaf, mainly to get hands-on experience about it (I used ServiceMix 3.x in the past, while I wish I can use universAAL in the future).

So far, I have a small NAS (was a Qnap, now runs debian), where my mp4 files are

I'd like to extend it by reusing some stuff from YAMJ-v3 (db, scanner, plugins, but not the webapp stuff), so I've been thinking about a karaf custom distribution, with:

That way:

Does this make sense to you? Looks feasible?

thx corrado

cschneider commented 6 years ago

That sounds feasible. I would maybe skip the jpa support though. JPA is difficult to do in general and even a bit harder on OSGi. Maybe jdbc4 is good enough for your case.

Christian

2017-10-09 15:08 GMT+02:00 corrado notifications@github.com:

thx Chris, excellent explanation indeed, appreciated.

May I abuse your competence and kindness by asking one more question, before closing the issue?

As you can see from the links below, I'm going thru several of these tutorials, as I'm trying to "port" (and extend) a simple tomcat webapp to karaf, mainly to get hands-on experience about it (I used ServiceMix 3.x in the past, while I wish I can use universAAL in the future).

So far, I have a small NAS (was a Qnap, now runs debian), where my mp4 files are

  • I developed a tomcat webapp first, which has 1 html5 page and 2 servlets:
    • the html5 page has 2 section: browsing (folders or categories) and playing (mp4 videos)
    • for browsing, it talks JSON with the "browsing servlet"
    • for playing, it plays HTTP stream from the "pseudo-stream" servlet
  • since the web page was displaying poorly on my smart TVs browsers, I developed a samsung tv app, which does the same as the web page, but looks a lot better and is easy to control with the TV remote.

I'd like to extend it by reusing some stuff from YAMJ-v3 (db, scanner, plugins, but not the webapp stuff), so I've been thinking about a karaf custom distribution, with:

That way:

  • any PC-like device will use HTML5 player from the war bundle web page
  • any TV-like device will use an app which does what the web page does

Does this make sense to you? Looks feasible?

thx corrado

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cschneider/Karaf-Tutorial/issues/38#issuecomment-335151843, or mute the thread https://github.com/notifications/unsubscribe-auth/AAdk6FG-pwJCyFm6dUc3vDL5g9e-NAAMks5sqhntgaJpZM4PyG_g .

-- -- Christian Schneider http://www.liquid-reality.de https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.liquid-reality.de

Computer Scientist http://www.adobe.com

ghost commented 6 years ago

thx Chris

regarding jpa vs jdbc...

I can see jpa uses hibernate, while jdbc uses raw SQL, right?

I've been using SQL-in-code since 2000 (asp pages was "new tech" at those times) and recently happened to use hibernate successfully in a tomcat webapp:

screenshot from 2017-10-09 17-15-46

what's harder on OSGi, using hibernate?

thx corrado

cschneider commented 6 years ago

if you are familiar with hibernate and jpa already then give it a try.

In OSGi the DataSource setup is a bit tricky .. especially if you need XA transactions. With the tutorial code it should be easier.

Christian

2017-10-09 17:15 GMT+02:00 corrado notifications@github.com:

thx Chris

regarding jpa vs jdbc...

I can see jpa uses hibernate https://github.com/cschneider/Karaf-Tutorial/blob/master/db/examplejpa/src/main/resources/OSGI-INF/blueprint/blueprint.xml, while jdbc uses raw SQL https://github.com/cschneider/Karaf-Tutorial/blob/master/db/examplejdbc/src/main/java/net/lr/tutorial/karaf/db/example/DbExample.java, right?

I've been using SQL-in-code since 2000 (asp pages was "new tech" at those times) and recently happened to use hibernate successfully https://www.linkedin.com/feed/update/urn:li:activity:6290800371988201472/ in a tomcat webapp:

[image: screenshot from 2017-10-09 17-15-46] https://user-images.githubusercontent.com/25619972/31344892-5528148c-ad14-11e7-9553-7cd374111e11.png

what's harder on OSGi, using hibernate?

thx corrado

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cschneider/Karaf-Tutorial/issues/38#issuecomment-335188876, or mute the thread https://github.com/notifications/unsubscribe-auth/AAdk6MJgAFh15ZLZIRJwWESGUGFmK5Fqks5sqjiJgaJpZM4PyG_g .

-- -- Christian Schneider http://www.liquid-reality.de https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.liquid-reality.de

Computer Scientist http://www.adobe.com

ghost commented 6 years ago

perfect

thx again