Closed mcreteau closed 9 years ago
hstaudacher
commented
9 years ago We are using the connector for several productive applications with high load. It would be good if you can provide a standalone example. E.g. an example where you can easily configure the service instances registered to reproduce the performance problem. It would also be nice to have some tim measurement in it to see the startup time.
Can you provide such an example?
mcreteau
commented
9 years ago Hi holger,
I’ll try to find the time to create a standalone example, but as mentioned in the issue, this problem can only be noticed on an embedded or very slow platform. When testing the same application on a standard PC, everything works smoothly.
In the mean time, I have made a “dirty” modification in JerseyContext class by using a Timer that is triggered on the add/remove resource to introduce a delay in resource registration. If another resource is registered before the timer is triggered, we cancel the timer and re-run it.
This allows me to reduce the startup and stop time of the publisher bundle from 40 seconds to around 1 second.
Mathieu CRETEAU
Référent Système BG
Hager Controls SAS
11 rue Pablo Neruda
33140 Villenave d’Ornon
France
N° Tel 0557359496
N° Fax 0556806662
Adresse : mathieu.creteau@hager.fr
SAS au capital de 5 000 000€
N° Id. TVA FR 75 451 540 744 / APE 2712Z
RCS Saverne 451 540 744
N° Siret 451 540 744 000 13
De : Holger Staudacher [mailto:notifications@github.com] Envoyé : lundi 15 septembre 2014 19:17 À : hstaudacher/osgi-jax-rs-connector Cc : mcreteau Objet : Re: [osgi-jax-rs-connector] Performances issues when adding or removing multiple resources (#60)
We are using the connector for several productive applications with high load. It would be good if you can provide a standalone example. E.g. an example where you can easily configure the service instances registered to reproduce the performance problem. It would also be nice to have some tim measurement in it to see the startup time.
Can you provide such an example?
— Reply to this email directly or view it on GitHub https://github.com/hstaudacher/osgi-jax-rs-connector/issues/60#issuecomment-55624439 . https://github.com/notifications/beacon/352929__eyJzY29wZSI6Ik5ld3NpZXM6QmVhY29uIiwiZXhwaXJlcyI6MTcyNjQyMDYxOCwiZGF0YSI6eyJpZCI6NDIyODkyODV9fQ==--657101c15254b075c6f1e657e8ba8f9b369084f3.gif
kaikreuzer
commented
9 years ago Hi, I was just about to enter the same issue, when I stumbled about this entry. I have introduced your connector in Eclipse SmartHome / openHAB 2, but I noticed the same performance issues as Mathieu when running it on a Raspberry Pi. I have a sample package available here: https://openhab.ci.cloudbees.com/job/openHAB2/lastSuccessfulBuild/artifact/distribution/target/distribution-2.0.0-SNAPSHOT-minimal-runtime.zip When this is started with start_debug.sh, I get the following log entries, which show that every single REST resource adds in average 5 seconds to the startup time:
17:36:09.213 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
17:36:17.286 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
17:36:19.464 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
17:36:22.509 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
17:36:25.410 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
17:36:28.376 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
17:36:31.793 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
17:36:35.849 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
17:36:40.303 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
17:36:44.365 INFO o.o.c.internal.CoreActivator[:55] - openHAB runtime has been started (v2.0.0, build 201411141008).Running the same on my MacBook, it looks of course much better:
12:01:54.946 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
12:01:55.009 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
12:01:55.061 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
12:01:55.125 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
12:01:55.232 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
12:01:55.322 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
12:01:55.403 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
12:01:55.503 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
12:01:55.643 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...Would love to have a solution for this as well as otherwise the OSGi-JAXRS-Connector disqualifies itself for use on embedded hardware (which would be a pity)!
hstaudacher
commented
9 years ago I will have time in the next two weeks to work on the connector again. This issue will b fixed in this period.
@kaikreuzer can I send you a pre-release to test? Just to see if the performance issue is resolved.
kaikreuzer
commented
9 years ago Sure, I'd love to be your beta tester :-)
kaikreuzer
commented
9 years ago Ping - two weeks are over :-) Did you make any progress?
ieugen
commented
9 years ago I would recommend amdatu jax-rs [1]. I know the people from luminis are using it in production. Unfortunately it's not available on Maven Central, but I think you can deploy it manually if needed.
kaikreuzer
commented
9 years ago Thanks for the hint, this looks indeed interesting as well.
hstaudacher
commented
9 years ago Scheduled for today. Sorry for the delay
hstaudacher
commented
9 years ago Hey guys, as you can see in the comment above I have created a fix. You can find a build containing this fix within this repo: https://dl.dropboxusercontent.com/u/5808972/com.eclipsesource.jaxrs.repository-4.1.1-issue60fix.zip
@kaikreuzer can you test it out and report if it works? I have tested it with one of our applications that registers about 60 resources and it improves the startup tremandously.
Regarding the solution. I searched a better way to modify the jersey resource model at runtime but from what I found it is not possible. Implementations like JRebel and co also call the reload method on the ServletContainer to do hot swaping. So, we still rely on the reload method but I wrapped it in a ScheduledExecutorService. It runs a runnable every 2500ms (configurable). The runnable checks if there are new or deleted resources and does a reload if true. If not it simply does nothing. The runnable starts with a delay of 1 second (felt best to during testing).
I creates the issue60 branch which contains the fix. i just wanted to hear your opinion before it goes to master.
kaikreuzer
commented
9 years ago Hi Holger, Thanks a lot, it looks good! All I see are now two initializations:
01:36:37.605 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...
01:36:44.811 INFO o.g.j.s.ApplicationHandler[:353] - Initiating Jersey application, version Jersey: 2.10.1 2014-06-30 13:53:32...and I guess with configuring the runnable with a bigger value, I could get it down to just one? Anyhow, startup time is now almost 30 s less on my RaspPi - mission complished!
hstaudacher
commented
9 years ago Hi @kaikreuzer thanks for testing. It's good to hear that it's better now. Anyway, yesterday I experienced something strange. I updated Jersey to 2.13.0 in master (without this fix). It seems the Jersey team has reworked their "ServletContainer" and this issue could be resolved without the fix described. I have created a bundle that registers 20 resources. With Jersey 2.13.0 I only get one initialization.
I have created a test build. Could you please test it and verify? Thanks. https://dl.dropboxusercontent.com/u/5808972/com.eclipsesource.jaxrs.repository-4.2.0-issue60fix.zip
kaikreuzer
commented
9 years ago Hm, with this version I do not get any log entry from Jersey anymore. The REST services are also not available until I restart the bundle com.eclipsesource.jaxrs.publisher (might be a startup order issue, I didn't further investigate this). The startup time has again increased by 10-15 seconds - so it does not seem to fully solve the problem.
hstaudacher
commented
9 years ago Akward. So, the fix from this issue is still needed. Will merge it today and test it further.
Hi I'm currently using JAXRSConnector on an embedded target, such as a Raspberry Pi. This works fine and I'm able to host and use my Rest API through JAXRS. One problem I'm facing is that as my project is growing, I register more and more Rest Resources. At the beginning the startup time of the publisher bundle was fast, now with around 20 resources, the publisher bundle takes around 40 seconds to start on my embedded target (with CPU to 100%). After debugging, I saw that this was due to the fact that the ServletContainer.Reload method is called each time a new service is added (or removed) via the ResourceTracker. This Reload method seems quiet CPU intensive, and during start/stop phases, it will be called for each resources although it would only require one call after init. The same problem exists when stopping the publisher bundle (for the same reason).