Closed GoogleCodeExporter closed 8 years ago
Hi Kai,
I think that this was a hard decision. Please let me know if you or someone
else is allready working on atmosphere integration. I can do this immediately.
greetings from Bochum
Oliver
Original comment by oliver.m...@gmail.com
on 21 Sep 2011 at 8:46
Hi Oliver, no, neither me nor anybody else has started with this feature. Feel
free to have a try :-)
The final API might need some discussion and mutual agreement - do you already
have concrete ideas about the how and what to push from the server?
Greetings from holidays in Locarno ;-)
Original comment by kai.openhab
on 21 Sep 2011 at 3:30
Hi Kai, I will check what is possible at the weekend.
My idea is to push everything what's now accessible over the REST API.
So if one connects over the REST channel he gets same result as before.
If one connect via websocket or comet he gets the push notifications.
Original comment by oliver.m...@gmail.com
on 23 Sep 2011 at 10:22
Sounds like a valid approach for the start and should hopefully be quite
straight forward. Looking forward to it!
Original comment by kai.openhab
on 23 Sep 2011 at 7:26
Any news on this issue?
Original comment by mishoboss
on 19 Oct 2011 at 11:01
I'm working on this. I used the last weeks to get an deeper look into
atmosphere framework and to find the best way how to integrate it into openHAB
without changeing the current REST API. You can expect the final release by end
of november.
The new REST API will support push notifications over websocket, HTTP streaming
and cometD.
Original comment by oliver.m...@gmail.com
on 19 Oct 2011 at 1:21
Hi Oliver, could you give us an update on the state of things?
Original comment by kai.openhab
on 27 Nov 2011 at 9:44
Hi, nearly finished ;) I still have to review and edit the maven dependencies.
I will check-in my code in the next days.
Original comment by oliver.m...@gmail.com
on 28 Nov 2011 at 12:53
The next days are over - where can I see the code? :-)
Original comment by kai.openhab
on 6 Dec 2011 at 9:03
Hi Kai,
sorry about the delay too much work and not enough time in the last few days.
If I get home earlier today I will check in today, otherwise tomorrow.
Original comment by oliver.m...@gmail.com
on 9 Dec 2011 at 12:53
Hi Kai,
check my repository olivermazur-websocket-comet.
You can use the REST API as before, but if you add the HTTP-Header
"X-Atmosphere-Transport" the request will be suspended. It will be resumed when
an item has changed. You can subscribe to different topics.
The first one is the all items channel
http://localhost:8080/rest/items
the other one is the single item channel e.g
http://localhost:8080/rest/items/Light_GF_Corridor_Ceiling/
When subscribing to the all items channel you get notified when an arbitary
item has changed.
When subscribing to the single item channel you get notified when the selected
item has changed.
I will add a small test page tomorrow or on monday.
Original comment by oliver.m...@gmail.com
on 10 Dec 2011 at 8:31
Hi Oliver,
Thank you very much! I just had a brief look at it and have some questions, if
you don't mind:
- What would you think of extending the support from only items to also
sitemaps? The current UI uses a kind of push support for sitemaps as it
requires to refresh whenever some item of the current page changes - I guess,
this use case would also be interesting to other UI implementations, e.g. the
planned Sencha tablet UI or also native clients.
- I guess the slf4j-api-1.6.1.jar was committed by mistake in the lib folder -
or do we really need this for something?
- The MANIFEST.MF still says "Bundle-RequiredExecutionEnvironment: J2SE-1.5" -
Afaik, the new Jersey version required Java6, right?
- Although I have added the required header, the response does not seem to get
suspended - in the log, I get a "ERROR o.a.c.BlockingIOCometSupport[:213] -
Unable to resume the suspended connection". Do you have any idea, what might be
wrong here?
Cheers,
Kai
P.S.: Thanks as well for rebasing your change on the latest default commits -
that made it very easy for me to pull your code!
Original comment by kai.openhab
on 11 Dec 2011 at 10:15
Thanks Oliver. I also have some questions as all this push and websockets
technology is new to me, but I'm definitely looking to build a Sencha 2.0 based
openHAB interface.
This is how I see things:
1. The interface looks for a given sitemap at /rest/sitemaps/<sitemap> via
regular REST request, no need for server push here I think
2. The server returns the XML of the items and the client builds the visual
interface
3. Every item has a "link" tag which is used for status updates or sending new
values. And here comes my question about the two approaches I think about:
----- 3.1. Every item makes a new connection to the server via regular REST
polling with given interval or via websocket with server push if it's supported
by the client. This way if there are 100 items, you have to make 100 HTTP
requests every few seconds or you have to maintain a 100 socket connections.
------ 3.2. You use /rest/items and poll only this resource that feeds with
data all items in the UI. This way you have only one HTTP request every few
seconds (or you could do it even every second) or only one socket connection.
However you need to do additional logic to parse and update the items on client
side.
Which of this approaches you see as the better approach and why?
Original comment by mishoboss
on 12 Dec 2011 at 1:37
[deleted comment]
Mihail, very simple, the best approach is to use the sitemap and that's why I
have asked Oliver to extend the server push support to it.
As explained on the REST wiki, you can access parts of sitemaps (I call it
"pages") like this: http://localhost:8080/rest/sitemaps/demo/FF_Bath
This request will return the items and their states of a certain page the user
is looking at. If you add a suspend here, it means that the client only has to
open this single connection and will be notified if (and only if!) there is a
status change for an item of the current page.
Original comment by kai.openhab
on 12 Dec 2011 at 1:44
Ahhh, yes. This simplifies a lot of things. Looking forward to see it.
Original comment by mishoboss
on 12 Dec 2011 at 2:32
No prob. I will extend the sitemap asap. From wednesday I have more time to do
this.
Original comment by oliver.m...@gmail.com
on 12 Dec 2011 at 10:54
Hi all,
As Kai and Thomas know I am working on an iOS native interface for openHAB. I
would be really interested in a brief explanation of the technology involved in
this and a little how-to to do some initial tests with iOS. How do i suscribe
to a notification? How do I work with it? Can I choose which items do I want to
get notifications for?
Thanks in advance,
Pablo Romeu
Original comment by pablo.ro...@gmail.com
on 13 Dec 2011 at 9:52
Hi Pablo,
for what you can request via REST, please see the wiki:
http://code.google.com/p/openhab/wiki/REST
This issue here will only add a "suspend" mechanism, so that the server only
returns the response as soon as some values have changed.
Original comment by kai.openhab
on 13 Dec 2011 at 11:16
Hi Kai,
And if the value continuously changes, it will send the values?
Also, keep in mind that the push notification service for iOS is fully managed
by an Apple server. Then, the app cannot suscribe to a third party websocket
for remote notifications, that's why I am asking for the functional details.
If a remote push notification (not in local network) is needed then we will
need a server capable of registerong to the desired websocket and then, sending
the push notification through the Apple Notification Service
Original comment by pablo.ro...@gmail.com
on 13 Dec 2011 at 11:43
Hi Oliver,
> I will extend the sitemap asap. From wednesday I have more time to do this.
When do you expect to finish that? Please note that I meanwhile did a couple of
changes to the REST API, I hope it does not cause any bigger merge problems for
you. Let me know, if you have any questions regarding my changes or if I can be
of any help.
Cheers,
Kai
Original comment by kai.openhab
on 18 Dec 2011 at 9:47
Hi Kai,
you can expect the next update on thursday. A testapp will be included.
I've stil some issues with websocket support. HTTP streaming and long-polling
will work. Moreover I will switch back to an earlier version of the atmosphere
framework. The current version throws Message Body writer exceptions when
dealing with List. objects.
Original comment by oliver.m...@gmail.com
on 20 Dec 2011 at 10:46
Hi,
adding the sitemap subscription feature to the test app took a littel more time
than expected so I'm not able to merge today, but tomorrow.
Original comment by oliver.m...@gmail.com
on 22 Dec 2011 at 10:10
Ok, so I won't review it tonight anymore and will go to bed instead :-)
Original comment by kai.openhab
on 22 Dec 2011 at 10:14
I just noticed now that you have pushed your changes on Dec 24 - thanks for the
christmas present ;-)
So I assume that it is ready for me to review and integrate? Will try to do so
tonight.
Original comment by kai.openhab
on 30 Dec 2011 at 10:10
There is a bug with the Comet suspended connection. Please see this issue:
http://code.google.com/p/openhab/issues/detail?id=60
Original comment by mishoboss
on 4 Jan 2012 at 8:12
I'm back from my ski trip. I will check the posted issues at the weekend.
Original comment by oliver.m...@gmail.com
on 6 Jan 2012 at 9:45
Give me until tonight, I am just doing a major refactoring of the code myself -
would be nice if you could review this on the weekend then!
Original comment by kai.openhab
on 6 Jan 2012 at 9:49
Ok, just committed my changes:
http://code.google.com/p/openhab/source/detail?r=bfb3c8a1e22a8dfc5fdde4c61340858
5ba0999f1
Oliver, fyi: I mainly changed two things (which resulted in rewriting most of
the code :-():
1. The way state change listeners are used - instead of using the
RESTApplication as a listener on ALL available items, I only register single
requests on the items that are concerned by this request. This is imho
"lighter" - if a user does not use REST/atmosphere at all, he won't have any
additional listeners on all his items, so it is least intrusive.
2. As far as I could see, your implementation always only broadcasted an
itemBean (the one where the status had changed). Instead, the new
implementation always returns the same result as a normal response to the uri -
just that it is suspended and resumed inbetween. The response will a a plain
text item status, a full item info, a page of a sitemap and its format might be
XML, JSON or JSONP, just as on normal requests (that gave me most headache...)
As I have never used the atmosphere framework before, I am not sure if my code
could be improved or simplified - so I welcome any feedback you might have!
P.S.: I still have this "unable to resume" error on normal (unsuspended)
requests - do you have any idea where this comes from and how we could get rid
off it?
Original comment by kai.openhab
on 6 Jan 2012 at 9:48
Hi Kai,
you should remove the resumeOnBroadcast = true attribute from the @Suspend
command.
By removing the resumeOnBroadcast the Atmosphere framework will decide when to
resume the suspended connection. HTTP streaming should work again if you do so.
to 1. full agree :)
to 2. I'm not sure if this is the best solution. If I subscribe to a page and
the server publishes new values I have to check / compare the state of each
item in the subscribing application. Normally a subscribing application is only
intrested in items with changed values. To handle both approches we can add
some (sub)resources to the API.
Like:
http://localhost:8080/rest/sitemaps/demo/FF_Corridor - just return the result
http://localhost:8080/rest/sitemaps/demo/FF_Corridor/SubscribePage -
subscribe and broadcast pages to this channel
http://localhost:8080/rest/sitemaps/demo/FF_Corridor/SubscribeItems -
subscribe and broadcast only items on this page
this should also resolve the "unable to resume" error. The "unable to resume"
error
is thrown when a client connects through a classic request. In this case the
request should be suspended first. Then the TransportListener.OnSuspend()
method should resume the request, but it seems that the connection is not 100%
suspended at that point. There are a few possible solutions for this:
1. separate classic and subscribing requests as described above
2. an own AtmosphereResourceHandler
3. suspend and broadcast immediately to this request
4. try to suspend only subscribing connections
...
Original comment by oliver.m...@gmail.com
on 10 Jan 2012 at 11:08
Hi, Oliver. As you probably know I develop a new openHAB UI and I use the
suspended connections. What I do is this:
On page change, the app first cancel the previous ajax call (if any) and make a
regular ajax request to get items data immediately. Then it makes a suspended
connection of "long-poll" type and waits for new data. On new data or 3 minutes
timeout it makes another suspended connection. This works fine so far.
> to 2. I'm not sure if this is the best solution. If I subscribe to a page and
the server publishes new values I have to check / compare the state of each
item in the subscribing application. Normally a subscribing application is only
intrested in items with changed values.
Yes, you're absolutely right here. In fact what I suggested before some time is
a new channel that supply with new data only changed items and only with data
that changes in a nice and easy to parse format. You can see my suggestion
here: http://code.google.com/p/openhab/issues/detail?id=56
Right now I update all page widgets with the new data if only one has changed
it's value. I don't compare anything, just update them all. It works pretty
good I must say. There is no noticeable delay because of performance issues.
However to make what you suggest we need some unique widget identification, so
the app will know which widget to update with the new data. Such an ID doesn't
exists right now.
And a little bit different topic - I need some info about the websocket
implementation. URL to connect, port, data format (HTTP packet or...)? Please,
feel free to contact me on my mail. Thank you.
Original comment by mishoboss
on 11 Jan 2012 at 6:52
> you should remove the resumeOnBroadcast = true attribute
> HTTP streaming should work again
Hm, I actually do not want streaming to work for these URIs as it imho does not
make sense for them (they are simply supposed to return the same result as on a
normal request).
> I'm not sure if this is the best solution.
It is a first step to introduce server-push by simply not adding any new kind
of resources/data types, but simply adding suspend support to what is already
there.
This already gives UIs the possibility to get notified by the server and they
can receive all required information. As the page data is not too much, the
performance overhead is usually also acceptable (as you can see from
mishoboss's comment).
Still, I understand your point that other subscription types (which then could
also support streaming mode) make sense. But I would like move this
implementation to after the 0.9.0 release (which is scheduled for in 10 days
from now), especially as it might need some further discussions and
specifications (see mishoboss's idea around item IDs etc.)
> The "unable to resume" error is thrown when a client connects through a
classic request.
Right. As a solution, I would probably prefer your solution 4 (try to suspend
only subscribing connections). The problem is that atmosphere automatically
suspends in any case as there is the @Suspend annotation. Do you know if one
can achieve the suspension by code as well? Then we could check for the
X-Atmosphere-Transport parameter and decide on this, whether we want to suspend
the request or not.
Original comment by kai.openhab
on 12 Jan 2012 at 3:49
Hi Kai,
some supplementary notes from my side. As I wrote in comment 22 there are some
issues in the websocket support. In the current setup websocket won't work and
the server returns "501 not implemented". Now I was able to solve this by
adding org.eclipse.jetty.continuation, org.eclipse.jetty.util,
org.eclipse.jetty.websocket to the Imported-Packages (Manifest.mf). With this
jetty will find the libs and is able to handle websockets.
But that's not all. The Websocket connections will run in the same troubles as
the HTTP streaming connections. To solve this you can perform the following
steps:
1. don't resume Websocket requests on broadcast (remove @resumeOnBroadcast)
2. when the connection is not resumed don't unregister the StateChangeListener
in the onBroadcast event.
Moreover I've updated my workspace to atmosphere 0.8.3 without issues yet
(still testing) and I'm looking for a nice "unable to resume" solution.
Original comment by oliver.m...@gmail.com
on 13 Jan 2012 at 4:57
Hi Oliver,
Thanks for this analysis - feel free to do these changes in your clone, I will
then pull the changeset from you!
Would be nice if we could have this stable by mid of next week, so it is in the
right shape for the 0.9.0 release.
Original comment by kai.openhab
on 13 Jan 2012 at 9:48
hi Kai you can find the modifications in my clone. With atmosphere 0.8.3 it
seems that the "unable to resume" error is reduced or gone.
Original comment by oliver.m...@gmail.com
on 17 Jan 2012 at 2:27
Very good news, thanks! Will pull the changes either today or tomorrow.
Original comment by kai.openhab
on 17 Jan 2012 at 2:34
I have just reviewed, tested and pulled your changes into the official repo.
Thanks a lot!
Original comment by kai.openhab
on 17 Jan 2012 at 5:32
Something was broken with http long-polling between 24.04.2012 and today. The
main problem is when several long-polling requests are open for the same
sitemap page the whole mechanism goes very wrong. Unpredictably it can send or
not send responses to selective requests when changes are happening.
Additionally, is there any documentation for server push available?
Original comment by belovic...@gmail.com
on 24 Jun 2012 at 7:36
I'd guess that this is related to the merge of issue 62, which I did yesterday.
@Oliver: Could you please check if there is now a problem with long-polling? If
so, I might better revert the merge.
Original comment by kai.openhab
on 24 Jun 2012 at 2:06
you are right. The mechanism to prevent double broadcasts for group events also
blocks some long-polling clients. If I remove the double broadcast protection
everything works like expected, but it's possible that you will receive several
answers if an group event occured.
Hope I can post a fix tomorrow otherwise I will give you an update.
Original comment by oliver.m...@gmail.com
on 24 Jun 2012 at 6:56
Ok, it seems that the problem is a little bit more complicated. Let us say an
"all off" event is executed and we have 10 active lights. Openhab will switch
off all lights one after another. Each light will raise the group event. If the
connection type is streaming or websocket you receive many events until all
lights are switched off. If we use long-polling the situation is similar but
some messages will be lost during the reconnect, so it is possible that the
last received message still contains some active lights.
In the previous REST API version we have a small sleep interval with the hope
that all group events are processed when the thread is resumed. But in this
case we have also no guarantee.
Maybe someone can give me a hint how this is solved in other bundles.
To solve the current issue I suggest the following:
1. I will prepare a version without the double broadcast protection -> it`s
still possible that some long polling messages will be lost.
2. I will work on a BroadcasterCache. The BroadcasterCache is an Atmosphere
feature but we have to cutomize it for our use case. -> no messages will be
lost but we still have unneccessary broadcasts
3. We have to work on a solution for the group events
Original comment by oliver.m...@gmail.com
on 25 Jun 2012 at 11:06
I am not sure I can follow all details. Why does the work on the streaming
support impact the long-polling? We deliberately chose the solution of a delay
for the long-polling to solve the group event problem (which is not a nice
solution, but the best we were able to come up with).
So first prio should be to get the long-polling in a working state again - so
that the existing functionality works as before. Is there any quick way to
achieve this?
Original comment by kai.openhab
on 2 Jul 2012 at 7:40
Hi Kai,
yes, I will add the long-polling delay again tomorrow.
Original comment by oliver.m...@gmail.com
on 2 Jul 2012 at 7:49
Finished, but I will test all use cases again before I commit the sources.
Original comment by oliver.m...@gmail.com
on 4 Jul 2012 at 8:44
I've comitted a few minutes ago. Long polling respectively non streaming
connections will be delayed (300ms) again. But as some tests shows me this
delay is not a guarantee that you will receive the correct result. The problem
is the same as before: A group event like "All Off" will raise many item
events, because each item in the group has its own listener and all these
listeners will broadcast to the client, but if we use long-polling it is
possible that we lost some messages during reestablishing the connection. To
reduce this I added the 300ms delay again. With this delay openHab has the
chance to process the group event and afterwards openHab will broadcast to the
clients (the API will filter double posts).
This will work in most cases but not in all.
. An example:
- 6 lights = on
- "All Off" triggered
- 300 ms delay
- 6 lights = off
- sent to client "All lights [0]"
-----------------------
- 33 lights = on
- "All Off" triggered
- 300 ms delay
- 28 lights = off
- sent to client "All lights [5]"
-> delay to short
- long-polling disconnect
- 33 lights = off
- long-polling connect
-> message lost "All lights [0]"
As you can see this is not an 100% solution, but it should be the same as
before.
Currently I'm working on a BroadcasterCache implementation for openHAB to solve
this problem.
Original comment by oliver.m...@gmail.com
on 6 Jul 2012 at 11:15
Thanks Oliver, I have pulled your changeset.
@Victor: Could you please verify that HABdroid works again as before?
Cheers, Kai.
Original comment by kai.openhab
on 7 Jul 2012 at 12:52
Ok, will check it a little bit later in the evening today and report.
The only clear solution for long polling, I believe, can be creating some sort
of queues where events are pushed to, and long-polling client polls events from
this queues then...
Original comment by belovic...@gmail.com
on 8 Jul 2012 at 1:48
Now it works as it used to! Thanks!
Original comment by belovic...@gmail.com
on 9 Jul 2012 at 5:42
Original comment by kai.openhab
on 9 Jul 2012 at 5:45
Original issue reported on code.google.com by
kai.openhab
on 17 Sep 2011 at 11:26