Server Sent Events support?

[tuxedo0801]

Hi there,

want to use SSE (http://stackoverflow.com/questions/11077857/what-are-long-polling-websockets-server-sent-events-sse-and-comet?rq=1) with NanoHTTPD.

Any hints how to do this? It's not yet implemented, or?

[tuxedo0801]

Had a deeper look into the source.

--> No, it's not yet implemented.

I tried to add this feature. But there are things that block me:

Idea was to extend Response class, so that one can use this class to send SSE-style events before returning the serve() method (which might close the connection).

But there is no interface for Response class. So I need to overwrite the complete class with my own implementation --> lot of code duplication.

I would add an constructor to feed in the HTTPSession to be able to access the socket via its streams. Next issue: The HTTPSession does know the input and outputstream, but only the inputstream has public access. Outputstream is private. So it requires me to also overwrite the HTTPSession. For doing this, I also need to overwrite the start() method. start() is public, but it uses myServerSocket, which is again private...

argh

I could easily copy&paste the complete NanoHTTP class and modify it to my needs. But what about updates? I would need to merge them anytime there is something new (bugfixes, ...).

Isn't the idea behind this tiny webserver that it's easily extendable? So why are so many things "private". Make them "protected" so that anyone can overwrite it ???!!! Any why is there an interface IHTTPSession, if it's so hard to use a custom implementation for the session?

[ritchieGitHub]

@LordFokas could you check this? Or was it included in your pullrequest?

[LordFokas]

Well I haven't read much into the specs yet, but I believe this could probably be easily implemented by someone who really knows what they're doing more or less in the same fashion NanoWSD sits on top of NanoHTTPD. I guess that'd mean a new module (and that'd better be called NanoSSED :p ) and a lot more unit tests.

I didn't really do much in regards to this, actually I doubt I did anything that changes this thread, but the point still stands: NanoHTTPD is hard to extend because of all the private stuff and inner classes. To really fix this you'd have to make all inner classes static, add a few more interfaces for already existing behavior and make most private stuff either protected or public. This is more or less what I did when I refactored NanoWSD, but NanoHTTPD still needs it and badly.

To sum this up:

• Extending NanoHTTPD to create HTTP-based protocols is hard due to how tightly everything is packed into a single file. It needs fixing, and badly.
• SSE Support is still an issue. Either deny it to @tuxedo0801 or reopen the issue until someone (probably himself) implements and PRs it. But I doubt it'll happen anytime before the NanoHTTPD file is cleaned up.

[tuxedo0801]

I could easily copy&paste the complete NanoHTTP class and modify it to my needs.

That's what I did in the meantime: https://github.com/tuxedo0801/KnxAutomationDaemon/blob/master/plugin-projects/cvbackend/src/main/java/de/root1/kad/cvbackend/NanoHttpdSSE.java

I habd to change a lot to made it possible.

Well I haven't read much into the specs yet, but I believe this could probably be easily implemented by someone who really knows what they're doing

It's not very complicated. In a short: You just have to keep the connection open and send data if there is something to send. The connection keeps open until client closes or connections breaks somehow.

but the point still stands: NanoHTTPD is hard to extend because of all the private stuff and inner classes.

I totally agree. A project that i sintended to be extended by others should contain a lot of private stuff. It should be easy to extend.

reopen the issue until someone (probably himself) implements and PRs it.

I would have created a PR, but the changes are not so easy and would maybe conflict with other stuff, so that I decided to copy&paste it and create "my own" variant. Maybe someone with a deeper knowledge of the structure and philosophie of nanohttpd could adapt my changes to the base?!

But I doubt it'll happen anytime before the NanoHTTPD file is cleaned up.

That's true...

br, Alex

[LordFokas]

I wonder if it wouldn't be easier to create some kind of server-push-to-client superclass for NanoWSD and NanoSSED. It would be harder to pack everything nicely, but from an OOP point of view it makes sense...

[LordFokas]

I don't have the knowledge it takes to build SSE all of a sudden, and I doubt I can put together the time and effort to accomplish such a big task any time soon, but once I do refactor the server classes I'll try to at least add some kind of server-push-to-client abstraction layer.

[ritchieGitHub]

@tuxedo0801 it seems you have the knowledge for it, could you create a pull request with the necessary changes? If the changes are not so bad we will include them in 2.3.0 otherwise they will go into 3.0.0 after a main refactoring.

[LordFokas]

If they came in 2.3.0 that'd be great because then it'd be easier to create a general abstraction layer and I wouldn't have to pull anything out of thin air to accomplish it :)

[LordFokas]

@tuxedo0801 v3.0.0 has been started and the files have been split. Most stuff is now protected or public. Would you like to attempt to implement this again? I'll be open to do any necessary structural modifications to accomodate SSED.

[nibmz7]

I was unsure of how to integrate it into the Response class code like the newFixedLengthResponse/newChunkedResponse method because of the usage of Object.wait. Only one class of which I decided to call SseSocket is needed.

SseSocket.kt

class SseSocket(source: Any) {

    private val support = PropertyChangeSupport(source)

    fun createSseResponse(): Response {
        val sseResponse = SseResponse()
        support.addPropertyChangeListener(sseResponse)
        return sseResponse
    }

    @WorkerThread
    fun fireEvent(message: String) {
        support.firePropertyChange("update", null, message)
    }

    private inner class SseResponse(): Response(Status.OK, null, null, 0), PropertyChangeListener {

        val pauseLock = Any() as Object
        var message: String? = ""

        override fun propertyChange(evt: PropertyChangeEvent) {
            synchronized(pauseLock) {
                this.message = evt.newValue.toString()
                pauseLock.notify()
            }
        }

        override fun send(out: OutputStream) {
            val gmtFrmt = SimpleDateFormat("E, d MMM yyyy HH:mm:ss 'GMT'", Locale.US)
            gmtFrmt.timeZone = TimeZone.getTimeZone("GMT")

            val pw = PrintWriter(
                BufferedWriter(
                    OutputStreamWriter(
                        out,
                        ContentType(mimeType).encoding
                    )
                ), false
            )

            pw.append("HTTP/1.1 200 OK\r\n")
            pw.append("Access-Control-Allow-Origin: *\r\n")
            pw.append("Content-Type: text/event-stream\r\n")
            pw.append("Date: ${gmtFrmt.format(Date())}\r\n")
            pw.append("Cache-Control: no-cache\r\n")
            pw.append("Connection: keep-alive\r\n")
            pw.append("Keep-Alive: timeout=5, max=1\r\n")
            pw.append("Transfer-Encoding: chunked0\r\n\r\n")
            pw.flush()

            while (true) {
                try {
                    synchronized(pauseLock) {
                        pauseLock.wait()
                        val data = "data: $message\n\n"
                        val chunkedOutputStream = ChunkedOutputStream(out)
                        chunkedOutputStream.write(data.toByteArray())
                        chunkedOutputStream.finish()
                    }
                } catch (e: Exception) {
                    support.removePropertyChangeListener(this)
                    break
                }
            }

            pw.close()
            out.close()
        }
    }
}

Usage

...
val someEventSocket = SseSocket(this)
val LOCK = Any()

override fun serve(session: IHTTPSession): Response {
 if (uri == "some_event") {
        synchronized(LOCK) {
             return someEventSocket.createSseResponse(myHostName)
         }
     }
}        

fun sendMessage(jsonMessage: String) {
       synchronized(LOCK) {
           someEventSocket.fireEvent(jsonMessage)
      }
 }

[sudarh]

I have just now started using routerNanohttpd and am very interested to know if there is any extensions or github contributions that provide SSE support as well.

[sudarh]

@tuxedo0801 @LordFokas can you please help clarify if my understanding is correct. i am new to this and any help is appreciated. What I figured based on the following tenets of SSE

1. You can only accept EventSource requests if the HTTP request says it can accept the event-stream MIME type; ==> Need to have a new SSERouter extending NanoHttpd that overrides serve() to process this

2. You need to maintain a list of all the connected users in order to emit new events

3. You should listen for dropped connections and remove them from the list of connected users; => AsyncRunner already has a list of all client requests, need to modify or extend so we can have SSE handling and housekeeping.

4. Respond with one or more valid server sent event messages, using the correct message format. Tell the client that the Content-Type being sent is “text/event-stream” indicating that the content will be valid server sent event messages => clientHandler run() should be overridden to not close the streams after the serve() response is sent back to client.

5. Tell the client to keep the connection alive, and not cache it so that events can be sent over the same connection over time, safely reaching the client. => this i am not sure how to do have to dig further.

We need to have a EventSource Handler that takes care of all the returning a response and also not closing the streams. Also, need to implement a protocol that handles reconnection events and maps it to the right eventsource stream.

[sudarh]

actually thinking more about this, the bruteforce version is the following:

1. in process(), check for the session header and if the "Accept" header is text/event-stream, then interpret that as EventSource request. Override the serve() to do the following:

a. wait on a event that needs to be sent back ( we can have some heartbeat events if required) based on a trigger from server ( can be a callback we can register on the serve()).

b. on notify() write back chunked data to the stream using the following format: set header content type : text/event-stream set cache-control header : no-cache ( this satisfies 5 above) set id based on the server keeping track of last data set sent

c. go back to waiting for another event from server using wait().

this way, we don't have to modify asyncrunner or clientHandler. But need to understand how to cleanly implement this in the best possible OOP way.

[nibmz7]

Does extending the Response class like what I did serve your purpose? It runs in a while loop with an object lock to ensure that the function returns based on a condition or if it catches an error. Hence ensuring the connection doesn't close (remains open).

For front end, there's EventSource web API already in place. So there's no need to do anything client side; only on browsers that is.

I'm new to this as well in fact new to everything so do pardon me for my explanation may seem a bit vague.

[sudarh]

@nurilyas7 Thanks. I am using RouterNanoHttpD and will have to extend this to account for differentiating SSE from other uris using the "Accept" header to check. But the serve() will eventually override and do something similar to what you have done. It will use wait() notify() on a lock for an underlying service to invoke callbacks with the event to be sent back to client

[thadikari]

That's what I did in the meantime: https://github.com/tuxedo0801/KnxAutomationDaemon/blob/master/plugin-projects/cvbackend/src/main/java/de/root1/kad/cvbackend/NanoHttpdSSE.java

Hi @tuxedo0801, the link to NanoHttpdSSE.java in your comment is broken now. Could you please update the link?