sdk apps and websockets

[tlodge]

Currently any sdk-built databox apps that display data will use websockets for streaming data to a client browser. I'm reliably informed that this will fail with the new core network, so need to figure out what needs to be put in place to make this work.

[Toshbrown]

The problem here is that the SDK UIs create a direct WebSocket connection between the browser and the app container inside databox to stream UI updates.

The simplest solution would be to handle this in the same way as we currently handle HTTP requests from app UIs, and proxy them through the core-container-manager.

But this begs the question, should the core-container-manager be a point of control for the flow of data onto and off of the databox? and how do we effectively secure these endpoints moving forward? Currently, anyone can access the data displayed in an app. This may improve once we integrate accounts and users.

Should we consider moving this functionality to other components, such as the export-service or core-network?

This could be a blocker for the 0.3.0 release, as we do not want to stop the SDK from working once the core-network is integrated.

cc/ @mor1 @cgreenhalgh @sevenEng

[sevenEng]

Maybe we could make SDK as privileged as CM? Concurrently, CM could access data from every app because core-network treat packets from CM as privileged and will always tunnel them through.

I'd like to understand the requirements more to see how to make things happen in a right way. Basically how will SDK address the apps when trying to open a websocket, using service names or get their IPs somehow? Before core-network, how does the communication get through, as I remembered, all apps will be put on a network databox-app-net? It'd be great if I could be pointed to codes about this, meanwhile i'll run with SDK and do some digging.

@tlodge @Toshbrown

[haddadi]

I believe we don't want the "outgoing" flow monitoring to be done by core-CM. This is also so that in case of disasters, export-service can be shutdown while CM continues it's operation. I recommend we isolate these two

== Hamed https://haddadi.github.io http://www.eecs.qmul.ac.uk/~hamed/

On 30 November 2017 at 14:25, Qi LI notifications@github.com wrote:

Maybe we could make SDK as privileged as CM? Concurrently, CM could access data from every app because core-network treat packets from CM as privileged and will always tunnel them through.

I'd like to understand the requirements more to see how to make things happen in a right way. Basically how will SDK address the apps when trying to open a websocket, using service names or get their IPs somehow? Before core-network, how does the communication get through, as I remembered, all apps will be put on a network databox-app-net? It'd be great if I could be pointed to codes about this, meanwhile i'll run with SDK and do some digging.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/me-box/databox/issues/206#issuecomment-348202544, or mute the thread https://github.com/notifications/unsubscribe-auth/ACy0rwYVgYkB97s0Wp3dpTRHLH2g-A96ks5s7rrhgaJpZM4QwhQv .

[haddadi]

Maybe a point for discussion on the next call?

[mor1]

I'm deeply confused I think :/

1. How and why is HTTP traffic being proxied through core-container-manager? That doesn't seem right... I think if we move to "export-service as a driver" then remote browsers will be able to connect via that, with traffic flowing through core-network according as SLA permits, won't we? (More and more convinced that introducing core-export-service was a mistake; sorry...)
2. It's not the SDK that needs privileges is it? Rather, the SDK-built containers need to be able to expose ports so that websocket connections can be created to local and remote browsers. (Which seems like a job for something to proxy external connections into core-network on the box, for core-network to take a decision as to which (virtual) network device data should be forwarded onto.

[Toshbrown]

@mor1

1a. All databox apps (and most drivers) offer a UI over HTTP on port 8080. Request to theses UIs are proxied through the core-container-manger. It was built this way because it was assumed that things outside the databox would not be able to directly contact apps/drivers. The core-container-manger serves the main UI so it seemed sensible to proxy the requests to App and driver UIs through it. As it turns out App containers could be accessed from the outside world and the SDK took advantage of this. Adding in the core-network has finally plugged this hole but it now breaks app built with the SDK.

1b. I'm not sure how the export-service helps in the case of delivering the UI, but sdk apps could use the export-service to publish the data to the UIs in browsers maybe. However, they do not know the IP of the UI when the app is installed so how they request permission to do so would be an open question.

2. The SDK needs no privileges, but the apps it produces need a way to display a UI, and that UI needs WebSockets to support fast updates based on incoming data. So yes, it feels to me like the core-network could play a role here. How the core-network could verify that the browser requesting the connection is permitted to access that particular app container and the data that might be exposed is an open question. As is whether it should also handle the HTTP requests as well, so no proxying is needed in the core-container-manager.

[cgreenhalgh]

My inclination would be

a) in the short term, to proxy websockets through the same proxy in the CM that handles app http interactions. I'm expecting all of this traffic - not just the databox UI per se - to run over HTTPS with (at least basic) authentication.

b) to migrate this into a separate http front end/proxy component which terminates all http/s requests to the databox, with the CM requests from the app/UI also passing through it

c) to work out exactly what we can and cannot enforce this way in terms of access control and information leakage for various types of client (databox app, standards-compliant browser, other) before deciding on a completely new way of handling (i) app/driver UIs and (ii) other off-the-box interactions.

relatedly d) to revisit the architectural realisation of the SDK views, e.g. are they best treated as app UIs? or is the view logically a separate authenticated viewer (not a databox app -running directly on the display device) that interacts with a (doesn't yet exist) home views driver?

(personally i think the export service could still have a place for high risk, high-value one-off transactions, although we haven't actually got many apps that work in that mode.)