Too Slow: Please provide Updates-Via shortcut to secure web socket address

[timbl]

In https://solid.github.io/notifications/websocket-subscription-2021 and https://solid.github.io/notifications/protocol in order to watch (subscribe for notification of changes to) some resource, one has to go though many more round trips than are really necessary to set up the web socket.

Yes it is useful to have a generic notification architecture, and yes it useful to have pod-wide metadata, but going through that all to watch a given resource is too bad for performance.

Why not give the client a short cut to the end of the maze? Imagine that the client had done a subscription, and give the WSS secure address in the headers on the first GET.

[kjetilk]

Performance tests aside, I think I have gradually come to appreciate that there might be request-response pairs (ping-pongs for short) that under some assumptions aren't necessary.

The proposed protocol requires a discovery ping-pong, and then authorization ping-pongs. These might also be towards different servers, so it doesn't necessary help to keep connections open either. Though I have been an advocate for using a generic storage description to mitigate header bloat (which I see as problematic for IoT use cases, and because I think caching could help ensure that you wouldn't need to get the storage description all the time), on balance if we can have less ping-pongs in the overall protocol, then it is certainly worth using a header for it, like the current Updates-Via.

Then, the next assumption I would make is that a client that is authorized to access notifications of a resource, is also authorized to access to the resource itself. That could change at some point, but for now, it seems like a fair assumption that the access controls on the resource and its corresponding notifications are the same. Thus, the ping-pongs to authorize could be made superfluous, when accessing the resource, certainly the server could supply the token(s) required to access the notifications too?

With these two considerations, surely, the current Updates-Via could be extended so that it isn't necessary to access a separate discovery mechanism and also not necessary to check authorization separately?

[elf-pavlik]

We discussed this issue today during the call. @acoburn mentioned scalability issues when using Websockets this way, as well as possible problems with aggressively issuing Capability URLs for all requests.

From my side, I could investigate how this can be done with StreamingHTTPSubscription2021 which doesn't rely on Capability URLs and uses the same AuthN/AuthZ as any other HTTP request. If we want to have a such special case, taking advantage of streaming HTTP responses seems to me the most straightforward approach.

[elf-pavlik]

Thus, the ping-pongs to authorize could be made superfluous, when accessing the resource, certainly the server could supply the token(s) required to access the notifications too.

@kjetilk what advantages do you see in defining a flow with some custom token, compared to using Capability URLs?

[kjetilk]

On torsdag 29. september 2022 23:48:42 CEST elf Pavlik wrote:

@kjetilk what advantages do you see in defining a flow with some custom token, compared to using Capability URLs?

I think Section 4 in that spec describes it pretty well. I'm particularly concerned that in this case, the URI itself is sensitive. Tokens are clearly things that must be protected, URIs shouldn't be secrets in general. I would prefer to use capability URIs very sparingly, and when there are no options, only very short-lived capability URIs.

[elf-pavlik]

I don't think it is clearly stated, but to my understanding, we assume that the Capability URL provided as source in the subscription response expires together with the subscription, I think the expiration should also be provided in the subscription response. In the Updates Via, the expiration time would need to be provided as an additional parameter. I still think this might be more straightforward than introducing custom tokens.

Given all the above, I still think that using streaming HTTP, rather than Websockets, saves us from having to use different AutnN/AuthZ mechanisms. Maybe before trying to find a solution to the problem with Auth over Websockets we could clarify why to take the problematic path in the first place?

[kjetilk]

I understand the desire to go for streaming HTTP, but this is about a minimal change.

With capability URLs, this is an example of an attack that I fear:

Say that Alice sets up a pod with append permissions to a large group of people, which is an expected thing to do. Mallory are amongst the users that gets append access to a resource on Alice's Pod. Now, say that Bob comes along, and Alice has given Bob read permissions on the resource.

Say that Mallory appends a message with "Cute kittens here: https://mallory.example/pics/cute-kitten.jpg" and Bob has a subscription to the resource, gets notified and clicks that link, sees cute kittens and has no reason to suspect that something is wrong. However, Bob's UA may have leaked the capability URL to Mallory since Mallory controls his server, through e.g. the Referer header or some other mechanism as indicated in the Capability URLs WD. Now, Mallory would have escalated his privileges to read notifications, right?

I would prefer to reserve capability URLs for cases where they are single-use.

As for expiration, indeed, a subscription is a commitment into the future, so it makes sense to set expiry and have clients redo auth* now and then.

[elf-pavlik]

@kjetilk I created a dedicated issue to discuss the scenario you brough up #115

[CxRes]

With the impending release of the revised spec, I believe we can re-introduce updates-via like headers. I propose that we allow the resource to advertise two headers, for any one "notification channel" and/or for any one "subscription resource", both chosen from the respective lists advertized in the discovery resource. The header values can simply be formatted as a serialized version of a "notification channel" and "subscription resource" data model.

We can debate whether to make this feature optional or mandatory (i.e. in cases where a notification service exists on a resource). Other than this, I do not see any other impediments in implementing this feature.

[csarven]

If we were to define a header, it needs Structured Field Values of HTTP https://www.rfc-editor.org/rfc/rfc8941.html

Link relation, although verbose may be simpler:

HEAD https://example.org/guinan/profile

Link: <wss://websocket.example/guinan/profile?auth=foobar>; rel="http://www.w3.org/ns/solid/notifications#receiveFrom"
Link: <http://www.w3.org/ns/solid/notifications#WebSocketChannel2023>; anchor="wss://websocket.example/guinan/profile?auth=foobar"; rel="type"

[CxRes]

Link relation, although verbose may be simpler This is confusing for the client, especially if it could potentially be mixed with other link relations.

Why not just?

HEAD https://example.org/guinan/profile

Update-Channel: id="https://channel.example/guinan/profile"; type="WebSocketChannel2023"; receiveFrom= "wss://websocket.example/guinan/profile; rate="PT1M"

Mirroring Discovery Resource:

"channel": [{
    "id": "https://channel.example/guinan/profile",
    "type": "WebSocketChannel2023",
    "topic": "https://example.org/guinan/profile",
    "receiveFrom": "wss://websocket.example/guinan/profile",
    "rate": "PT1M"
  }]

and similarly for Subscriptions in another header (with potentially two sub-cases: Subscription specific to topic and Subscription specific to pod).

[csarven]

Because that'd be a hack. There is a distinction between HTTP representation metadata and data, and serialising and stuffing the latter into former doesn't make sense in a HEAD. If it boils down to using a single request, GET makes more sense where that information can be available from the representation data. That's technically possible for resources that can have a representation using a concrete RDF syntax, but not for others. That's In addition to some challenges in handling server and client-managed data in the same resource. Even if we were to define a SFV and send it off to IANA, it would be based on properties that we currently use with potential concerns on extending or modifying the field-value expectations that match the models.

The existing Link relation approach is just barely good enough to help the recipient navigate to where they need to. But, to be absolutely clear, in practise it is going to be generally only useful/feasible in cases where ResourceServer and SubscriptionServer [1] - are available from the same-origin or at the very least the interface is controlled by the same software. Besides arbitrary syntactical differences between Updates-Via and the Link relation approach, there is not a whole lot different - they both expose sensitive information in the header.

If we were to go in that direction in the spec, irrespective to the header and field-value that's used, 1) we'll need to highlight the scenario in which it is useful with advisory statements, 2) decide on the requirement level - because that's what it is going to come down to. To demonstrate interop, it needs to be required, all meanwhile knowing that some implementations will not go for it or inapplicable for some deployments.

[1] There is indeed a trade-off here at least with respect to speed gains with (a simpler?) conforming to both classes of products (ResourceServer, SubscriptionServer) vs. decentralised products with potentially different ownership/implementations/policies and so forth.

---

What are the differences concerning exposure of sensitive information like the channel URL in the response of a HEAD header value vs. in the response of a POST body, if the original request is authenticated/authorized? Is there a CWE or an RFC security consideration making that distinction? Or is this all essentially a matter of being able to split ResourceServer and SubscriptionServer as separate products implementable by different software? Also noting that HEAD, GET and POST are cacheable.

[acoburn]

Providing a capability URL in the response to every GET request will present a serious security problem.

We have over ten years of experience working with OAuth2, which has a similar pattern, where a security token is included in a response header (i.e. the code in the Authorization Code flow). Since the publication of OAuth2 in 2012, nearly every enhancement to OAuth2 has addressed the security vulnerabilities of that code value. We now have PKCE, DPoP, etc, to help secure that value along with the deprecation of insecure flows such as implicit and hybrid that are unable to provide effective protection to that code value.

If we provide this sort of capability URL in every response header, whether via Link Headers or Updates-Via, we will be ignoring all of the security advice from the last 10+ years from OAuth2. This approach simply does not hold water from a security perspective, and it is a non-starter for any server that aims to handle sensitive user information.

[csarven]

Putting aside the fact that OAuth2 is currently implicitly/indirectly encouraged for use - which may change in the future, and in fact the Notifications Protocol is not tied to a specific authn/z mechanism - and that we definitely want to take advantage of existing advisory, how is a POST body deemed to be secure in contrast to a header in GET/HEAD? Or at least, can we detail the scenarios in which one is less secure than the other? AFAIK, in both cases the channel URL that's deemed to be sensitive (besides the public channel case) is made available when the requester is authorized.

[acoburn]

A POST body is safer because it is the response of a particular, dedicated interaction with a server at a particular endpoint, rather than just being present in every response, for every client. One can place more constraints around that endpoint.

[csarven]

Focusing specifically on the observability of the channel URL for authorized requests - certainly not every response for every client - I don't see glaring differences between representation metadata and data. I acknowledge that the interaction flow to obtain the channel URL may be different between the requests as well as separate or additional constraints that can be placed on topic resource and subscription resource, but that doesn't seem pertinent.

[acoburn]

My concern is related to response headers on Solid Resources of the type described in https://github.com/solid/notifications/issues/110#issuecomment-1402067536, where a capability URL is returned.

HEAD https://example.org/guinan/profile

Link: <wss://websocket.example/guinan/profile?auth=foobar>; rel="http://www.w3.org/ns/solid/notifications#receiveFrom"

If a (Solid) Resource Server provides that on every authorized response, then every authorized client that interacts with Solid will have access to that capability URL. I have no concerns about providing a link to an endpoint where an Agent can negotiate for this capability URL, but providing this on every authorized response is the issue. This really will not stand up to security scrutiny.

[csarven]

Is there a particular reason why that URL needs/expected to be the same for every authorized client? My understanding is that it need not be the same and something like Vary: Authorization can be used to make the distinction per request/representation.

[kjetilk]

The Vary header specifically takes a header name as value, so it would have to be something else, but yeah, I guess Vary: Updates-Via could be the thing.

However, there is a performance cost to every Vary value too, another thing that reduces the number of cache hits. The system performance can therefore be quite application specific. I have been very uncomfortable about Capability URLs in general and beyond single-use in particular. I have not been so concerned about having a token that is not part of the URL since the assumptions to protect URLs tend to be different from other fields, but if that has also been shown to cause problems given the experience with OAuth2, I think we would be ill-advised to ignore that experience.

That makes me think, would it be possible to detail ways that the ping-pongs can be short-cutted by caching in common situations? Do we have optimization potential there?

[acoburn]

Caching is yet another dimension of this that complicates the presence of capability URLs in response headers. If a client caches responses (it should), then capability URLs in these headers makes it very difficult to effectively use these caches, since the capability URL may have expired. POST responses are not cached.

[csarven]

Authorization header is a thing (RFC 7235) but if I'm reading the specs correctly, response with Vary: Authorization won't be cached - assuming that Authorization was in the request.

POST responses can be cached (RFC 7231) but to be clear, the request semantics of POST targeting subscription resource won't be cacheable - 200/Content-Location is not required or a meaningful response (Notifications Protocol).

Irrespective to where the capability URL is observable from, when it expires, client will have to go through the requests (GET/HEAD or POST) again to obtain a new one.

I just want to be clear that the exposure of the capability URL does not appear to be fundamentally different for the two approaches (GET/HEAD vs POST) for authorized request.

I won't pretend to know the inner depths of https://www.rfc-editor.org/rfc/rfc7234#section-3.2 but it seems that exposure of the capability URL is removed or limited when caching is forbidden or not expected.

Enabling clients to cache the topic resource is certainly useful and punting responses that can vary to POST subscription resource response seems good too, but I don't quite yet see if/how/when Vary: Authorization factors into all/some responses.

[CxRes]

I feel as though a straw man is being built out of the example I have created and there is much confusion. We need to differentiate between flows for public channels and private+custom channels. That example above is valid only for public channels (as would have been advertised in a Description Resource). In that case, a WebSocket channel will not have a cURL (it is a permanent URL) and is independent of the client! The client is free to cache it.

For the case of private+custom channel, the best we can do is to advertise a subscription service via a header (additionally to a Description Resource). The fact remains that having a cURL for WebSockets in the Topic Resource header will result in a dedicated channel per Topic Resource. Apart from the caching and security issues @acoburn cites, this will, in all probability, also be poor for performance, the moment a client wishes to subscribe to multiple topics. With the subscriptions flow (as we have conceived in the protocol), clients can subscribe to multiple topics in one shot and watch for the changes on all the topics in one channel, making client's life simpler. In the former, clients have to connect to multiple channels and manage all those channels. Not only are there more round trips to first connection in the aggregate, imagine the overhead for the client if the connection drops.

Specific responses:

Because that'd be a hack. ... concrete RDF syntax, but not for others.

I do not follow this completely. Maybe you can explain this part to me in person.

in practise it is going to be generally only useful/feasible in cases where ResourceServer and SubscriptionServer [1] - are available from the same-origin or at the very least the interface is controlled by the same software.

Are you talking about having cURLs in the header here? Otherwise, the situation with the header is equivalent to that of a Description Resource. The Resource Server has to know what to advertise, whether on the header or in the Description Resource.

1) we'll need to highlight the scenario in which it is useful with advisory statements, 2) decide on the requirement level - because that's what it is going to come down to.

\1) It is useful when client wants a shortcut for the servers' preferred channel and/or subscription. The client is free to look at description resource for more options. Servers can optimize their choice based on demand. 2) It definitely should be optional, as the description resource mechanism is more comprehensive. Furthermore, it can be tacked on without touching the present protocol.

[csarven]

I am fairly certain that one of the key challenges all along has been about sharing protected WebSocket endpoints (or notification channels' receiveFrom as per the Notifications Protocol terminology). Put differently, the work wasn't about finding a way to share public endpoints because we already had/have a solution for that.

One of my responsibilities is to facilitate consensus. So, I'm simply asking for clarifications, references, and so forth for assumptions or preferred designs from everyone, even if it is obvious to them. And certainly seeking consent where necessary. Please bear with me.

Again, I want to first look at security (information exposure), then caching, then both of them together, and perhaps some other dimension if we need to.

---

If there are any disagreements on the following or clarification is needed, please quote and ask.

• Solid Notifications Protocol does not require specific authn/z mechanisms but encourages reusing existing ones, e.g., the Solid Protocol. In practise, we know it'll work alongside the Solid Protocol, but most importantly, Solid Notifications Protocol is not intended to be designed for specific authn/z.
• Unauthorized requests get a 403, and authorized get a 200.

Example resources from the specs:

• Topic resource: https://example.org/guinan/profile
• Subscription resource: https://websocket.example/subscription

The following examples (with simplified code) focus on authorized requests (requests include the Authorization header) since unauthorized GET and POST requests to topic, description, and subscription resources will get a 403 with no expectation to reveal any information beyond stating that the request is forbidden.

A.

HEAD https://example.org/guinan/profile

204
Link: <A>; rel="http://www.w3.org/ns/solid/notifications#receiveFrom"

B.

POST https://websocket.example/subscription

200
{
  "receiveFrom": <B>
}

Focus on information exposure:

1. Can A and B be the same URL? If not, why not?
2. Is either one of those HTTP responses less secure than the other? If so, how?
3. Must A be the same URL in every response? Why? Same question for B.

Focus on caching:

4. When A and B are different for each client must responses include Vary: Authorization?
5. When A and B are different for each client and there is no caching, would the response to question 2. be different?

---

As mentioned earlier, there are design choices here but I want to make sure that we are looking at the claims both in isolation and in context of everything else. If approach A fundamentally raises (more) concerns about caching (because targeting topic resource) than approach B, then we can evaluate it based on that.

If the interaction flow that leads to approach B is claimed to be "too slow", then caching concerns for approach A needs to be acknowledged. And ultimately, whether using approach A and giving up on caching topic resource is still preferable over approach B (or alongside B). All things being equal and caching aside, I don't find approach A less secure than B, but perhaps your answers to the questions above (and others) can help me better understand the assumptions or intended designs.

---

I would add one additional thing about approach B in that topic resource and subscription resource can have different access controls, so, while one may be able to read the topic resource, they may not be able to subscribe for updates. That is potentially possible in approach A but it entails addressing something uncharted - Authorization rule distinguishing access modes between the topic resource and notification endpoints. I don't want to delve too far into this use case, whether there are actual implementations to point to, or even discussing how separate access controls on topic resource and notification endpoint may relate or one another.

[CxRes]

Again, I want to first look at security (information exposure), then caching, then both of them together, and perhaps some other dimension if we need to.

I would add that you need to consider the stated concerns in the context of multiple Topic Resources per channel, multiple features per channel and modifiable channels.

I want to emphasize that multi topic support is not an icing on the cake, but something that must be baked into the cake (it adds an extra dimension of constraint to the design). With linked data (think in terms of type indexes or shape trees), there will rarely be scenarios where a client wants to connect to a single resource. A. (\<A> as wss:// cURL) for secure WebSockets clearly fails this test: Ask will using mechanism A. be more efficient when the client has to receive notifications on multiple topics simultaneously (Isn't efficiency why we are having this debate in the first place)? A is guaranteed to scale more poorly than B (Assuming \<A> is a cURL for the Topic). So thinking about its other properties (even assuming it passes all the other tests you pose, which imho it does not) should be moot.

This is not to diminish the primacy of security and caching concerns in general. The reason we humans choose a perspective and rank their importance when dealing with complexity is because of our limited cognition. But ultimately we want to satisfy not just one property, but all properties that we can.

[timbl]

@elf-pavlik noted that StreamingHTTPSubscription2021 would allow the same auth to be used as for normal GET requests. Though switching from web sockets to streaming HTTP is quite a change, if there is no way of just passing the same auth with the web socket request itself, and if using a capability URI is therefore needed with web sockets (whether is is from a header or from a POST), and we really don't like capability URIs, then is the best design to have a [link] header of some sort pointing to the streaming endpoint?

[timbl]

(Why can't Web socket requests carry the normal auth tokens of a get request? Browser Limitation?)

[elf-pavlik]

A POST body is safer because it is the response of a particular, dedicated interaction with a server at a particular endpoint, rather than just being present in every response, for every client. One can place more constraints around that endpoint.

:+1:

Possibly comparable explicit intent could be achieved with https://www.w3.org/TR/dx-prof-conneg/.

In the end it is up to the RS to chose which (if any) pre-established notification channels to advertise and which subscription resources/services.

---

StreamingHTTP has the advantage of using regular HTTP request with Authorization header. Which doesn't require use of Capability URLs. If we talk about speed, I expect it would be the fastest of all options, especially if the Resource Server and Notifications Sender are on the same origin.

---

I would strongly suggest that we continue working in this issue without blocking release of v0.2 of Notification Protocol. Implementations can already start updating to it and we could aim at resolving this issue for v0.3

[csarven]

The Solid Notifications Protocol includes the following based on the temporary identifier considerations from the W3C Self-Review Questionnaire: Security and Privacy:

What temporary identifiers do the features in this specification create or expose to the web? The subscription response payload can contain a capability URL to protect the notification channel which is only exposed to authorized Subscription Clients.

For GET/HEAD response header including capability URL, here is one possible response to the same consideration:

The topic resource's HTTP response headers can contain a capability URL to protect the notification channel which is only exposed to authorized Subscription Clients.

I suggest that we also acknowledge and exercise a response to the following considerations, whether in terms of requirements or advisory for all approaches:

6.

If a standard exposes a temporary identifier to the web, the identifier should be short lived and should rotate on some regular duration to mitigate the risk of this identifier being used to track a user over time.

7.

When a user clears state in their user agent, these temporary identifiers should be cleared to prevent re-correlation of state using a temporary identifier.

8.

If features in this spec create or expose temporary identifiers to the web, how are they exposed, when, to what entities, and, how frequently are those temporary identifiers rotated?

[CxRes]

StreamingHTTP has the advantage of using regular HTTP request with Authorization header. Which doesn't require use of Capability URLs. If we talk about speed, I expect it would be the fastest of all options, especially if the Resource Server and Notifications Sender are on the same origin.

This is one place I strongly disagree with you @elf-pavlik. Well, actually you are most probably correct in assuming that StreamingHTTP will likely be the fastest, but only for initializing notifications for one topic resource at a time. However, the moment you need to receive notifications on, say, 50 resources (which is the likely scenario with linked data), WebSockets will trounce StreamingHTTP in performance. We don't need benchmarks to demonstrate this! A WebSocket will require one roundtrip to mint the cURL and then receive notifications on all the topic resources; Whereas StreamingHTTP will have to make a subscription request per resource, i.e. 50 roundtrips! Similar consideration applies for modification of channel and resubscription.

[elf-pavlik]

Please consider trying my early branch of streaming HTTP notifications for CSS

• https://github.com/CommunitySolidServer/CommunitySolidServer/pull/1878

Easy way to test it:

• GET some public read resource (eg. WebID doc)
• pick rel="http://www.w3.org/ns/solid/terms#updatesViaStreamingHTTP2023" from link header
• curl that linked endpoint
• change the document using a client of your choice

It also does authorization on that notification endpoint but you need to deal with the tokens to try it