whatwg / html

HTML Standard
https://html.spec.whatwg.org/multipage/
Other
8.12k stars 2.67k forks source link

Setting headers for EventSource #2177

Open chicoxyzzy opened 7 years ago

chicoxyzzy commented 7 years ago

Seems like there is no way to add Authorization header or any other headers for EventSource. Is there any reason it shouldn't be possible?

annevk commented 7 years ago

The main reason to not allow this is that this feature can somewhat easily be written on top of fetch(). I'm not sure we need put more effort into the "legacy" XMLHttpRequest, WebSocket, and EventSource networking APIs.

chicoxyzzy commented 7 years ago

Hmm... I have not seen any SSE alternatives based on Fetch yet. Do you know any implementations?

annevk commented 7 years ago

No, sorry. It's also still a little early since only Chrome supports https://streams.spec.whatwg.org/ for now. But you can build any kind of abstraction on top of that. You would also no longer be limited by the SSE format and such.

chicoxyzzy commented 7 years ago

How should we cancel a Fetch stream? Cancelable Promises proposal is withdrawn.

domenic commented 7 years ago

stream.cancel()

annevk commented 7 years ago

Closing this since I don't think we'll be able to find sufficient interest to extend this API.

danielwegener commented 7 years ago

@annevk What would indicate a sufficient interest?

SSE is IMO a really nice standard for implementing simple web-based queue-consumers with out of the box at least once consumption semantic and I think it has very real use-cases. But as stated above it obviously lacks the following capabilities:

There are also quite a lot active server-side implementations of SSE out there.

It's use-cases really come to shine with event-based/ reactive backends. Something that is, compared to REST, not necessarily the thing yet, but (given all the reactive/async/streaming movement going on in the backend world) seems to be relevant enough (probably needs references).

It would of course be possible to build something similar with fetch - but it would not follow any standard and assumptions like "hey this client library and that server library supports SSE - so we can probably wire them together" will no longer hold. And beside this spec there is no such thing as standard GET for event streams.

So, I'd second to keep the EventSource/SSE standard alive and move it forward to support custom headers.

annevk commented 7 years ago

I'm not sure, but over the past decade I've heard this request from three people or so, which is not really enough. Also, you can continue to use server-sent events and the server-side implementations thereof, you would just need to implement the client-side bits using fetch() if you want to transmit more headers.

LarsKemmann commented 7 years ago

Make it four. :) We recently chose SSE over Web Sockets for a reactive app because it's much simpler to program against and the built-in reliability is nice. Having to work around the lack of custom headers support by putting the authorization token in the query string is probably the one thing really missing from the standard. Why should we pause use of a perfectly good technology and wait until replacements come out?

kmayer commented 7 years ago

I have a working implementation, forked from Yaffle's polyfill.

https://github.com/Yaffle/EventSource/compare/master...kmayer:kmayer-2

annevk commented 7 years ago

Given the (renewed) interest in this I'll reopen. We'll still need implementer interest.

mkurz commented 7 years ago

Great!

tyoshino commented 7 years ago

Could someone with edit permission correct EvenSource -> EventSource in the title?

I think this is easy to implement on Chromium and wouldn't introduce so much complexity. Wanna hear other vendors' interest.

ricea commented 7 years ago

I work on the team that maintains EventSource for Chrome. EventSource is not being actively developed. We feel resources are better spent in filling the gaps in the more generally-useful facilities provided by fetch().

To give some idea of scale, fetch() body streaming is already used on ten times as many pages as EventSource in Chrome.

I acknowledge that the resumption facilities provided by EventSource are very convenient. However, you can already achieve the same things, with more control, in browsers that support fetch() streaming. I expect that every everfresh browser will support fetch() streaming before any browser supports setting headers for EventSource.

To be clear, I don't oppose improvements to EventSource. @danielwegener's arguments in https://github.com/whatwg/html/issues/2177#issuecomment-278254601 are very persuasive. However, my team cannot justify spending any resources on it.

In summary: possible? yes. Likely to happen? no.

essen commented 6 years ago

I'll +1 on wanting to set the Last-Event-ID header in the initial request. The query string can be used as a pretty simple workaround but it's unfortunate to not be able to use an existing mechanism for the initial request.

I was looking into using fetch to consume the event-stream but it seems like I would have to reimplement the parsing and automatic reconnects which is a lot more work than I'm willing to go for.

webmutation commented 6 years ago

So if I understand correctly what is being said here it that SSE and WS are old technologies that should no longer be supported and authorization is not important to use it from a browser, even though every other client library does support Authorization headers... And while there is no real cross browser support for fetch or any protocol that would enable developers to use like WS or SSE, the only option is to send what would normally go thru a header as a query string.

@ricea would it really take more than a 2 week sprint to add this to chrome? Honestly?!

LarsKemmann commented 6 years ago

Exactly. I love the promise (pun intended) of fetch and streams, but doing right by the existing standards adds customer value now for a set of use cases that are becoming increasingly important while the implementers' roadmaps can continue to (slowly) converge on stream-composable fetch-based solutions.

julienmachon commented 5 years ago

2 years later, not a single browser has full support for fetch() streaming and we still can't set headers on EventSource :man_shrugging:

chicoxyzzy commented 5 years ago

@julienmachon actually fetch streaming is supported in these browsers:

It's still would be great to use headers in SSE though.

julienmachon commented 5 years ago

@chicoxyzzy according to this https://caniuse.com/#feat=streams we only have partial support for fetch streaming but maybe it's enough for reading serve-sent event (although Firefox under a flag is a bit of a bummer)

If you have some fetch streaming examples playing nicely with SSE, I'd love to see them! :grimacing:

chicoxyzzy commented 5 years ago

@julienmachon all of browsers support ReadableStreams in streaming response body so it should be enough for most cases.

Unfortunately I don't have any examples with fetch streaming right now, I use WebSockets mostly.

eokoneyo commented 5 years ago

Hi there I'm also stumped with SSE not supporting headers after all this while, if anyone does have an example for using the fetch streaming alternative that has been suggested it'd be awesome if they drop a gist link

GirishKumar0310 commented 5 years ago

For setting authorization headers yaffle polyfill seems to be pretty good. It works on IE as well https://github.com/Yaffle/EventSource I have tested in chrome, fire and IE

You could pass in header and also control the heart beats

//See to that EventSource is tagged to Polyfill version and must for chrome and firefox else it would fall back to native eventsource which does not support headers

    var EventSource = EventSourcePolyfill; 
    const versionEventSource = new EventSource('http://localhost:8080/versions',{
      headers: {
        'Authorization': 'my jwt token'
      },
      heartbeatTimeout: 300000 //5 minutes else 45 seconds by default, it initiates another request
    });

Only cons is that in chrome EventStream windows we cannot see the messages as it is not it's native EventSource. Otherwise it is all good.

ejdaly commented 5 years ago

@maziey93 @julienmachon the Yaffle/EventSource library actually uses fetch streams internally (and falls back on xhr) - so you could just think of that library as a working implementation of using fetch streams to consume server sent events. (...an implementation that also happens to conform to the EventSource spec).

https://github.com/Yaffle/EventSource/blob/master/src/eventsource.js

@ricea

EventSource is not being actively developed. We feel resources are better spent in filling the gaps in the more generally-useful facilities provided by fetch().

Are these really conflicting interests for browser vendors, when it's likely (?) that native implementations are using Fetch internally also? Or could quite easily use fetch streams - as in the polyfill above?

EventSource could perhaps accept a Request object (optionally) in place of the URL string (i.e. the same parameters as Fetch currently accepts): https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/fetch#Parameters

(could then pass custom headers etc.. via the Request object)

eokoneyo commented 5 years ago

@ejdaly thanks for clarifying about Yaffle/EventSource. I do love the idea of being able to pass a Request object, in fact it'd be a no brainer if native implementations use fetch internally.

mmaqsood commented 5 years ago

It would also be great if there was an officially supported polyfill or package that basically replaced the browser eventsource, implementing fetch internally. Would satisfy both parties.

If the polyfill mentioned above is great, maybe validate it and put it as a suggested polyfill to use instead of the default browser eventsource.

chicoxyzzy commented 5 years ago

With new Chromium-based Edge browser release we will have SSE support in all major browsers

mmaqsood commented 5 years ago

Is the recommended approach, if using browser Eventsource, to pass authentication tokens via cookies or the url? Those two seem like the only options.

You can still protect your site from csrf by having your sse events trigger no state changes - just fetches of data, or prevent your site from being loaded by a different domain.

Toflar commented 4 years ago

Am I the only one not understanding the reasoning behind this? I mean, I understand there‘s fetch streaming but you still need to implement reading the standardized SSE response yourself in JavaScript. It‘s slower and there are no native debugging tools contrary to EventSource. So we have a standard but should implement the browser side ourselves because of not being able to set headers? Although EventSource is 95% there, native, stable and fast? I‘m slightly confused :)

eokoneyo commented 4 years ago

@Toflar I really do understand your frustrations, I’ve had to abandon my use case for SSE because although the polyfill for EventSource works it feels obtrusive unlike the native implementation. This issue has been tagged needs implementer interest I think that’s what we should focus on if we want to land header support for EventSource

UnKnoWn-Consortium commented 4 years ago

@Toflar I have tried to implement an EventSource-like consumer with Fetch. Actually the API (ReadableStream and TextDecoder) is still native. They have just split it into individual components.

The only issue is with browser support. Firefox still needs users to manually enable support for ReadableStream for Fetch.

dudo commented 4 years ago

I'm confused that EventSource api is considered legacy... almost 4 years ago?! So if we can't get header support, can the streaming fetch at least show up in the network tab nicely?

alevidcosta commented 4 years ago

I went through quite a few posts in an attempt to figuring out if the auth token be sent in the EventSource() call. Although there are polyfill alternatives that allow adding headers while others mentioned sending auth token over ssl.

Instead of using polyfill EventSource() or sending the auth token in the query params over ssl, send a eventSource identifier (eventSrcUUID) over ssl in the params of EventSource url as follows :-

On user authentication, eventSrcUUID is generated along with the sseEmitter on the server and place in a sseEmitterMap. The sseEmitter is also placed alongside user session data and the eventSrcUUID is sent to the client along with user auth info.

Client retrieves the eventSrcUUID from the response and invokes the EventSource() call with the eventSrcUUID in the param. On the server, the sseEmitterMap is referenced to retrieve the the eventSrc object and the entry is removed from the sseEmitterMap to disallow further invocations. The sseEmitter object saved in session data is used to send event notifications to client.

PrasannaVenkadesh commented 4 years ago

I read through the above discussions. If EventSource is HTTP, I couldn't imagine a HTTP request without the ability to add headers. It's quite natural to think of headers when we speak HTTP.

@Toflar I really do understand your frustrations, I’ve had to abandon my use case for SSE because although the polyfill for EventSource works it feels obtrusive unlike the native implementation. This issue has been tagged needs implementer interest I think that’s what we should focus on if we want to land header support for EventSource

@eokoneyo How do I express my interest for it? Is this the place or is there some other place for expressing our interest?

Toflar commented 4 years ago

I just want to get one thing straight here: I'm not "frustrated" :) I was confused because being able to send meta data (aka headers) along with HTTP requests seems like the most normal thing to do for me. But I've been in the (OSS) developer community for far too long to be frustrated with things. It's just never been implemented by anyone so if I need it, it's also my duty to implement/fix things. Simple :) But I'd be interested in the meaning of "needs implementer interest" as well.

danielwegener commented 4 years ago

From what I understood OAuth 2.1 will forbid the usage of bearer tokens in query params. In OAuth 2 it is "somewhat" allowed:

SHOULD NOT be used unless it is impossible to transport the access token in the "Authorization" request header field or the HTTP request entity-body rfc6750#section-2.3

(which sounds like a case for SSE) but draft-ietf-oauth-security-topics-15#section-4.3.2 seems to aim to forbid it. As a result, SSE can likely not be used with OAuth2.1 bearer token authorization because it is impossible to transport the bearer access token (I think this counts as "frustrated" :) ).

kalgon commented 3 years ago

One point which has not been addressed in this long thread is how a OAuth token passed in the Authorization header would be refreshed when it expires (and how/when does the EventSource detect it has expired)?

ericlaw1979 commented 3 years ago

I don't think there's any need to specify that at this level; you could either have the server send a message that cause the client to proactively re-establish the connection with a refreshed token, or you could have the server close the connection and have the client reestablish it with the latest token if the old one expired. https://stackoverflow.com/questions/24564030/is-an-eventsource-sse-supposed-to-try-to-reconnect-indefinitely

vanjad-sixt commented 2 years ago

Is it really necessary to terminate the connection when the token expires? The request is authorized by the server upon receiving and this (already authorized) connection is then kept permanently open while transferring the response (in chunks).

Comparing to non-streamed HTTP requests: Given that you use a token with 30s expiry time and the client issues a POST request When the server grants authorization but takes 60s to generate the response Do you expect the server to respond with unauthorized?

ultimaweapon commented 2 years ago

I just noticed the existence of SSE and found out it is what I'm looking for to notify the client without introduce WebSocket into my server. Then this issue broke my heart due to I'm using OIDC.

kuriel-trivu commented 2 years ago

any update?

tomuench commented 1 year ago

Well, one year later @ultimaweapon.

I stumble over the same problem with OIDC. It would be a simple solution, if developers could set the header by themselves.

I found a possible workaround on stackoverflow: https://stackoverflow.com/questions/28176933/http-authorization-header-in-eventsource-server-sent-events

Maybe, WebTransport will be an appropriate candidate to build our use case. See https://w3c.github.io/webtransport

rodorgas commented 1 year ago

I'd like to share this polyfill that uses fetch streaming instead of XHR polling: https://github.com/Azure/fetch-event-source

We can easily set headers with this polyfill.

hperrin commented 1 year ago

Thank you @rodorgas for the solution.

It seems to me like this should be considered a known security vulnerability, since it forces developers to put auth tokens in the URL and URLs (even in HTTPS) are vulnerable. Even without an attack like that, URLs are often stored in log files, meaning auth tokens would be stored there too.

I think the solution here is to either add a custom header feature or just fully deprecate and eventually remove EventSource altogether. From a security standpoint, leaving it unchanged seems like a worse approach.

(Feel free to correct me if I'm misunderstanding anything here.)

ericlaw1979 commented 1 year ago

it forces developers to put auth tokens in the URL and [URLs (even in HTTPS) are vulnerable] (https://www.securityweek.com/hackers-can-intercept-https-urls-proxy-attacks/).

While I'm inclined to agree that putting auth tokens in URLs is distasteful on several levels, I'm compelled to note that that particular issue was patched[1] a month before the article was even written seven years ago.

[1] https://bugs.chromium.org/p/chromium/issues/detail?id=593759#c72

thanhnt-fabbi commented 1 year ago

I find SSE to be a very standard, nice and simple way to send events to the browser. Then I found it doesn't support set header. I was surprised by that.

drocha87 commented 1 year ago

I share the felling of @ultimaweapon. When I discovered EventSource and realized that it works over HTTP, the first thing that came to mind was the ability to set headers, as it feels like a natural expectation. However, this issue has been open for approximately seven years, so I assume it won't be implemented anytime soon, which is quite disappointing.

herasimau commented 1 year ago

It's 2023, and it's really frustrating that SSE still doesn't support custom headers. With all the tech updates we've seen, it's surprising this hasn't been fixed. Developers have been asking for it for years. You'd think they'd have listened by now, but nope, we're still waiting.

stephen-dirtcreative commented 1 year ago

The seemingly prevailing attitude that "this new API can do all these things if you take time to implement them, so these older APIs we created (to do 99% of the implementation) can go pound sand" is disheartening, and does nothing to promote trust or interest in using any API new or old. I could implement on this new API, but what's that down the road? The next shiny thing is just over the horizon? It may take 5+ years to be usable, and have a totally different interface, but we're all going to mark the current API legacy and divert all resources to the new stuff? Does the WHATWG need to start an LTS requirement for their API proposals so at least we have some idea how to gauge the utility of adopting an API if it's just going to be staleware in x months / years?

If Fetch is the way, can't the EventSource API be rebased onto Fetch, making it continue to be useful and relevant? Just a thought.

atambi802 commented 1 year ago

Hello everyone! Just my 2 cents for this, I have an api, that is an SSE. It just spits data over local network and I just want people who have my bearer token to use it only, or else it is just open to anyone. I understand it isn't the most important thing, but adding some support of headers to the object would be so much better then making a websocket, and having to deal with the whole workflow with that.