Implement EventSource support

[gavv]

This issue depends on two other issues:

• 244

• 245

UPDATE: both of them are completed now.

---

Add support for EventSource, a.k.a. Event Stream, a.k.a. SSE (Server-Sent Events).

It was requested a while ago here: https://github.com/gavv/httpexpect/discussions/145

We already have support for websockets, and support for SSE will have similar design, but simpler:

• Request will have new method WithEventSource (similar to Request.WithWebsocketUpgrade). It should just set Accept header field to text/event-stream (it should use withHeader method).

• Response will have new method EventSource (similar to Response.Websocket). It should check that response Content-Type header is text/event-stream and create a new instance of EventSource struct, attached to response body (it should use checkContentType method).

• EventSource method should also check if response body is bodyWrapper. If it is, it should call bodyWrapper.DisableRewinds, to prevent bodyWrapper from trying to save the whole (likely infinite) response in memory.

• EventSource struct (similar to Websocket struct) represents SSE connection. It should have methods: Alias, WithReadTimeout (set timeout for next read), WithoutReadTimeout (disable timeout for next read), Expect (read next message), Disconnect (close connection).

• EventSource.Expect should read next event from response body and return a new instance of Event.

• Event struct (similar to WebsocketMessage struct) represents a single event retrieved from server. It should have methods: Raw, Alias, ID (returns String), Name (returns String), Data (returns String).

---

Decoding of SSE events can be implemented using this package: https://github.com/alevinval/sse. It provides Decoder struct, which can read events from http.Response body.

New methods and structs should be covered with unit tests. In addition, we need a new e2e (end to end) test, similar to e2e_websocket_test.go.

We also need a new example similar to _examples/websocket.go. To make sure that our SSE implementation is compatible with spec, it is suggested to use another package for SSE server in that example. A good candidate is https://github.com/r3labs/sse.

---

Wiki: https://en.wikipedia.org/wiki/Server-sent_events Spec: https://html.spec.whatwg.org/multipage/server-sent-events.html

[g41797]

As far as I understand it's not urgent issue I just started my open source journey and pretty sure, that this feature will take some time Pros: no one depend on additional question - according to .gitignore, goland is used as ide i am working with vscode

"Waiting for reply" :-)

[gavv]

You're welcome!

[g41797]

for further reading: https://en.wikipedia.org/wiki/Server-sent_events

https://blog.axway.com/learning-center/apis/api-streaming/server-sent-events Event: The event’s type. It will allow you to use the same stream for different content. A client can decide to “listen” only to one type of event or to interpret differently each event type.[Additional param?] Retry: The time to use before the browser attempts a new connection ...[???]

http://html5doctor.com/server-sent-events/

https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events

• message [unnamed] events OnMessage
• custom [named] events CustomHandler [addCutomListener]

https://html.spec.whatwg.org/multipage/server-sent-events.html

source = new EventSource( url [, { withCredentials: true } ])

withCredentials - check reqs.

Last-Event-ID header - check usage

====>

https://shopify.engineering/server-sent-events-data-streaming https://meronhayle.me/implementing-sse-using-go/ https://thedevelopercafe.com/articles/server-sent-events-in-go-595ae2740c7a

https://bigboxcode.com/go-server-sent-events-sse https://copyprogramming.com/howto/server-sent-events-golang https://www.timeplus.com/post/websocket-vs-sse

https://www.reddit.com/r/golang/comments/pm71iq/fully_featured_speccompliant_html5_serversent/ https://go-gin.onrender.com/room/hn

https://docs.servicestack.net/server-events

https://github.com/grciuta/sse-golang-example

[g41797]

go web servers for dummies: https://www.dudley.codes/posts https://pace.dev/blog/2018/05/09/how-I-write-http-services-after-eight-years.html https://www.alexedwards.net/blog/the-fat-service-pattern

--> https://threedots.tech/post/common-anti-patterns-in-go-web-applications/

https://medium.com/nerd-for-tech/golang-build-a-simple-web-server-and-interact-with-it-689ee0f4d1de https://chidiwilliams.com/post/writing-cleaner-go-web-servers/ https://drstearns.github.io/tutorials/gomiddleware/ https://www.geeksforgeeks.org/how-to-build-a-simple-web-server-with-golang/ https://thecodeway.hashnode.dev/building-a-lightweight-web-server-with-golang https://html.spec.whatwg.org/multipage/server-sent-events.html

---

https://www.openapis.org idl

[g41797]

                    EventSource vs SSEClient

Usage of HTML5 standard interface name causes confusion

interface EventSource : EventTarget { constructor(USVString url, optional EventSourceInit eventSourceInitDict = {});

readonly attribute USVString url; readonly attribute boolean withCredentials;

// ready state const unsigned short CONNECTING = 0; const unsigned short OPEN = 1; const unsigned short CLOSED = 2; readonly attribute unsigned short readyState;

// networking attribute EventHandler onopen; attribute EventHandler onmessage; attribute EventHandler onerror; undefined close(); };

dictionary EventSourceInit { boolean withCredentials = false; };

SSEClient struct represents SSE connection. It should have methods:

• Alias ???
• WithReadTimeout
• WithoutReadTimeout
• Expect (read next event)
• Disconnect (close connection).

EventSource should be interface - specific implementation depends on used SSE/HHTP package. This approach allows to use different client/server SSE/HTTP implementations.

For websocket: we use gorilla/websocket internally

For SSE: Internal implementation of EventSource based on https://github.com/r3labs/sse It may be replaced with custom EventSource

Response will have new method SSEClient etc....

httpexpect structs ===> SSEClient - - - > EventSource (visible only for SSEClient)

[g41797]

   Usage of 3rd party SSE  vs own implementation

3rd party (e.g. https://github.com/r3labs/sse)

1. • shorter dev. time [+]
2. • more stable code - supported by community [+]
3. • lack access to http header etc. for testing of low-level info [-]

Need clarification what's E2E testing for SEE:

• validation of low level info [infra level] + events content/order [app layer]
• events content/order [app layer]

For app layer usage of 3rd party is preferable choice

[g41797]

             From github.com/r3labs/sse/v2

type Event struct { timestamp time.Time ID []byte Data []byte Event []byte Retry []byte Comment []byte }

From HTTP5 : Event handler	Event
onopen	"open"
onmessage	"message"
onerror	"error"

Check events supported by sse

[g41797]

Regarding wsUpgrade bool (see WithWebsocketUpgrade)

Should be replaced with something like:

type clientType int

const ( typeRegular clientType = iota // HTTP client typeWebSocket // WebSocket typeEventSource // EventSource (Server Sent Event) )

Consider to add this field also to responseOpts (used for creation of response)

[g41797]

What about shortcut:

Request will have new method EventSource() (similar to Response.Websocket). It should check TBD , create a new instance of EventSource struct.

EventSource struct (similar to Websocket struct) represents SSE connection. It should have methods:

• Alias,
• WithReadTimeout (set timeout for next read),
• WithoutReadTimeout (disable timeout for next read),
• Expect (read next event from [3rd party/own impl.] and return a new instance of Event),
• Disconnect (close connection),

[Note: How to retrieve unknown in advance number of events without test failure?]

Event struct (similar to WebsocketMessage struct) represents a single event retrieved from server. It should have methods:

• Raw,
• Alias,
• ID (returns String),
• Name (returns String),
• Data (returns String).

Looks closer to client side flow

I have no problem to implement former design, but before implementation it worth to discuss alternatives

Waiting for reply

[g41797]

Headers:

• "Cache-Control", "no-cache"
• "Accept", "text/event-stream"
• "Connection", "keep-alive"

According to https://html.spec.whatwg.org/multipage/server-sent-events.html: Set request's cache mode to "no-store". new issue #https://github.com/r3labs/sse/issues/159

Set request's initiator type to "other"

Request.WithEventSource() - validate flow: assert for already called WithEventSource()/WithWebsocketUpgrade()

EventSource methods: called by Request func eventSourceTransform(r *Request) { for k, v := range eventSourceRequestHeaders() { r.withHeader(k, v) } }

called by Response func eventSourceMatch(r *Response) { r.HasContentType("text/event-stream", "UTF-8") r.Status(200) }

[g41797]

Decoding of SSE events can be implemented using this package: https://github.com/alevinval/sse. It provides Decoder struct, which can read events from http.Response body.

Installation of alevinval/sse - failed Open new issue https://github.com/alevinval/sse/issues/61

Meantime 3 sources from sse were copied to local repo for further development

[g41797]

Additionally to Expect EventSource will have low-level Read function:

func (es EventSource) Read() (ev Event, err error, timeOut bool){ ...... }

Chans+timeout: https://golangbyexample.com/select-statement-with-timeout-go/ https://stackoverflow.com/questions/49872097/idiomatic-way-for-reading-from-the-channel-for-a-certain-time

Timers: https://gobyexample.com/timers https://stackoverflow.com/questions/50223771/how-to-stop-a-timer-correctly

https://stackoverflow.com/questions/8593645/is-it-ok-to-leave-a-channel-open

[g41797]

looks this feature has low priority