scramjetorg / transform-hub

Scramjet Transform Hub (STH) is a runtime supervisor that can run data processing programs called Sequences and manage local resources on any Linux server, Docker on small edge servers, and even large-scale Kubernetes clusters in the cloud or datacenters. It connects to Scramjet Spaces in Scramjet Cloud Platform.
GNU Affero General Public License v3.0
67 stars 8 forks source link

Send 100 Continue when ready to receive topic downstream #1010

Closed patuwwy closed 10 months ago

patuwwy commented 11 months ago

What?

:large_blue_diamond: Disable auto sending HTTP 100 Continue

NodeJs auto sends HTTP 100 Continue when it is not strictly handled on server. We handle it in other way and send when needed so it is sent twice.

https://nodejs.org/api/http.html#event-checkcontinue _https://github.com/nodejs/node/blob/v18.x-staging/lib/_http_server.js#L1110_

:large_blue_diamond: Topic downstream queue update

:large_blue_diamond: Refactoring

Move TopicRouter / ServiceDiscovery types to @scramjet/types


:heavy_exclamation_mark: User should wait for HTTP 100 Continue and then start to upload data.

:heavy_exclamation_mark: @scramjet/cli / @scramjet/api-client CAN'T listen for 100 Continue as we use node-fetch which doesn't support continue request's event. Data will be sent immediately.

If data are sent with no wait for 100 Continue some bytes can be lost when request is interrupted before we started handling these data.


Review checks:

These aspects need to be checked by the reviewer:

a-tylenda commented 10 months ago

verified

image

a-tylenda commented 10 months ago

image