search

tremor-rs / tremor-www-docs

Tremor https://docs.tremor.rs site
Apache License 2.0
24 stars 22 forks source link

Make a clear distrinction between streaming and package based connectors #167

Open Licenser opened 3 years ago

Licenser commented 3 years ago

Describe the problem you are trying to solve

Some of our connectors are package based where one chunk of data == one event, udp, kafka, HTTP, WS are some examples.

Others are stream based where there is no alignment to event boundaries, TCP and file are examples. Those usually require a preprocessor and postprocessor to add framing.

We do not make it clear in the documentation which connector behaves which way and has which guarantees, that can lead to confusion.

We should be clearer about this.

afzal442 commented 3 years ago

Hi @Licenser, I want to look into it!

skoech commented 3 years ago

Hello @afzal442! Welcome to Tremor! We are happy you want to contribute. How would you like to go about this?

afzal442 commented 3 years ago

Thanks @skoech! I wondered which doc's part for this issue needs to be fixed.

skoech commented 3 years ago

We would need to update the docs of all the connectors (onramps/offramps).

skoech commented 3 years ago

So, TCP is stream-based while the rest are all packet-based.

afzal442 commented 3 years ago

Makes sense. I found those links as https://github.com/tremor-rs/tremor-www-docs/blob/main/docs/artefacts/offramps&onramps.md . Any next suggestions?

skoech commented 3 years ago

Yes, good. :) Then maybe create your own fork to work on, if you haven't yet?

afzal442 commented 3 years ago

Hy @skoech, your discord handle plz?

skoech commented 3 years ago

I'm shay#4695. I already reached out to you. Sorry I didn't mention it was me. 😀