search

airyhq / airy

💬 Open Source App Framework to build streaming apps with real-time data - 💎 Build real-time data pipelines and make real-time data universally accessible - 🤖 Join historical and real-time data in the stream to create smarter ML and AI applications. - ⚡ Standardize complex data ingestion and stream data to apps with pre-built connectors
https://airy.co/docs/core
Apache License 2.0
373 stars 44 forks source link

Document how source content works #1075

Open chrismatix opened 3 years ago

chrismatix commented 3 years ago

Current state

We have a "as transparent as possible" approach to dealing with source messages and event payloads, which we acquired through years of operating a multi-source streaming messaging system. This is not reflected in the docs

Desired state Whenever there is a source content field in the APIs there should be a document page that we can point to that explains what kind of values users can expect on this field.

lucapette commented 3 years ago

I think I would add a section in this document: https://airy.co/docs/core/sources/introduction called "Content". We explain we have no opinion about the content of the messages and people should refer to the docs of the source provider to construct valid payloads (as we also do not validate them)

chrismatix commented 3 years ago

I think the idea for the content is good, but "concepts" seems an even better section because this topic encapsulates unique learnings we made. So we could have it at concepts/source-content and link that in the sources introduction.

lucapette commented 3 years ago

Agreed. Loved the suggestion!