fission / fission-workflows

Workflows for Fission: Fast, reliable and lightweight function composition for serverless functions
Apache License 2.0
361 stars 40 forks source link

input and output data formats for fission functions #223

Closed tfahringer closed 5 years ago

tfahringer commented 5 years ago

Is there somewhere a good description of ALL input and output data formats for fission functions? I understand Fission functions can only be invoked based on events. If this happens, what kind of input data can be passed to functions (I assume by value only), what formats?Is the input data encoded in the event or is it separate data?I understand data streams can be input into Fission functions. Is any format accepted such as JSON, CSV, AVRO, binary and text format?Can I also input vidoes, images, data logs to Fission functions.I cannot find an single document for Fission that describes all options for input and output data for fission functions.Thanks for any hint

erwinvaneyk commented 5 years ago

Hi @tfahringer - Fission functions themselves are agnostic of the data encoded in the body of the HTTP request.

For Workflows, there is a distinction between unstructured (blob) data and structured data. Structured data is parsed, which makes it query-able in other tasks. This allows you to use expressions to reference subsets of the data { output('TaskA').myField } Unstructured data is passed from outputs to inputs uninterpreted, as raw data. You cannot use expressions on it.

Upon receiving the response of a Fission function, the workflows checks the content-type whether it is structured data. The following formats are supported:

Indeed, this should be better documented! 🙂